B. Hugging Face & Local Models

📄 File 2: docs/appendix-huggingface-reference.md

# Appendix B: Hugging Face & Local Model Reference

This appendix explains how to use Hugging Face as a sustainable, version-controlled "Model Registry" for Tier 3 (Sovereign/Local) AI execution.

## 1. Why Hugging Face?
Think of Hugging Face as the **Docker Hub of AI**. Instead of relying on proprietary, black-box APIs, you can pull verified, open-weight models directly to your machine, ensuring zero data egress and $0 token costs.

## 2. Installation & Authentication

```bash
# 1. Install the Hugging Face Hub CLI
pip install -U "huggingface_hub[cli]"

# 2. Authenticate (You will need a free Hugging Face account and an Access Token with "Read" permissions)
huggingface-cli login
# Paste your token when prompted

3. Downloading a Model (The docker pull equivalent)

Instead of streaming from an API, download the model weights to your local machine.

# Download Qwen 2.5 Coder 7B Instruct (optimized for DevOps tasks)
huggingface-cli download Qwen/Qwen2.5-Coder-7B-Instruct

# Download Llama 3 (alternative)
huggingface-cli download meta-llama/Meta-Llama-3-8B-Instruct

Note: Ensure you have sufficient disk space (a 7B model is ~5-8GB).

4. Serving the Model Locally

Once downloaded, you need a local inference engine to serve it. We recommend Ollama for simplicity.

# 1. Install Ollama (https://ollama.com)
# 2. Pull the model into Ollama (it will use the Hugging Face weights if configured, or pull its own optimized version)
ollama pull qwen2.5-coder:7b

# 3. Start the local server (runs in the background)
ollama run qwen2.5-coder:7b

5. Integrating Local Models with Aider

Once your local model is running, you can instruct Aider to use it. This is mandatory for NV1/government work or handling sensitive credentials.

# Start Aider and force it to use the local Ollama endpoint
aider --model ollama/qwen2.5-coder:7b

# Or, add it to your .aider.conf.yml for persistent use:
# model: ollama/qwen2.5-coder:7b

6. Security & Sovereignty Checklist

• [x] Model weights are stored on my local machine or approved corporate storage.
• [x] No prompts or code are sent to third-party cloud APIs.
• [x] The local environment is isolated from the public internet (if air-gapped).

---

🔗 Return to Chapter 2: Token Economics & The 3-Tier Strategy ```

---