Our recent AI collaboration was a masterclass in local environment optimization and agentic tool refactoring! Here’s a breakdown of what we accomplished:
### 1. Refactoring Open WebUI Tools to FastMCP
We started by investigating a `tools.py` script that was originally generated (via aider) for Open WebUI to interface with WordPress. After clarifying that Open WebUI manages tools within its internal database rather than the local filesystem, we decided to pivot. We refactored the script into a standalone **FastMCP (Model Context Protocol)** server. This architecture forms the backbone of modern, on-premise sovereign computing environments—such as the Sovereign AI & Private Cloud Infrastructure we engineer for Southern California clients. This upgrade means the WordPress tools can now be universally accessed via STDIO by any compatible agent, like Claude Code or Claude Desktop.
### 2. Optimizing the Local LLM Proxy
Checking the `litellm_config.yaml` setup revealed that the environment was proxying requests to an older `qwen2.5-coder:7b` model. Since a fresh, much more capable `14b` parameter version had just been downloaded, we immediately updated the LiteLLM proxy to point to the new `qwen2.5-coder:14b` model to ensure top-tier coding performance.
### 3. Deep Cleaning and Consolidating
We ran a deep clean on the local Docker and Ollama environments:
– Consolidated redundant model tags in Ollama (`deepseek-coder-v2:lite` and `qwen2.5-coder:7b-16k`) to declutter the model list.
– Uninstalled the now-deprecated 7B Qwen model since the proxy was successfully upgraded to 14B.
– Performed a full Docker prune, stripping out massive unused images like `zaproxy`.
### 4. Why FastMCP Saves Tokens (and Time!)
An interesting takeaway from moving tools to an MCP server is how highly token-efficient this architecture is for local LLMs:
– **No Implementation Details in Context:** The actual Python code (like DB credentials, requests logic, and WP CLI execution) is kept entirely on the MCP server. It is never sent to the LLM’s context window.
– **Tiny Prompt Footprint:** Instead of generating or reading complex scripts, the LLM simply receives a highly compressed JSON signature of the tools available.
– **Less Output Generation:** When the LLM decides to take action, it doesn’t need to burn tokens writing out long `curl` commands. It just generates a tiny JSON payload to trigger the tool, which makes responses much faster.
**The result?** A highly optimized local LLM ecosystem, a robust FastMCP server for WordPress management, massive token savings, and over 6 GB of reclaimed disk space. A very productive session!
—
### Deploy Sovereign AI in Southern California
Looking to deploy private, on-premise LLMs, air-gapped agentic tools, or high-performance FastMCP servers without leaking intellectual property to third-party cloud APIs? At **OC Computer Fix**, we engineer, deploy, and harden dedicated sovereign AI hardware across **Orange County, Los Angeles, and San Diego Counties**. Explore our [Sovereign AI Infrastructure Engineering](https://occomputerfix.com/#advanced) solutions or reserve a seat in our hands-on weekend [Local AI & Hardware Masterclasses](https://occomputerfix.com/#masterclasses) to build your private AI cluster.
Leave a Reply