Model HQ by LLMWARE runs powerful AI Agent workflows on your device or private infrastructure with 250+ models, no cloud dependency, no token costs, and no coding required.
For a more detailed overview, check out LLMWare's blog post introducing Model HQ.
Integrating Microsoft Foundry Local into Model HQ gives users the ability to utilize these models completely on their own computers, allowing for offline working and keeping sensitive data private.
Foundry Local is an end-to-end local AI solution for shipping applications that run entirely on the user's device. It provides an easy-to-use SDK (C#, JavaScript, Rust, and Python), a curated catalog of optimized models, and automatic hardware acceleration—all in a lightweight package.
To learn more, check out Microsoft's article on Foundry Local.
1. Installing and Integrating Model HQ
First, download Model HQ from llmware.ai/checkout. We offer different versions for AMD, Apple, Intel, NVIDIA, and Qualcomm processors.
After downloading Model HQ, download Foundry Local from terminal if you have not already installed it.
For AMD, Intel, NVIDIA, and Qualcomm:
winget install Microsoft.FoundryLocal
For Apple:
brew install foundry-local
Head to the "Integrations" page. Click "Test" to try out Microsoft Foundry Local. You should see a message bar with a successful test outcome.
If the test is unsuccessful, try reinstalling Foundry Local. If the issue persists, please contact Microsoft for additional support.
3. Downloading models
Head to the "Models" page, click "All", and you'll see a dropdown menu of models to choose from, including Foundry models.
Select the desired model from the dropdown menu and click download button. The download should take a while (around 2-3 minutes depending on your internet speed).
3. Testing models
Now, we can test the downloaded model.
3.1 Testing Foundry Local models using Model HQ's Chatbot
One way to do so is using our Chatbot. Head over to the "Chat" section of Model HQ, then click "Model" to select the Foundry model you want to test.
Hit the next arrow, and like before, another successful loading message box should appear. If the load was unsuccessful, try redownloading the Foundry model.
As an example, I chose the qwen2.5-1.5b-instruct-qnn-npu:2-foundry model and asked "Where is Paris".
Here is the response I received:
3.2 Testing Foundry Local models using Model HQ's agents
A second test you can perform is in the "Agents" page of Model HQ - Agents are able to complete multi-step tasks and make decisions autonomously to accomplish a goal.
I have selected the Contract Analyzer agent.
Click next to see a flow of multiple nodes containing contract-related questions the agent will answer. Many are RAG (Retrieval Augmented Generation) questions, meaning the agent will retrieve information from the input document, likely a contract, to answer the question.
To switch the RAG model to the one you want to test, select your chosen model from the dropdown menu labeled "RAG Model". You can also switch the Chat model the same way.
I have chosen the phi-3-mini-4k-instruct-qnn-npu:3-foundry model, which is a model that runs on NPU (Neural Processing Unit).
Once the model is selected, click next, then click "Run (Demo)".
Quick tip:
For Windows users, you can view your GPU, CPU, and NPU usage as the model does its work by opening Task Manager and heading to the Performance tab. Since I chose a model that runs on NPU, I can see my NPU in use as the model answers the RAG questions.
For Mac users, open Activity Monitor to monitor CPU and system resource usage.
Here is the output, along with my computer's NPU in use.
What's next?
We have seen how to integrate Microsoft Foundry Local into LLMWare's Model HQ, and we have tested these Foundry models using LLMWare's Chatbot and Contract Analyzer agent.
For another example workflow on using Foundry Local models, see LLMWare's Qualcomm Developers Blog.
Email us with the subject line "Request for Model HQ Free Trial".
Top comments (0)