DEV Community

Florian Lutz
Florian Lutz

Posted on AI-assisted

Bring your own Key: Connecting VS Code to your private Inference Platform

Introduction

Running your own inference platform is only half the fun if nobody can use it from their editor. With Bring your own Key (BYOK), VS Code lets you plug any OpenAI-compatible endpoint, like a LiteLLM gateway, directly into the Copilot Chat. This post shows the setup in a few steps.

Info: The screenshots were taken with VS Code version 1.147.0. The UI might look slightly different in other versions.

Step 1: Open the Chat and add a model

After starting VS Code, the Chat view is shown on the right. In the box "Choose how you want to use Copilot" you can either sign in to use the GitHub Copilot models or add your own. Click on Add Models.

VS Code welcome screen with the Chat view and the Add Models button

Hint: If you don't see the Copilot Chat, press Ctrl + Shift + I (German keyboard: Strg + Shift + I) to open it.

Step 2: Choose a provider

The Language Models dialog opens. Click on Add Models again and you get a list of supported providers like Anthropic, Azure, Google or OpenAI. For a self-hosted platform select Custom Endpoint.

Language Models dialog with the provider list and Custom Endpoint selected

Step 3: Name the group

Next, VS Code asks for a Group Name. This is just the display name under which your models are grouped, e.g. Private AI Platform. Confirm with Enter.

Group Name input with the value Private AI Platform

Step 4: Enter the API key

Now enter the API key for your endpoint, e.g. the LiteLLM virtual key you created for your user, and confirm with Enter. The key is stored as a secret by VS Code and is not written to disk in plain text.

API Key input for the Private AI Platform

Step 5: Configure the models

Finally VS Code opens the chatLanguageModels.json file in your user profile. The provider entry with the name, vendor customendpoint and a reference to the stored secret is already there. You only have to fill in the model details:

chatLanguageModels.json with the generated provider entry

[
    {
        "name": "Private AI Platform",
        "vendor": "customendpoint",
        "apiKey": "${input:chat.lm.secret.c45e343}",
        "apiType": "chat-completions",
        "models": [
            {
                "id": "<model id as exposed by your endpoint>",
                "name": "<display name in VS Code>",
                "url": "https://<your-endpoint>/v1/chat/completions",
                "toolCalling": true,
                "vision": true,
                "maxInputTokens": 128000,
                "maxOutputTokens": 16000
            }
        ]
    }
]
Enter fullscreen mode Exit fullscreen mode
  • id: the model name your endpoint expects in the request
  • name: how the model is shown in the model picker
  • url: the endpoint of your platform
  • toolCalling / vision: set according to what your model supports. Tool calling is required for Agent mode
  • maxInputTokens / maxOutputTokens: should match the context window of your deployment

After saving the file, the model shows up in the model picker of the Chat and is ready to use.

Conclusion

With just a few clicks and a small JSON file, VS Code talks to your own inference platform. Your code and prompts never leave your infrastructure, and you can still use the full Copilot Chat experience.

Top comments (0)