Run Ollama on a per-minute GPU, then point your app at an OpenAI-compatible API

Your laptop can't hold the model you want, and you don't want to spend an afternoon on CUDA and port forwarding. This walkthrough rents a GPU billed per minute, runs Ollama on it with a public HTTPS endpoint, and calls that endpoint with curl and the OpenAI Python client.

Disclosure: I work on CloudGPU.

What you need

• A CloudGPU account (email or Google sign-up) with a prepaid balance you can top up by card or USDT. Rates are listed on the pricing page.
• Python with pip install openai.

Step 1: Deploy the Ollama template

Open Deploy, choose the Ollama template, select a GPU card with enough VRAM for the model you plan to run, and click Deploy. The Deploy page shows the cards you can pick. To size a model, see our Ollama GPU guide.

Step 2: Find the API endpoint

The docs say: "Every running Ollama instance gives you an OpenAI-compatible endpoint. Open the instance in your Dashboard and click the API button." They also say "Replace with the id shown on your instance row." The endpoint looks like this:

https://<random-id>-i.cloudgpu.app

Check the docs for the current wording. The URL works only while the instance runs. The terms say service URLs are "not authenticated by default" and that you "should treat them as a credential and not share them publicly", so keep it private. If you need authentication, configure it inside your instance.

Step 3: Pull a model

The docs describe a Pick Model button for pulling any model from the Ollama library after deploy. As an example, this guide uses llama3.1:8b (Llama 3.1 8B on ollama.com). Check it fits the VRAM of the card you picked. You can also run ollama pull llama3.1:8b yourself if you connect over SSH.

Step 4: Smoke-test it with curl

List the models on the instance:

curl https://<random-id>-i.cloudgpu.app/api/tags

Send one chat turn (non-streaming). keep_alive is the Ollama setting that controls how long a model stays loaded after the last call; the CloudGPU docs example uses "30m".

curl https://<random-id>-i.cloudgpu.app/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "stream": false,
    "keep_alive": "30m",
    "messages": [{"role": "user", "content": "Say hello in five words."}]
  }'

If a model has been unloaded, Ollama has to load it again before it can answer.

Step 5: Call it with the OpenAI Python client

Ollama serves an OpenAI-compatible API under /v1 (see Ollama's OpenAI compatibility docs), so the openai package works with a different base_url:

from openai import OpenAI

client = OpenAI(
    base_url="https://<random-id>-i.cloudgpu.app/v1",
    api_key="ollama",  # any non-empty string, per the CloudGPU docs
)

r = client.chat.completions.create(
    model="llama3.1:8b",  # a name from /api/tags
    messages=[{"role": "user", "content": "Explain LoRA in one sentence."}],
)
print(r.choices[0].message.content)

Anything else that accepts an OpenAI-compatible base URL can use the same endpoint and model name.

Want ComfyUI instead?

The Deploy page also lists a ComfyUI template. The docs say templates that serve an API (Ollama and ComfyUI) get a public HTTPS URL, and that the ComfyUI template installs ComfyUI only: it does not include FLUX or other model weights, so you download the weights you need yourself once the instance is running. The same advice about keeping the URL private applies.

Destroy the instance

When you're done, destroy the instance. GPU time is billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance. The rules are in the terms. Destroying an instance deletes everything stored on it, so copy out anything you need first. The terms also say an instance is destroyed automatically if your balance reaches zero.

Troubleshooting

• Model not found or wrong name. Use a name exactly as /api/tags returns it.
• The endpoint does not answer. Check in your Dashboard that the instance is still running and that you copied the id from the instance row. A destroyed instance no longer serves its URL.
• The first request after a model was unloaded fails or times out. Ollama may still be loading it into VRAM; retry.
• No API button or endpoint on the instance. Send the instance ID through the contact page.

CloudGPU is in public beta with no SLA.

Prefer a hosted model to your own GPU? The OpenAI-compatible gateway is billed per token from the same prepaid balance and is documented in the API docs.