DEV Community

Cover image for From adapter to deployment: merging LoRA weights and serving with vLLM or a Space
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

From adapter to deployment: merging LoRA weights and serving with vLLM or a Space

#finetuning #deployment #vllm #serving

A fine-tuned adapter sitting in a folder is not a product. FineTune Studio offers three inference paths — local, a vLLM server, or a Hugging Face Space — because the right one depends on who will call the model and how often. Here is how I choose, and the checks I run before calling it deployed.

Keep the adapter or merge it?

Keeping the adapter separate lets you hot-swap behaviours on one base model and keeps files small. Merging bakes the adapter into the base weights, producing a single model that any serving stack can load without adapter support. Merge when you have one behaviour and want simplicity; keep adapters when you serve several tasks from one base.

Three serving paths

  • Local inference — the training machine loads the model and answers requests. Right for demos, evaluation and internal tools; wrong for anything with real traffic.
  • vLLM server — batched, fast, OpenAI-compatible API. Right for production on a GPU you control. Supports LoRA adapters directly, so merging is optional.
  • Hosted Space — a public or gated endpoint for sharing a demo. Right for portfolios and hackathon judges; be mindful of cold starts and free-tier limits.

Checks before you call it shipped

  1. The served model reproduces your evaluation outputs. Precision differences and a missing chat template produce a subtly different model.
  2. The chat template used at serving time is the one used in training.
  3. Stop tokens are configured; otherwise the model talks until the token limit.
  4. Throughput and memory are measured with realistic concurrency, not one request at a time.

Operating it

Put the endpoint behind the same reliability layer as any LLM API: timeouts, retries, structured logs with the model version in every record. A fine-tuned model is a dependency you now maintain; version the adapter, the dataset and the evaluation results together so you can answer "what changed?" six months later.

Training is the exciting half. Serving is the half that makes it real, and it is where students separate themselves from tutorial followers.

About Pranjul Rathour

Pranjul Rathour holding a trophy and a certificate of merit after a win
Trophy and certificate after a win

Pranjul Rathour presenting evaluation criteria — feasibility, innovation, practicality, problem solving — on a projector screen
Walking a room through evaluation criteria

Pranjul Rathour presenting BrandHive on a projector screen
Presenting BrandHive

Pranjul Rathour in a shirt and tie holding a microphone in front of a career-opportunities slide
A career session for students

Pranjul Rathour presenting with a microphone in front of a slide reading 'Now what's the conclusion?'
Presenting to a room

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-06.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)