Building an AI-Augmented Web Portfolio with Qwen, WebLLM and WebGPU
Build an AI-augmented web portfolio with Next.js 16.2.12, React 19, Tailwind CSS v4, WebLLM, WebGPU, and Qwen2.5–1.5B-Instruct running locally in the browser.
Most portfolios are built to be browsed. You open the homepage, read the About section, look through a few projects, maybe download a résumé, scroll and leave.
I wanted to change how visitors interact with a web portfolio. Not by putting a chatbot in the corner of the screen, but by making the AI part of the portfolio itself.
So I built an AI-Augmented Web Portfolio.
The website is still more or less a normal web application. There are pages, projects, experience, skills, navigation and all the things you'd expect from a typical portfolio. The difference is that there is another layer sitting on top of all that. A local AI system that can understand questions, find relevant information, navigate the portfolio and have a conversation with the visitor.
Before we dive into this, let's take a moment to align ourselves with the way of thinking. We are not just throwing everything at the LLM. No, if someone says "show me your projects", I don't need an LLM to figure out what that means. Or if someone asks for a basic fact about my experience, I don't need an LLM for that either. The application handles those things itself. The model is there when a more contextual answer is useful. That became the basic idea behind the architecture:
Don't make the AI the application. Make the AI another component of the application.
The Tech Stack
- Framework: Next.js 16.2.12 (App Router, Static Generation)
- Language: TypeScript 5.8
- UI Library: React 19.2 (Canary)
- AI Runtime: WebLLM (MLC-AI)
- AI Model: Qwen2.5–1.5B-Instruct-q4f16_1-MLC (4-bit quantized)
- Inference: WebGPU
- Styling: Tailwind CSS v4 (CSS-first config)
- Animation: Motion v12 (formerly Framer Motion)
- Voice: Web Speech API (STT) + ElevenLabs (TTS via Proxy)
- Deployment: Vercel
- Testing: Playwright
You can use your own stack and I am sure there are plenty of alternatives out there but for me at the moment of developing this project, this stack was the best option available to my knowledge. The important part isn't any individual library. It's where the pieces run.
The portfolio UI, routing logic, fallback engine and LLM inference run on the client. While the voice output is the exception: ElevenLabs is accessed through a small server-side proxy so the API key doesn't have to be exposed in the browser. The voice input and output is an additional accessibility feature and can be excluded if a truly serverless development is desired.
The Architecture: A Cooperative Brain
The first version of the idea was simple:
Visitor
↓
LLM
↓
Answer
That worked, but it created a problem.
The model takes time to initialize. The browser has to load the model shards, and WebGPU has to initialize. Furthermore, there are plenty of situations where using an LLM is simply unnecessary.
So the next question was "What can the application do when the model isn't available and when does the LLM step in?"
Instead of making every interaction go through the model, the portfolio became a cooperative parallel handoff system across several tiers of intelligence.
The application doesn't sit around waiting for the LLM to become ready. It does what it can immediately. When the model is available, it can take the interaction further.
1. Offloading Reasoning to a WebWorker
Running an LLM in the browser is not exactly lightweight. There is model initialization, WebGPU setup, token generation, and streaming to deal with.
I didn't want that work competing with the UI thread. So WebLLM runs inside a Web Worker.
The worker owns the model instance and communicates with the main application through messages. Accessing WebGPU inside a Worker is the secret sauce, as it offloads the heavy GPGPU work from the main thread entirely.
A simplified version looks like this:
// src/lib/worker.ts
const MODEL_ID = "Qwen2.5-1.5B-Instruct-q4f16_1-MLC";
class WebLLMEngineSingleton {
static instance: MLCEngine | null = null;
static async getInstance() {
if (this.instance === null) {
this.instance = await CreateMLCEngine(MODEL_ID, {
initProgressCallback: (report) => {
self.postMessage({
status: "progress",
data: {
progress: Math.round((report.progress ?? 0) * 100),
},
});
},
});
}
return this.instance;
}
}
The singleton isn't there because singletons are fashionable. It's there because I only want one model engine managing the local inference session.
If React causes the surrounding application to re-render, I don't want another initialization path accidentally creating another engine — triggering parallel ~950MB redownloads of the model weight shards and ballooning the ~1.6GB VRAM active execution footprint which is already quite heavy.
The worker then sends progress and generated text back to the main thread:
worker.onmessage = (event) => {
if (event.data.status === "progress") {
setProgress(event.data.data.progress);
}
if (event.data.status === "stream") {
setMessages((prev) =>
updateLastMessage(prev, event.data.text)
);
}
};
The application consumes the results. The UI doesn't have to directly manage the model.
2. Don't Ask an LLM to Do Something a Router Can Do (Tier 1)
One of the simplest things the assistant can do is navigate the portfolio.
If someone says: "Show me the projects." I don't need Qwen to reason about that.
A small command router can handle it instantly. The router scores known keywords against the available views and returns the strongest match.
// src/hooks/useCommandRouter.ts
function detectView(input: string) {
const text = input.toLowerCase().trim();
const scores = new Map<ViewKey, number>();
for (const [view, data] of Object.entries(viewMap)) {
const keywordScore = data.keywords.reduce(
(total, kw) =>
total + (text.includes(kw) ? data.weight : 0),
0
);
scores.set(view, keywordScore);
}
const [bestView, bestScore] = [
...scores.entries(),
].sort((a, b) => b[1] - a[1])[0];
return bestScore > 0
? { view: bestView, score: bestScore }
: { view: null, score: 0 };
}
It's deliberately boring. And that's the point. This prevents the model from inventing a navigation decision when the application already knows the answer.
3. The Fallback Engine (Tier 2)
Navigation isn't the only thing that can bypass the model. There are plenty of questions where the portfolio already has the information:
- Who is Shahriar?
- What technologies does he use?
- What is his experience?
For these, there is a lightweight intent engine that evaluates queries in <5ms.
// src/lib/fallback-engine.ts
const intentPatterns = [
{
name: "who_is",
keywords: ["shahriar", "bio", "background"],
weight: 3,
answer: answerAboutBio,
},
{
name: "skills",
keywords: ["tools", "tech stack", "languages"],
weight: 2,
answer: answerSkillsTools,
},
];
export function buildFallbackAnswer(userText: string) {
const lower = userText.toLowerCase();
const hit = intentPatterns.find(
(p) => scoreIntent(lower, p) > 0.3
);
return hit ? hit.answer(userText) : null;
}
This gives the application something useful to say even if the LLM isn't ready. The portfolio shouldn't feel broken during the several seconds the model takes to initialize.
4. Then Let the Model Take Over (Tier 3)
Once the local model is ready, questions that don't fit the deterministic paths can be passed to Qwen.
Instead of just matching: "What are your skills?" the model can deal with something more open-ended:
"I'm looking for someone with experience working across support, networking and AI tooling. What have you actually built that would be relevant?"
That's a much better use for an LLM.
The application already has the portfolio data. The model's job is to turn that information into a useful response for the visitor.
Known problem
↓
Deterministic solution
Unknown / contextual problem
↓
Local LLM
That's a much better division of labour.
5. Letting the Model Control the Interface
I didn't just want the model to talk about the portfolio. I wanted it to be able to interact with the portfolio.
If a visitor asks to see my projects, or asks a question where navigating to the projects section would be helpful, the model can emit a control directive such as:
INITIATING_NAVIGATION: projects
The application detects that directive and performs the navigation itself. The model doesn't directly manipulate the React application.
Because LLMs stream token-by-token, a tag might split across two chunks. We store the accumulated stream and check the full text rather than assuming a control directive will arrive in one chunk:
// src/hooks/usePortfolioWorker.ts
const visibleText = rawText
.split("\n")
.filter((line) => !line.startsWith("INITIATING_"))
.join("\n");
if (rawText.includes("INITIATING_NAVIGATION:")) {
const view = rawText
.split("INITIATING_NAVIGATION:")[1]
.trim()
.toLowerCase();
if (VALID_VIEWS.includes(view)) {
onNavigate(view);
}
}
The idea is simple: The model proposes. The application validates. Following this idea keeps the LLM inside a controlled boundary.
6. The TTS and STT voice problem.
The AI itself is local, but voice is different.
We can also enable speech input for the portfolio by using the browser's Web Speech API. For speech output, I used ElevenLabs free tier.
The TTS request goes through a stateless Next.js API route:
Browser
│
│ text
▼
Next.js API route
│
│ API key stays here
▼
ElevenLabs
│
│ audio
▼
Browser
The API route protects our API keys and gives me somewhere to apply basic request controls. Requests are capped at 500 characters to protect quotas, and there is an ephemeral in-memory IP-based rate limit within the serverless module scope to prevent basic cost spikes.
// src/app/api/tts/route.ts
const RATE_LIMIT_MAP = new Map<string, { count: number; reset: number }>();
It isn't pretending to be a production abuse-prevention system, but it's enough for this project. I hope…
7. The UI Shouldn't Have to Wait for AI
This is probably the most important lesson from the project.
A common pattern with AI interfaces is:
User
↓
Loading...
↓
AI
↓
Response
I wanted something closer to:
User
↓
Application responds immediately
↓
Local model becomes available
↓
AI adds more context
If WebGPU isn't available, the deterministic parts can still work. If a question has an obvious answer, the application doesn't need to wake up a language model.
8. Testing the Handoff
This kind of interface is slightly harder to test than a normal website.
User asks question
↓
Fallback answer appears
↓
Local model becomes available
↓
Model response streams
↓
Control directive detected
↓
Navigation occurs
I used Playwright to test these interaction paths and make sure the handoff doesn't leave the interface in an awkward state.
The Trade-Off
Instead of my server paying for every inference request, the visitor's machine does the work. That comes with some obvious trade-offs:
- The model has to be downloaded.
- WebGPU support matters.
- Device memory and GPU capability matter.
- Performance is dependent on user's machine.
- Voice still relies on external services.
Conclusion
The goal was to reimagine a website where the visitor feels less like browsing a collection of pages and more like interacting with the persona behind it.
That's ultimately what I wanted from the project.
Try it
Live Demo: https://shahriarhaqueabirportfolio.vercel.app
Source: https://github.com/shahriarhaqueabir/ShahriarHaqueAbirPortfolio
Top comments (0)