This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

NetCortex is an on-call network engineer for small ISPs and gaming cafes. It runs on local, open AI.

NetCortex was built collaboratively with @anaskhan3459 , with both of us contributing to the project.

A small internet provider or a gaming cafe usually has one or two technicians, a few MikroTik routers, and no network operations team. When the network gets slow, every problem arrives as the same complaint: "the internet is slow." Is one customer eating the bandwidth? Is the main line down? Is DNS slow? Or is it the upstream provider, so there's nothing to fix locally? Finding out, and then typing the right commands into a router late at night, is the hard part.

NetCortex does that work and asks before it touches anything:

1. Watches. A collector pings targets and times DNS lookups every 10 seconds.
2. Catches. When the numbers go bad, it opens an incident.
3. Explains. Gemma 4, running locally, reads the metrics through tools and writes a plain-English diagnosis, plus a short customer message in English and Urdu.
4. Proposes one safe fix. The model picks one of four tested fix templates and fills in the values. It never writes router commands itself.
5. Tests it first. The fix is applied to a virtual MikroTik router, checked, and rolled back before a person sees it.
6. Waits for a human. The owner sees the diagnosis, the exact commands and the rollback commands, then taps Approve or Reject.
7. Verifies. After applying, it watches the metric. If things didn't improve, it undoes the fix on its own.

Who would pay: small ISPs and gaming cafes. We think a small monthly fee per site is realistic for something that cuts evening outages and complaints. We haven't sold it to anyone yet.

Demo

[VIDEO: ]

Everything runs at home: a laptop, a Mac mini, and a virtual router. The incidents are simulated, because we can't break a real ISP for a demo. Gemma's reasoning, the router commands, the dry run and apply on real RouterOS, the workflow, the verification and the rollback are all real.

[SCREENSHOT: dashboard, a heavy-user incident awaiting approval, with diagnosis, evidence and the router commands]

[SCREENSHOT: the same incident resolved, with the timeline: applied → verifying → resolved, ping 36.9 → 17.4 ms]

[SCREENSHOT: the incident that rolled itself back]

Code

aunraza19 / NetCortex

Reasoning and diagnostics layer for networks

NetCortex

An on-call network engineer for small ISPs and gaming cafés that runs on local hardware It watches a site, opens an incident when something goes wrong, has a local Gemma model explain the likely cause and pick one fix from tested MikroTik templates, dry-runs that fix on a virtual router, waits for a person to approve it, applies it, and rolls it back if the key metric doesn't improve.

collector -> TimescaleDB -> rule-based detector -> Temporal incident workflow
  -> Gemma agent (tools) -> template-only fix -> CHR dry run -> human approval
  -> apply -> watch -> verify -> resolved, or automatic rollback

Run the demo

Needs: Windows, uv, the Temporal CLI, QEMU (for the CHR test router), Ollama with gemma4:e4b reachable at OLLAMA_BASE_URL, and a .env made from .env.example (Tiger Cloud DATABASE_URL, CHR_PASSWORD, optional SENTRY_DSN_BACKEND).

uv sync
powershell -ExecutionPolicy Bypass -

…

View on GitHub

To run the demo: infra\scripts\start-demo.ps1 starts Temporal, the virtual router, the worker, the API and the collector, checks the model server, and prints the dashboard URL. Details are in the README.

How I Built It

The open AI at the core

• Gemma 4 E4B, an open-weight model, runs in Ollama on a Mac mini M4 (16 GB). The laptop reaches it over the home Wi-Fi.
• Pydantic AI runs the agent. It gives Gemma three tools (read metrics, read forecasts, list the fix templates allowed for a cause) and forces a typed report: summary, cause, confidence, evidence, one template, its parameters, and customer messages in English and Urdu.
• Temporal (open source) makes the whole thing durable. Every model call and tool call is its own Temporal activity, through Pydantic AI's TemporalAgent. If the worker crashes mid-investigation, it resumes where it stopped, and waiting hours for a human costs nothing.

Safety: the model never writes a router command

Letting a language model type into a router is a bad idea, so it doesn't.

• Templates only. Four fixes in YAML: queue_limit, failover_route, priority_queue and dns_servers. Each one has typed parameters, apply commands, rollback commands, and a rule for what "better" means. A fifth choice, none, means "tell the owner, change nothing."
• Parameters are checked. Types, ranges and IP formats are validated. A bandwidth cap must be at most half of the subscriber's current rate. On a failed check, the error goes back to Gemma for one retry; if that fails too, the incident becomes notify-only.
• Every fix is tested first. Export the config, apply, export and diff, roll back, export again. A fix passes only if it changed something and the config came back exactly as it was. GitHub Actions boots its own virtual router and runs the same test on every pull request that changes fix code.
• Old config is never touched. We found a real bug here: when a fix failed because a queue with that name already existed, the rollback still ran and deleted the older queue. Now every object a fix creates is named after its incident (nc-1a2b3c4d), and a failed fix undoes only what it created.
• A person approves every change.

The rollback that really happened

During testing we injected an uplink outage. About 90 seconds in, our rule-based detector saw DNS lookups failing and called it "slow DNS". Gemma agreed, with confidence 0.75, and proposed switching the router's DNS servers. The dry run passed, because the commands were valid, and we approved it.

The workflow then measured DNS again: 12.0 ms before, 12.3 ms after. The template requires a 30% drop, so it rolled the change back on its own and closed the incident as "rolled back."

The fix was wrong because our detector was wrong, and the safety net caught it. We then fixed the detector: an outage now takes priority over slow DNS, and a test reproduces the old mistake.

Thinking on or off?

Gemma 4 can "think" before answering. We tried both on three hand-made incidents (heavy user, uplink outage, slow DNS), two runs each:

	Thinking off	Thinking on
Median time per investigation	9.0 s	33.6 s
Right cause	6/6	4/6
Right fix template	5/6	3/6
Right parameters	5/6	3/6

Thinking was almost 4× slower and less accurate on these cases. It's a small sample, but the choice was easy: NetCortex runs with thinking off, at temperature 0.2. In a separate run of 11 investigations, 10 produced a valid report with the right template and parameters. The one failure came from a field the model wasn't supposed to fill; we removed that field from what the model produces.

Observability with Sentry

Every incident is one Sentry trace, and the trace ID is the incident ID. Inside it: every Gemma call, every tool call, the dry run with its router API calls, the apply, the verify.

One thing we had to fix: Sentry's Pydantic AI integration recorded nothing under Temporal. The agent loop runs in Temporal's workflow sandbox and the model calls run in activities, so the integration never saw a call. We wrapped the model ourselves. Each Gemma request now opens a gen_ai.chat span with the model name and token counts. In one heavy-user incident, Gemma made five calls: 1,136–1,879 input tokens, 16–238 output tokens, 0.6–14.5 s each.

[SCREENSHOT: Sentry trace with gen_ai.chat spans and token counts]

We also found that a monitoring span was landing inside incident traces, because parallel activities shared one Sentry scope. Each activity now gets its own scope, and a test proves it.

[SCREENSHOT: Temporal timeline of the incident, including the 2-hour approval wait]

Data

The collector has been running on a real home connection in Pakistan, sending five pings to each of five targets (my router, the ISP's first hop, 1.1.1.1, 8.8.8.8, and a Valve CS2 relay in Dubai) and timing DNS lookups through two resolvers. That's over 12,000 rows in TimescaleDB on Tiger Cloud.

One real finding: on this connection, DNS lookups through 8.8.8.8 took 34.1 ms at the median against 11.2 ms through 1.1.1.1, three times slower. Pings show a similar gap (21.9 vs 6.8 ms), so it's the network path, not the resolver. That's exactly the kind of change the dns_servers template exists for.

The typical evening wasn't slower in our data (median ping to 1.1.1.1: 6.7 ms in the evening, 6.8 ms after 1 am). Only the slow tail grew: the 95th percentile went from 20.9 to 29.9 ms. It's one evening, so we don't read much into it.

Results

End-to-end runs on the full stack: laptop, Gemma on the Mac mini, virtual router, Tiger Cloud. Times after the first column are from when the incident opened.

Scenario	Fault → incident	Report	Dry run passed	Outcome	Key metric
Heavy user	66 s	14.6 s	20.8 s	resolved	ping 56.0 → 24.9 ms
Uplink down	151 s	14.2 s	20.9 s	resolved	loss 83% → 0%
Slow DNS	121 s	11.8 s	17.9 s	resolved	DNS 440.5 → 23.3 ms
Outage read as slow DNS	~90 s	18 s	24 s	rolled back	DNS 12.0 → 12.3 ms

The "after" numbers improve because the simulator scripts the recovery. What the table shows is that each step ran, on real systems, in the time shown.

Honest limits

• Incidents are simulated. The virtual router carries no customer traffic.
• The trigger is rule-based. Gemma does the diagnosis; simple rules decide when to open an incident.
• We cut features. We also built TabPFN forecasting and documentation search, but they weren't reliable enough by the deadline, so they're not in the demo.
• Narrow scope: MikroTik only, four fix templates, never a real router yet.
• Small evaluation: 11 agent runs, 12 thinking-mode runs, a handful of end-to-end runs.

Next: read-only mode on one real router at one real site, then approved fixes; more causes and templates; documentation search the agent can cite.

Why Does Open Innovation Matter?

For this product, a closed AI API would break it in three ways:

1. Privacy. The agent sees subscriber IP addresses, traffic rates and router config. With a cloud LLM API, all of that goes to someone else's server. With Gemma on a Mac mini, the AI never leaves the building.
2. It has to work when the internet doesn't. The tool that diagnoses a dead uplink can't depend on that uplink to reach a cloud model. Local Gemma keeps working.
3. Cost. A small ISP can't run a per-token bill every time the network hiccups. With an open model it buys one small computer, once.

Open weights also let us change how the agent behaves and measure it. We switched thinking off because our own test showed it was 4× slower and less accurate for this job, and nothing locked us into one provider's defaults. Temporal and Pydantic AI being open source let us see why Sentry's integration missed our model calls, and fix it.

To be fair: we didn't benchmark a closed model. Our claim is that a closed API can't meet these three requirements at all, not that it would diagnose worse.

In this demo, metrics live on Tiger Cloud and traces go to Sentry. An on-site install would use self-hosted TimescaleDB, which is open source, and Sentry is optional.

My Agent Session

We built NetCortex with coding agents and recorded the sessions with Entire: 42 checkpoints on our commits. Two worth reading:

• Building the agent: [https://entire.io/gh/aunraza19/NetCortex/commit/39f389abdb42f555aac19ef2d98f555e8224220b]
• Finding the rollback bug that deleted pre-existing router config: [https://entire.io/gh/aunraza19/NetCortex/commit/bf45a5245b114e63fca6c90597b620db85604d06]

The full transcripts are also public in the repo under refs/entire/checkpoints/*.

Prize Categories

• Best Use of Gemma: Gemma 4 E4B runs locally in Ollama and does every diagnosis, tool call and fix choice.
• Best Use of Temporal: monitoring and incident workflows; every model and tool call is a durable activity; human approval is a Temporal signal that can wait two hours.

• Best Use of Sentry Agent Tracing: one trace per incident, gen_ai spans with token counts, and two tracing bugs found and fixed (see above).

• Best Use of GitHub Copilot: GitHub Actions runs lint and tests, and boots a virtual MikroTik to test every fix template on each pull request.

---

Built by [@aun_raza_43a28e3c6de26b5c] and [@anaskhan3459 ].