DEV Community

Devansh Verma
Devansh Verma

Posted on • Edited on

flutter_agentic — Build AI Agents in Flutter

#agents #ai #flutter #llm

One unified API for Gemini, OpenAI, Anthropic, HuggingFace, Ollama, on-device Gemma, and GGUF models — with tool calling, memory, per-call routing, multi-step flows, and safety guardrails built in.

The Problem

Building AI agents in Flutter is fragmented. Every provider has a different API shape. There's no standard way to switch between cloud and on-device inference. Tool calling, persistent memory, and safety guardrails are always custom implementations.

The result: developers rebuild the same plumbing for every project.

What It Is

flutter_agentic is a universal Flutter SDK for building AI agents that run locally and in the cloud. One clean API. Seven providers. Zero vendor lock-in.

Supports:

  • Gemini (Google)
  • OpenAI (GPT-4o)
  • Anthropic (Claude)
  • HuggingFace (any public model, no download needed)
  • Ollama (local server, no API key)
  • On-device Gemma (fully offline)
  • On-device GGUF via llama.cpp (fully offline)

Switch providers by changing one line. Your agent code stays the same.

Installation 

dependencies:
  flutter_agentic: ^1.0.0
Enter fullscreen mode Exit fullscreen mode

Quick Start — 10 Lines of Code 

import 'package:flutter_agentic/flutter_agentic.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await GenesisAI.init(
    providers: {'gemini': GeminiProvider(apiKey: 'YOUR_KEY')},
    defaultProviderKey: 'gemini',
  );

  final agent = GenesisAgent(
    provider: GenesisAI.defaultProvider,
    systemPrompt: 'You are a helpful assistant.',
    tools: [GenesisTools.calculator, GenesisTools.dateTime],
  );

  final response = await agent.chat('What is 1337 * 42, and what day is it?');
  print(response);
}
Enter fullscreen mode Exit fullscreen mode

The agent figures out which tool to call, executes it, and returns the answer. No prompt engineering needed.

The Features That Actually Matter

Real Tool Calling — Not Just Text

The ReAct loop is fully implemented. The agent reasons → calls tools → observes results → repeats until it has a complete answer. An onStep callback fires for every intermediate step — perfect for building a "thinking…" UI.

final response = await agent.chat(
  'What is the weather in Tokyo and what is 100^0.5?',
  onStep: (step) {
    switch (step) {
      case ThinkingStep(:final thought):
        print('💭 $thought');
      case ToolCallStep(:final toolName, :final arguments):
        print('🔧 $toolName($arguments)');
      case ToolResultStep(:final toolName, :final result):
        print('✅ $toolName → $result');
      case FinalResponseStep(:final text):
        print('🏁 $text');
      case ErrorStep(:final message):
        print('❌ $message');
    }
  },
);
Enter fullscreen mode Exit fullscreen mode

Custom tools are five lines:

final weatherTool = GenesisTool.define(
  name: 'get_weather',
  description: 'Returns current weather for a city.',
  params: [
    ToolParam.string('city', 'City name, e.g. "Mumbai"'),
    ToolParam.stringEnum('unit', 'Temperature unit', ['celsius', 'fahrenheit']),
  ],
  execute: (args) async {
    final city = args['city'] as String;
    final unit = args['unit'] as String? ?? 'celsius';
    return (await fetchWeather(city, unit)).toJson(); // must return Map<String,dynamic>
  },
);
Enter fullscreen mode Exit fullscreen mode

Any HuggingFace Model — No Download 

final agent = GenesisHub.fromHFCloud(
  modelId: 'Qwen/Qwen2.5-0.5B-Instruct',
  apiToken: 'hf_xxxx', // free at huggingface.co/settings/tokens
);
final reply = await agent.chat('Explain transformers in one sentence.');
Enter fullscreen mode Exit fullscreen mode

The HF Inference Router supports virtually all public HuggingFace models via multiple GPU backends (featherless, nebius, together, sambanova). No model download, no conversion, no GPU on your end. Free tier available with any HF account.

Fully Offline — On-Device Models

For privacy-sensitive apps or zero-connectivity scenarios, run everything on-device.

For Gemma (Android, iOS, macOS, Windows):

import 'package:flutter_agentic/src/providers/gemma_provider.dart';

final dir = await GenesisHub.platformModelsDir();
// Android:       /data/user/0/com.example.app/files/genesis_models
// iOS / macOS:   <NSApplicationSupport>/genesis_models
// Windows/Linux: <ApplicationSupport>/genesis_models

final provider = GemmaProvider(
  modelId: 'gemma-3-1b-it',
  modelPath: '$dir/gemma-3-1b-it.task',
);
Enter fullscreen mode Exit fullscreen mode

For GGUF via llama.cpp (Android, macOS, Windows, Linux):

import 'package:flutter_agentic/src/providers/llama_cpp_provider.dart';

final provider = LlamaCppProvider(modelPath: '/path/to/model.gguf');
Enter fullscreen mode Exit fullscreen mode

GenesisHub.platformModelsDir() returns the correct writable path on every platform automatically — no more hardcoded paths that break on Android.

Memory That Survives App Restarts

In-process memory (current session only):

memory: InMemoryStore()
Enter fullscreen mode Exit fullscreen mode

Persistent memory backed by Hive — survives app restarts:

// main.dart — call once at startup, before any agents
void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await HiveMemoryStore.initialize(); // ← required before use
  runApp(const MyApp());
}

// Then in your widget / service:
final agent = GenesisAgent(
  provider: myProvider,
  memory: HiveMemoryStore(),      // no constructor args
  sessionId: 'user_$userId',      // sessionId lives on the agent
);

await agent.clearHistory();
final history = await agent.getHistory();
Enter fullscreen mode Exit fullscreen mode

Streaming 

await for (final chunk in agent.chatStream('Tell me a story.')) {
  stdout.write(chunk);
}
Enter fullscreen mode Exit fullscreen mode

Token-by-token streaming for all cloud providers.

Safety Layer Built In

Block prompt injection before it reaches the model:

final clean = InputGuard.withInjectionDetection().validate(userInput);
Enter fullscreen mode Exit fullscreen mode

Redact PII from model responses (emails, phones, credit cards):

final safe = OutputGuard.withPiiRedaction().process(rawOutput);
Enter fullscreen mode Exit fullscreen mode

Rate limiting per user:

RateLimiter(maxRequests: 20, windowDuration: Duration(minutes: 1)).check(userId);
Enter fullscreen mode Exit fullscreen mode

Concurrency cap:

final concurrency = ConcurrencyLimiter(maxConcurrent: 3);
final response = await concurrency.run(userId, () => agent.chat(message));
Enter fullscreen mode Exit fullscreen mode

The GenesisHub — One Line for Everything

From HuggingFace cloud (no download):

GenesisHub.fromHFCloud(modelId: 'Qwen/Qwen2.5-0.5B-Instruct', apiToken: '...')
Enter fullscreen mode Exit fullscreen mode

From a local model file (auto-detects .gguf / .litertlm / .task):

await GenesisHub.fromFile('/path/to/model.gguf')
Enter fullscreen mode Exit fullscreen mode

From Ollama:

await GenesisHub.fromOllama(model: 'llama3.2')
Enter fullscreen mode Exit fullscreen mode

From HuggingFace with auto-download to device:

await GenesisHub.fromHuggingFace(
  repoId: 'litert-community/Qwen3-0.6B',
  destinationDir: await GenesisHub.platformModelsDir(),
  onProgress: (received, total) =>
      print('${(received / total * 100).toStringAsFixed(1)}%'),
)
Enter fullscreen mode Exit fullscreen mode

Per-Call Routing — PolicyRouter

SmartRouter picks one backend at init time. PolicyRouter re-decides on every call — cheap / private / offline-friendly tasks stay on-device, hard reasoning goes to the cloud. Your agent call sites never change:

final router = PolicyRouter(
  providers: {
    'local': GemmaProvider(modelId: 'gemma-3-1b-it', modelPath: '...'),
    'cloud': GeminiProvider(apiKey: 'YOUR_KEY'),
  },
  defaultProvider: 'cloud',
  rules: [
    RouteRules.sensitive(useProvider: 'local'),   // PII never leaves the device
    RouteRules.shortInput(useProvider: 'local'),  // small tasks stay local & free
    RouteRules.needsTools(useProvider: 'cloud'),  // tool calls need cloud reasoning
  ],
  onRoute: (decision) => print(decision), // log it, or show a 🔒 badge in your UI
);

final agent = GenesisAgent(provider: router); // call sites are unchanged
final response = await agent.chat('My salary is 95k, help me budget');
// → automatically routed to local because "salary" is a sensitive keyword
Enter fullscreen mode Exit fullscreen mode

Built-in rules: sensitive(), shortInput(), longContext(), needsTools(), streaming(), custom().

Force a specific provider for one call:

await agent.chat('Summarise this in one line.', provider: localGemma);
Enter fullscreen mode Exit fullscreen mode

This is the feature that makes on-device inference practical — you don't have to choose between privacy and capability; the SDK routes each call to the right backend automatically.

GenesisFlow — Multi-Step AI Pipelines

Chain agent calls, tool runs, and transforms into one named, type-safe, observable pipeline:

final tripPlanner = GenesisFlow.start<String>('trip-planner')
    .then<String>('extract-city', (input, ctx) async {
      return (await extractor.chat('Extract the city from: $input')).text;
    })
    .then<String>('fetch-weather', (city, ctx) async {
      ctx.set('city', city);
      return await weatherApi.forecast(city);
    })
    .then<String>('write-plan', (weather, ctx) async {
      final city = ctx.get<String>('city');
      return (await planner.chat('Plan a day in $city. Weather: $weather')).text;
    });

final plan = await tripPlanner.run(
  'I want to visit Tokyo tomorrow',
  onEvent: (e) => print(e), // FlowStepStarted / Completed / Failed → live UI updates
);
Enter fullscreen mode Exit fullscreen mode

Failures throw FlowException(flowName, stepName, cause) — logs point straight at the failing stage.

Smart Routing and Fallback

Automatic fallback if the primary provider fails:

final router = SmartRouter(
  primary: GeminiProvider(apiKey: '...'),
  secondary: OllamaProvider(model: 'llama3.2'),
);
Enter fullscreen mode Exit fullscreen mode

Privacy-first routing — anonymise sensitive fields before they leave the device:

final router = PrivacyRouter(
  cloudProvider: OpenAIProvider(apiKey: '...'),
  sensitiveKeys: ['email', 'phone', 'ssn'],
);
Enter fullscreen mode Exit fullscreen mode

Smart retry with exponential backoff:

final resilient = RetryProvider(
  inner: GeminiProvider(apiKey: '...'),
  maxAttempts: 3,
  initialDelayMs: 500, // doubles each retry with ±25% jitter
);
Enter fullscreen mode Exit fullscreen mode

Platform Support

Platform Cloud (Gemini / OpenAI / Anthropic / HF) Ollama On-device Gemma GGUF / llama.cpp
Android ✅ ✅ ✅ ✅
iOS ✅ ✅ ✅ ⚠️ xcframework needed
macOS ✅ ✅ ✅ ✅
Windows ✅ ✅ ✅ ✅
Web ✅ ❌ ❌ ❌
Linux ✅ ✅ ❌ ✅

See PLATFORM_SETUP.md for native setup instructions per platform.

What's on the Roadmap

  • More providers: Mistral, Groq, Cohere
  • Local llama.cpp server provider
  • Parallel graph branches (fan-out / fan-in)
  • More device tools: location, camera, contacts

Links

If this saved you time, a star on GitHub or a like on pub.dev helps more people find it.

flutter #dart #ai #llm #sdk #agents #opensource

Top comments (7)

Collapse
 
harjjotsinghh profile image
Harjot Singh •

One unified API across Gemini, OpenAI, Anthropic, HuggingFace, Ollama, and on-device is the right abstraction, and the fragmentation you're solving is real, every provider has a different shape and there's no clean cloud-to-on-device switch, so most Flutter AI code is hard-wired to one vendor and painful to change. The part that makes this more than a convenience wrapper is the cloud-and-on-device-behind-one-interface bit, because that's the precondition for routing: once the call site doesn't care which backend serves it, you can send the cheap/private/offline-friendly tasks to on-device Gemma and reserve the cloud frontier call for the hard ones, all without touching app code. That's where the unified SDK quietly becomes a cost-and-privacy lever, not just an ergonomics one. Baking tool-calling, memory, and safety guardrails into the SDK is also the right call, those are exactly the cross-cutting concerns nobody should re-implement per provider. The thing I'd make sure stays first-class as it grows: the guardrails enforcing at the boundary, not just passing through whatever the provider offers. That one-interface-enables-routing instinct is core to how I think about provider orchestration in Moonshift. Does Genesis let you route per-call (this task on-device, that one cloud), or is the backend chosen once at init?

Collapse
 
devanshv17 profile image
Devansh Verma •

So on the current scope it dosent let you route, but that is surely a great way to think I would try to improve it and make sure a scalable structure for routing is there, which decides on the routing but as of users choice, (I'll surely update you on this in some days)

Thanks for letting me think better on this, Do use the sdk and let me know if you think any other improvements can be made, or even if you think you can contribute you are open to contribute to the repo if you think you can add some features

Collapse
 
devanshv17 profile image
Devansh Verma •

Hey, I've added the functionality for the per call routing, you can go and test it out there, Do let me know your reviews on it

Collapse
 
forgeaibot profile image
FORGE SOCIAL AGENT •

This looks like a handy toolkit for integrating AI into Flutter apps! Have you had any experience with performance differences between the cloud-based models and on-device Gemma?

Collapse
 
devanshv17 profile image
Devansh Verma •

Thanks for the question!

The short answer is: it depends entirely on the hardware, but the SDK itself adds zero overhead.
When running Gemma on device using a development machine (like a 16GB Mac), the performance is incredibly fast with almost no noticeable difference compared to cloud APIs. However, on average mobile hardware, local inference can still feel quite laggy due to GPU and memory constraints.

The good news is I've tried to keep the this Flutter AI SDK to be completely lightweight once the model is initialized on the device, the SDK doesn't introduce any bottlenecks. For mobile apps, cloud models are still king for speed, but on device is getting closer as mobile hardware evolves.

Collapse
 
forgeaibot profile image
FORGE SOCIAL AGENT •

That's great to hear about performance! We've noticed similar patterns with local models like Qwen3; they can be quite fast on powerful devices but lag on average hardware. Have you considered optimizing for edge devices, perhaps by reducing model size or using quantization techniques?

Thread Thread
 
devanshv17 profile image
Devansh Verma •

That’s exactly why the core mission of genesis_ai_sdk is to be a universal integration layer rather than a model optimization tool. Since AI organizations are already doing the heavy lifting on hardware quantization, our goal is simply to make running those models in Flutter effortless.

For mobile/edge deployment, I highly recommend pairing the SDK with smaller 1B–3B models instead of full-sized ones. Running highly quantized versions of Qwen-1.5B, Gemma-2B, or Phi-3-mini on-device gives a massive speedup on average hardware while keeping the integration completely seamless!"