Like many projects, this one started with a problem. Mine was a speeding ticket. My developer brain immediately jumped from
frustration to system design: "What would it take to build a better alert system? One that's silent, private, and works without a
constant internet connection?"
This post is the technical deep dive into the system I built: BuzzOff. We'll cover the architectural decisions, the Python data
pipeline, the offline-first "Country Pack" system using SQLite and R-trees, and the core logic of the Flutter app.
Let's get into the code.
** Core Architectural Decisions**
- Offline-First is Non-Negotiable: A car is a Faraday cage on wheels. Mobile data is unreliable. An app that relies on a constant network connection to check for nearby cameras is doomed to fail. This meant the core logic and data had to live on the device.
- Do the Heavy Lifting on the Backend: While the app must work offline, the user's phone is not the place to be processing gigabytes of raw, messy data from dozens of sources. This led to a backend-heavy architecture where a powerful data pipeline does all the hard work upfront.
- The Right Tools for the Job:
- Backend: Python with FastAPI for its incredible async performance, dependency injection system, and automatic documentation via Pydantic.
- Database: PostgreSQL with the PostGIS extension for its powerful geospatial querying capabilities during data processing.
- Mobile App: Flutter to build a single, beautiful, and performant app for both iOS and Android from one codebase.
- Admin Panel: React to build a quick and functional internal dashboard for managing the data pipeline.
** Deep Dive: The Data Pipeline
**
The heart of the entire platform is the data pipeline. Its job is to ingest chaotic data from the real world and forge it into clean,
efficient, and distributable "packs".
** 1. Ingestion: The Adapter System
**
I designed a pluggable "adapter" system in Python to fetch data from various sources. The first and most important is for
OpenStreetMap (OSM). Using the Overpass API, I can pull down camera data for an entire country with a single query.
`1 # A simplified look at fetching data from OSM
2 import httpx
3
4 # This query finds all nodes tagged as "speed_camera" within Israel's borders
5 OVERPASS_QUERY = """
6 [out:json];
7 area["ISO3166-1"="IL"][admin_level=2];
8 (node["highway"="speed_camera"](area););
9 out body;
10 """
11
12 def fetch_osm_cameras():
13 # Make a POST request to the Overpass API
14 response = httpx.post(
15 "https://overpass-api.de/api/interpreter",
16 data={"data": OVERPASS_QUERY}
17 )
18 response.raise_for_status()
19 # The camera data is in the "elements" key
20 return response.json()["elements"]
21
22 print(f"Found {len(fetch_osm_cameras())} cameras in OSM!")`
This raw data is often messy—duplicates, misplaced tags, etc. It gets dumped into a "raw_cameras" table in our PostgreSQL database
for the next stage.
** 2. Processing & Deduplication
**
Once the data is in PostgreSQL, I use the power of PostGIS to clean it up. The most critical step is deduplication. If one source
says a camera is at (lat, lon) and another says it's 15 meters away, they are likely the same camera. I run a geospatial query to
find and merge any cameras within a certain threshold (e.g., 50 meters) of each other.
- The "Country Pack" Generator
This is the most crucial part of the architecture. How do you get the data onto the phone for offline use?
I generate a SQLite file for each country. This isn't just a simple table; it's a highly optimized, portable database. The real magic
is using an R-tree spatial index.
An R-tree is a special database index for geospatial data. Instead of indexing a simple value, it indexes a 2D area (a "bounding
box"). This allows for incredibly fast "what's near me?" queries.
Here's the Python code that generates a pack:
1 # A conceptual look at the pack generator
2 import sqlite3
3
4 def generate_pack(country_code, cameras):
5 db_file = f"packs/{country_code}.db"
6 conn = sqlite3.connect(db_file)
7 cursor = conn.cursor()
8
9 # 1. Create the main camera data table
10 cursor.execute("""
11 CREATE TABLE cameras (
12 id TEXT PRIMARY KEY,
13 lat REAL NOT NULL,
14 lon REAL NOT NULL,
15 type TEXT
16 )
17 """)
18
19 # 2. Create the R-tree virtual table. This is the magic.
20 # It stores the bounding box for each camera.
21 cursor.execute("""
22 CREATE VIRTUAL TABLE cameras_rtree USING rtree(
23 id, min_lat, max_lat, min_lon, max_lon
24 )
25 """)
26
27 # 3. Insert camera data into both tables
28 for cam in cameras:
29 cursor.execute(
30 "INSERT INTO cameras VALUES (?, ?, ?, ?)",
31 (cam['id'], cam['lat'], cam['lon'], cam['type'])
32 )
33 # For an R-tree, the min/max lat/lon for a point are just the same value
34 cursor.execute(
35 "INSERT INTO cameras_rtree VALUES (?, ?, ?, ?, ?)",
36 (cam['id'], cam['lat'], cam['lat'], cam['lon'], cam['lon'])
37 )
38
39 conn.commit()
40 conn.close()
41 print(f"Generated {db_file} successfully!")
The resulting .db file is the "Country Pack." The app downloads this single file, giving it all the data it needs to run completely
offline.
** Deep Dive: The FastAPI Server
**
The FastAPI backend serves the generated packs and provides an API for the admin panel. Its use of Pydantic for data validation is
fantastic. You define your data shape once and get validation and serialization for free.
1 # From backend/app/schemas/developer.py
2 from pydantic import BaseModel
3
4 # This Pydantic model defines the structure for a camera submission
5 # FastAPI will automatically validate incoming requests against this
6 class Camera(BaseModel):
7 lat: float
8 lon: float
9 type: str = "fixed_speed"
10 speed_limit: int | None = None
11 address: str | None = None
An endpoint to get pack metadata looks beautifully simple:
1 # From backend/app/api/routes/public.py
2 @router.get("/packs/{country_code}/meta")
3 def get_pack_meta(country_code: str):
4 # Logic to find the latest pack file for the country
5 pack = find_latest_pack(country_code)
6 if not pack:
7 raise HTTPException(status_code=404, detail="Pack not found")
8 return {
9 "version": pack.version,
10 "camera_count": pack.camera_count,
11 "file_size_bytes": pack.file_size_bytes,
12 "checksum_sha256": pack.checksum_sha256,
13 }
** Deep Dive: The Flutter App's Proximity Engine
**
The Flutter app's job is to use the downloaded SQLite pack to check for nearby cameras. Here’s the core logic that runs every few
seconds while driving:
- Get the phone's current GPS location (lat, lon).
- Calculate a search "bounding box" (e.g., 2km north/south/east/west) around the current location.
- Execute a query against the local SQLite database's R-tree index:
1
SELECT id FROM cameras_rtree WHERE min_lat > ? AND max_lat < ? AND min_lon > ? AND max_lon < ?
This query is incredibly fast, even with tens of thousands of cameras in the pack.
- For the handful of cameras returned by the R-tree query, run a more precise Haversine distance calculation to get the exact distance in meters.
- If any camera is within the alert radius (e.g., 500 meters), check the user's GPS heading. If they are moving towards the camera, trigger a haptic feedback (vibration).
- Debounce the alert for that camera ID for a set period (e.g., 2 minutes) to prevent constant vibrations.
This multi-stage filtering process—a broad query with the R-tree followed by a precise calculation on a small subset—is the key to
making the app both fast and battery-efficient.
** Lessons Learned
**
- Pre-calculation is powerful: Doing the heavy data processing ahead of time on a server is a classic but effective pattern for mobile apps.
- SQLite is a beast: Don't underestimate SQLite. For read-heavy applications on the edge, it's an incredible tool, especially with extensions like R-tree.
- The right tool for each job: Combining the strengths of Python for data, Flutter for UI, and PostGIS for geospatial processing was a winning formula.
The project continues to evolve. I've since built a Python SDK to allow community contributions, and I'm working on automating the
entire data pipeline. What started as a simple problem became a fascinating journey into full-stack application development.
I hope this deep dive was useful. You can find more about the project at buzzoff.me. I'd love to hear your feedback on the
architecture in the comments below
Top comments (0)