DEV Community: Mukhtar

Turn Any API Into a SQL Database

Mukhtar — Wed, 20 May 2026 20:29:36 +0000

GitHub, Jira, Salesforce, Stripe—600+ sources → SQLite

The 2-Minute Version:

# Install surveilr
brew tap surveilr/tap && brew install surveilr

# Create a Singer tap script (see below for example)
# then ingest it
surveilr admin init -d project.db
surveilr ingest files -r ./github.surveilr[singer].py -d project.db
surveilr orchestrate adapt-singer -d project.db --stream-prefix github_

# Query it
surveilr shell -d project.db

-- Query GitHub commits from the last 7 days
SELECT author, message, timestamp
FROM github_commits
WHERE timestamp > datetime('now', '-7 days')
ORDER BY timestamp DESC;

Your API data is now SQL. Query it like any other database.

The Problem with API Data

You've got project data scattered across platforms:

GitHub — Issues, commits, pull requests
Jira — Tickets, sprints, workflows
GitLab — Merge requests, pipelines
Salesforce — CRM data, opportunities
Stripe — Payments, subscriptions

Each has its own API. Each has rate limits. Each has different authentication.

And you need to answer questions like:

"Show me all commits related to high-priority Jira tickets"
"Which GitHub issues have no linked pull requests?"
"Track deployment frequency over time across all repos"

The usual approach:
Write custom scripts for each API, export to JSON/CSV, wrangle in pandas, hope nothing breaks.

The surveilr approach:
Turn everything into standard SQLite tables you can query with SQL—forever.

What surveilr Does

surveilr ingests data from APIs using Singer taps—Python scripts that follow the Singer protocol.

There are 600+ pre-built Singer taps for:

GitHub, GitLab, Bitbucket
Jira, Linear, Asana
Salesforce, HubSpot, Zendesk
Stripe, Shopify, QuickBooks
Postgres, MySQL, MongoDB
Google Analytics, Mixpanel
...and hundreds more

surveilr executes these taps and transforms their output into queryable SQL tables.

How It Works: Singer Taps

Singer taps are Python scripts that output JSONL (JSON Lines):

{"type": "SCHEMA", "stream": "commits", "schema": {...}}
{"type": "RECORD", "stream": "commits", "record": {"sha": "abc123", "author": "alice", ...}}
{"type": "RECORD", "stream": "commits", "record": {"sha": "def456", "author": "bob", ...}}
{"type": "STATE", "value": {"bookmark": "2024-05-12"}}

surveilr reads this output and creates SQL tables automatically.

You don't write SQL schemas. surveilr infers them from the Singer output.

Step 1: Find or Create a Singer Tap

Option 1: Use an Existing Tap

Browse Singer Hub for 600+ taps:

tap-github — GitHub repos, issues, commits, PRs
tap-gitlab — GitLab projects, merge requests, pipelines
tap-jira — Jira issues, sprints, users
tap-postgres — Extract from Postgres databases
tap-stripe — Payments, customers, subscriptions

Install with pip:

pip install tap-github

Option 2: Write Your Own

Singer taps are just Python scripts. Here's a minimal example:

#!/usr/bin/env python3
import json
import sys

# Print schema
print(json.dumps({
    "type": "SCHEMA",
    "stream": "users",
    "schema": {"properties": {"id": {"type": "integer"}, "name": {"type": "string"}}}
}))

# Print records
for user in fetch_users_from_api():
    print(json.dumps({
        "type": "RECORD",
        "stream": "users",
        "record": {"id": user.id, "name": user.name}
    }))

That's it. surveilr handles the rest.

Step 2: Create a Capturable Executable

surveilr runs Singer taps as capturable executables—scripts with a special naming pattern:

<name>.surveilr[singer].<extension>

Examples:

github.surveilr[singer].py
jira.surveilr[singer].sh
stripe.surveilr[singer].js

The [singer] marker tells surveilr: "This script outputs Singer format."

Example: GitHub Tap Script

#!/usr/bin/env python3
import subprocess
import os
import json

# Configuration
config = {
    "access_token": os.getenv("GITHUB_ACCESS_TOKEN"),
    "repository": os.getenv("GITHUB_REPOSITORY"),  # e.g., "owner/repo"
    "start_date": os.getenv("GITHUB_START_DATE", "2024-01-01T00:00:00Z")
}

# Run the Singer tap
subprocess.run([
    "tap-github",
    "--config", "-"
], input=json.dumps(config), text=True)

Save as github.surveilr[singer].py and make it executable:

chmod +x github.surveilr[singer].py

Step 3: Set Environment Variables

export GITHUB_ACCESS_TOKEN="ghp_yourtoken"
export GITHUB_REPOSITORY="microsoft/vscode"
export GITHUB_START_DATE="2024-01-01T00:00:00Z"

Step 4: Initialize Database

surveilr admin init -d project.db

Standard SQLite database.

Step 5: Ingest the Singer Tap

surveilr ingest files -r ./github.surveilr[singer].py -d project.db

surveilr:

Executes the script
Reads the Singer JSONL output
Stores raw data in uniform_resource

Step 6: Transform to SQL Views

surveilr orchestrate adapt-singer -d project.db --stream-prefix github_

This creates SQL views like:

github_commits
github_issues
github_pull_requests
github_users

Now you can query them with standard SQL.

Step 7: Query Your Data

surveilr shell -d project.db

Find all commits from the last 7 days

SELECT
    commit_sha,
    author,
    message,
    timestamp
FROM github_commits
WHERE timestamp > datetime('now', '-7 days')
ORDER BY timestamp DESC;

Track commit activity by author

SELECT
    author,
    COUNT(*) AS commits,
    MIN(timestamp) AS first_commit,
    MAX(timestamp) AS last_commit
FROM github_commits
WHERE timestamp > '2024-01-01'
GROUP BY author
ORDER BY commits DESC;

Find open issues without pull requests

SELECT
    i.number,
    i.title,
    i.created_at
FROM github_issues i
LEFT JOIN github_pull_requests pr
  ON i.number = pr.issue_number
WHERE i.state = 'open'
  AND pr.number IS NULL
ORDER BY i.created_at DESC;

Cross-platform query: Jira issues + GitHub commits

SELECT
    j.key AS jira_ticket,
    j.summary,
    c.commit_sha,
    c.message,
    c.timestamp
FROM jira_issues j
JOIN github_commits c
  ON c.message LIKE '%' || j.key || '%'
WHERE j.status = 'Done'
ORDER BY c.timestamp DESC;

This is impossible with platform APIs alone.

Why Not Just Use API Clients?

You could write Python scripts with requests or use platform SDKs.

But:

API clients don't give you SQL.
You're stuck with JSON responses you have to parse.

surveilr outputs queryable tables.
Standard SQL from day one.

APIs have rate limits.
Hitting them repeatedly during analysis is painful.

surveilr stores data locally.
Query as much as you want, no API calls.

Custom scripts don't compose.
How do you join GitHub data with Jira data in your script?

surveilr stores everything in one database.
Cross-platform joins just work.

Scripts don't track history.
You get a snapshot, then it's gone.

surveilr supports incremental updates.
Singer taps use STATE messages to bookmark progress.

Forensic Project Analysis

Once your project data is queryable, you can investigate patterns:

Find commits made during incident windows

SELECT commit_sha, author, message, timestamp
FROM github_commits
WHERE timestamp BETWEEN '2024-05-10 14:00' AND '2024-05-10 16:00'
ORDER BY timestamp;

Track issue resolution time

SELECT
    key,
    summary,
    julianday(resolved_at) - julianday(created_at) AS days_to_resolve
FROM jira_issues
WHERE status = 'Done'
ORDER BY days_to_resolve DESC;

Find high-priority issues that took too long

SELECT
    key,
    summary,
    priority,
    created_at,
    resolved_at
FROM jira_issues
WHERE priority IN ('High', 'Critical')
  AND julianday(resolved_at) - julianday(created_at) > 30
ORDER BY created_at DESC;

These insights are buried in platform UIs. SQL makes them visible.

Real-World Uses

For Developers

Track your own commit patterns
Find issues assigned to you across platforms
Identify repos with low test coverage
Build custom dashboards with Datasette

For Engineering Managers

Calculate team velocity over time
Track deployment frequency
Analyze code review turnaround
Identify process bottlenecks

For Product Teams

Link customer requests to shipped features
Track feature adoption via API usage
Measure time from idea to deployment
Build product analytics pipelines

For Compliance (Oh, By the Way)

SOC 2: Show complete change management history
Change Control: Link every production change to an approved ticket
Audit Trails: Provide queryable evidence of who changed what when
Separation of Duties: Prove different people committed vs. approved

But the real value is permanent, queryable operational data.

Combine Data Sources

The power comes from joining data across platforms.

GitHub + Jira: Find unlinked work

SELECT
    c.commit_sha,
    c.message,
    c.author
FROM github_commits c
WHERE NOT EXISTS (
    SELECT 1 FROM jira_issues j
    WHERE c.message LIKE '%' || j.key || '%'
)
ORDER BY c.timestamp DESC;

Stripe + Salesforce: Match payments to deals

SELECT
    s.opportunity_name,
    p.amount,
    p.created_at AS payment_date
FROM salesforce_opportunities s
JOIN stripe_payments p
  ON s.stripe_customer_id = p.customer_id
WHERE p.status = 'succeeded'
ORDER BY p.created_at DESC;

This is why SQLite matters—everything is in one database.

Automate Daily Syncs

Create a script to keep your database up to date:

#!/bin/bash
# daily-project-sync.sh

DB="/secure/project-tracking.db"

# Ingest GitHub
surveilr ingest files -r ./github.surveilr[singer].py -d "$DB"

# Ingest Jira
surveilr ingest files -r ./jira.surveilr[singer].py -d "$DB"

# Transform to views
surveilr orchestrate adapt-singer -d "$DB" --stream-prefix github_
surveilr orchestrate adapt-singer -d "$DB" --stream-prefix jira_

echo "✓ Project data sync completed: $(date)"

Run it daily, and your database accumulates history automatically.

Open the Database in Other Tools

Because it's SQLite, you can use any SQLite tool:

Datasette — Instant web UI for your project data
Metabase — Build team dashboards
DuckDB — Fast analytics on SQLite files
Python pandas — pd.read_sql("SELECT * FROM github_commits", conn)
Grafana — Visualize metrics over time
Observable — Create interactive notebooks

You own the database. Analyze it however you want.

Troubleshooting

Tap script not executing?

Make sure it's executable: chmod +x script.surveilr[singer].py
Check that it outputs valid JSONL to stdout

No tables created after adapt-singer?

Verify the stream prefix matches: --stream-prefix github_
Check that the Singer tap output SCHEMA messages

"Module not found" errors?

Install the Singer tap: pip install tap-github
Make sure it's in your PATH

What's Next?

You just turned GitHub, Jira, or other APIs into queryable SQL tables.

Now combine with other data sources:

Query Your File System — Join commits with file changes
Turn Email Into SQL — Correlate issues with email threads

Or explore more:

surveilr.com — Full documentation
Singer Hub — 600+ pre-built taps
Singer Spec — Build custom taps

The Bottom Line

API data shouldn't be trapped in JSON responses.

Platforms give you REST APIs and pagination. That's it.

surveilr gives you SQL.

Want to know:

What changed in your codebase last quarter?
Which issues took longest to resolve?
How deployment frequency correlates with bug reports?
Which team members are most active?

Just write a query.

Your API data is now a SQLite database you own forever.

No API rate limits. No vendor lock-in. Just standard SQL.

Ready to query your project data? Install surveilr and ingest your first API.

Get Started →

Turn Your Email Into a SQL Database

Mukhtar — Tue, 19 May 2026 16:56:44 +0000

Your inbox is data. Query it like data.

The 2-Minute Version:

# Install and ingest your Gmail
brew tap surveilr/tap && brew install surveilr
surveilr admin init -d email.db
surveilr ingest imap \
  -u you@gmail.com \
  -p "your-app-password" \
  -a imap.gmail.com \
  -d email.db

# Query it
surveilr shell -d email.db

-- Find every email from a specific sender
SELECT subject, date, "from"
FROM emails
WHERE "from" LIKE '%@vendor.com%'
ORDER BY date DESC;

Your inbox is now a SQL database. Query it like any other data.

The Problem with Email Search

Gmail and Outlook have search boxes. They work for simple queries.

But try this:

"Show me all emails from Q1 2024 where attachments were over 10MB"
"Find email threads that mention both 'invoice' and 'overdue'"
"List all external recipients I've emailed in the last 6 months"
"Which emails have attachments but no reply?"

Most email clients can't answer these questions.

Here's why:

Email search is designed for finding messages, not analyzing communication patterns.

surveilr turns your inbox into a SQLite database where these queries become trivial.

What surveilr Does

surveilr connects to any IMAP server and extracts everything into SQL tables:

From, To, CC, BCC
Subject lines and dates
Message content and headers
Attachments (with optional extraction)
Thread relationships
Folder structure

The output is standard SQLite. Use any SQL tool you want.

Step 1: Get an App Password

For Gmail:

Go to Google Account → App Passwords
Generate an app password
Copy it (you'll use this instead of your regular password)

Step 2: Create a Database

surveilr admin init -d email.db

Standard SQLite file. Nothing fancy.

Step 3: Ingest Your Email

surveilr ingest imap \
  -d email.db \
  -u your.email@gmail.com \
  -p "your-app-password" \
  -a imap.gmail.com \
  -f "INBOX"

surveilr connects via IMAP and inserts messages into ur_ingest_session_imap_acct_folder_message.

Useful flags:

Filter by status:

--status unread    # Only unread messages
--status starred   # Starred/flagged
--status all       # Everything (default)

Extract attachments:

--extract-attachments uniform-resource  # Store in database

Show progress:

--progress  # Display download progress

Limit messages:

-b 1000  # Process up to 1000 messages

Step 4: Query Your Inbox

Open the SQL shell:

surveilr shell -d email.db

Now you can ask questions that Gmail search can't answer.

Find all emails from a specific domain

SELECT
    "from",
    subject,
    date
FROM ur_ingest_session_imap_acct_folder_message
WHERE "from" LIKE '%@vendor.com%'
ORDER BY date DESC;

Note: from is quoted because it's a SQL keyword.

Find emails with large attachments

SELECT
    m.subject,
    m."from",
    m.date,
    a.name AS attachment_name,
    a.size_bytes / 1024 / 1024 AS size_mb
FROM ur_ingest_session_imap_acct_folder_message m
JOIN ur_ingest_session_attachment a
  ON m.message_id = a.message_id
WHERE a.size_bytes > 10485760
ORDER BY a.size_bytes DESC;

Track email volume over time

SELECT
    DATE(date) AS day,
    COUNT(*) AS emails
FROM ur_ingest_session_imap_acct_folder_message
WHERE date > date('now', '-30 days')
GROUP BY DATE(date)
ORDER BY day;

Find emails you sent but never got a reply to

SELECT
    subject,
    "to",
    date
FROM ur_ingest_session_imap_acct_folder_message
WHERE "from" LIKE '%you@company.com%'
  AND message_id NOT IN (
    SELECT in_reply_to
    FROM ur_ingest_session_imap_acct_folder_message
    WHERE in_reply_to IS NOT NULL
  )
ORDER BY date DESC;

These queries are impossible in Gmail.

List Available Folders

Not sure what folders you have?

surveilr ingest imap \
  -u your.email@gmail.com \
  -p "your-app-password" \
  -a imap.gmail.com \
  --list-folders

Then ingest specific folders:

surveilr ingest imap \
  -u your.email@gmail.com \
  -p "your-app-password" \
  -a imap.gmail.com \
  -f "[Gmail]/Sent Mail" \
  -d email.db

Export Results

Need to share findings? Export as JSON:

surveilr shell -d email.db --cmd "
SELECT subject, \"from\", date
FROM ur_ingest_session_imap_acct_folder_message
WHERE subject LIKE '%invoice%'
" --output json > invoices.json

Or CSV for spreadsheet analysis:

surveilr shell -d email.db --cmd "
SELECT * FROM ur_ingest_session_imap_acct_folder_message
" --output csv > all-emails.csv

Why Not Just Export to CSV?

You could export your inbox to CSV or use a backup tool.

But:

Exports are point-in-time snapshots.
You can't incrementally update them.

surveilr supports continuous ingestion.
Run it weekly and accumulate history over time.

CSVs don't preserve relationships.
How do you link attachments to messages?

surveilr uses proper foreign keys.
Attachments reference their parent messages.

CSVs are hard to query.
You're back to scripting with pandas or awk.

surveilr gives you SQL.
Join, filter, aggregate—all standard SQL.

Forensic Email Analysis

Once your inbox is queryable, you can investigate patterns:

Find emails that mention specific keywords across years

SELECT subject, "from", date
FROM emails
WHERE (subject LIKE '%confidential%' OR content LIKE '%confidential%')
  AND date > '2020-01-01'
ORDER BY date DESC;

Track who you email most frequently

SELECT
    "to",
    COUNT(*) AS email_count
FROM emails
WHERE "from" LIKE '%you@company.com%'
GROUP BY "to"
ORDER BY email_count DESC
LIMIT 20;

Find emails sent on weekends or late at night

SELECT subject, "to", date
FROM emails
WHERE strftime('%w', date) IN ('0', '6')  -- Sunday or Saturday
   OR CAST(strftime('%H', date) AS INTEGER) > 22
   OR CAST(strftime('%H', date) AS INTEGER) < 6
ORDER BY date DESC;

These insights are buried in your inbox. SQL makes them visible.

Real-World Uses

For Developers

Track bug reports sent via email
Find API key or credential leaks in sent mail
Analyze support ticket volume
Search across years of technical discussions

For Operations

Audit external communications
Track alert emails and response times
Find recurring issues mentioned in email
Monitor vendor correspondence

For Legal/E-Discovery

Respond to discovery requests with SQL queries
Prove communication timelines
Find all emails related to a specific matter
Extract conversations by date range

For Compliance (Oh, By the Way)

HIPAA: Track communications containing PHI
SOX: Maintain 7-year email records with queryable evidence
GDPR: Respond to "right to access" requests with SQL
E-Discovery: Legal hold and litigation support

But the real value is permanent, queryable communication history.

Open the Database in Other Tools

Because it's SQLite, you can use any SQLite tool:

DB Browser for SQLite — Visual inspection
Datasette — Instant web UI
Python pandas — pd.read_sql("SELECT * FROM emails", conn)
DuckDB — Fast analytical queries
Metabase — Build dashboards
R / tidyverse — Data analysis workflows

You own the database. Query it however you want.

Security Best Practices

Use App Passwords, Not Your Main Password

Never put your main email password in scripts. Use app-specific passwords that you can revoke.

Store Passwords Securely

Use environment variables:

export IMAP_PASSWORD="your-app-password"
surveilr ingest imap -u you@gmail.com -p "$IMAP_PASSWORD" -a imap.gmail.com

Or use a password manager's CLI:

surveilr ingest imap -u you@gmail.com -p "$(pass show gmail/app-password)" -a imap.gmail.com

Encrypt the Database

SQLite databases can be encrypted. Use tools like SQLCipher if you need encryption at rest.

Troubleshooting

"Authentication failed"?

Make sure you're using an app password, not your main password
Verify IMAP is enabled in your email settings
Check that 2FA isn't blocking IMAP access

"Connection timed out"?

Verify the IMAP server address
Check if your firewall blocks port 993
Some providers require enabling "less secure app access"

"No messages found"?

Use --list-folders to see available folders
Folder names are case-sensitive
Gmail uses [Gmail]/All Mail not All Mail

What's Next?

You just turned your inbox into a queryable database.

Now combine it with other data:

Query Your File System — Join emails with file metadata
Extract API Data — Add GitHub, Jira, or other sources

Or explore more:

surveilr.com — Full documentation
Installation Guide — Other platforms

The Bottom Line

Your inbox shouldn't be a black box.

Email clients give you search bars and folders. That's it.

surveilr gives you SQL.

Want to know:

What you emailed last quarter?
Who sends you the most messages?
Which threads have attachments but no replies?
How your communication patterns changed over time?

Just write a query.

Your email is now a SQLite database you own forever.

No export limits. No API restrictions. Just standard SQL.

Ready to query your inbox? Install surveilr and ingest your first folder.

Get Started →

Turn Your File System Into a SQL Database

Mukhtar — Sun, 17 May 2026 17:05:29 +0000

Stop grepping. Start querying.

The 2-Minute Version:

# Install and scan your Documents folder
brew tap surveilr/tap && brew install surveilr
surveilr admin init -d fs.db
surveilr ingest files -r ~/Documents -d fs.db

# Query it
surveilr shell -d fs.db

-- Find every PDF modified this month
SELECT file_path, size_bytes, last_modified
FROM files
WHERE extension = 'pdf' AND last_modified > date('now', 'start of month');

That's it. Your filesystem is now a SQLite database you can query forever.

The Problem with grep and find

You've been there:

find ~/Documents -name "*.pdf" -mtime -30 | xargs ls -lh

It works. Until you need to:

Track changes over time
Join file metadata with other data
Build complex filters
Share results in a structured format
Query the same data six months later

Then your one-liner becomes a 200-line Bash script nobody can maintain.

Here's a different approach:

Turn your filesystem into a permanent SQLite database you can keep querying with standard SQL.

No scripts. No parsing. Just tables.

What surveilr Does

surveilr scans directories and captures everything into SQLite:

File paths and names
Sizes and timestamps
Content hashes (SHA-256)
Extensions and MIME types
Permissions and ownership

The output is a standard .db file you can:

Query with any SQLite tool
Open in DB Browser, DBeaver, DataGrip
Read with Python pandas or R
Keep forever (SQLite files last decades)
Share with teammates (just copy the file)

Step 1: Create a Database

surveilr admin init -d my-files.db

This creates an empty SQLite database. That's it. No configuration, no setup, no servers.

You can open this file in any SQLite client right now and see the schema.

Step 2: Scan a Directory

surveilr ingest files -r ~/Documents -d my-files.db

surveilr walks the directory tree and inserts metadata into two tables:

uniform_resource — Core metadata (hash, size, timestamps)
ur_ingest_session_fs_path_entry — File-specific data (paths, extensions)

Pro tip: Run multiple times to track changes over time. Each scan creates a new session.

Step 3: Query Like You Mean It

Open the SQL shell:

surveilr shell -d my-files.db

Now you can run queries that would be painful in Bash.

Find all PDFs larger than 5MB

SELECT
    e.file_path_abs,
    e.file_basename,
    u.size_bytes / 1024 / 1024 AS size_mb
FROM uniform_resource u
JOIN ur_ingest_session_fs_path_entry e
  ON u.uniform_resource_id = e.uniform_resource_id
WHERE e.file_extn = 'pdf'
  AND u.size_bytes > 5242880
ORDER BY u.size_bytes DESC;

Track file modifications over time

Run multiple scans, then query across sessions:

SELECT
    e.file_basename,
    COUNT(DISTINCT u.content_digest) AS versions,
    MIN(u.last_modified_at) AS first_seen,
    MAX(u.last_modified_at) AS last_modified
FROM uniform_resource u
JOIN ur_ingest_session_fs_path_entry e
  ON u.uniform_resource_id = e.uniform_resource_id
WHERE e.file_basename = 'report.docx'
GROUP BY e.file_basename;

Find duplicate files by content hash

SELECT
    u.content_digest,
    COUNT(*) AS copies,
    SUM(u.size_bytes) / 1024 / 1024 AS total_mb_wasted
FROM uniform_resource u
JOIN ur_ingest_session_fs_path_entry e
  ON u.uniform_resource_id = e.uniform_resource_id
GROUP BY u.content_digest
HAVING copies > 1
ORDER BY total_mb_wasted DESC;

Find files that changed in the last 7 days

SELECT
    e.file_path_abs,
    u.last_modified_at,
    u.size_bytes
FROM uniform_resource u
JOIN ur_ingest_session_fs_path_entry e
  ON u.uniform_resource_id = e.uniform_resource_id
WHERE u.last_modified_at > datetime('now', '-7 days')
ORDER BY u.last_modified_at DESC;

These queries work on any SQLite database—forever.

Transform Structured Files Into Queryable Tables

surveilr doesn't just capture file metadata—it can extract and transform content from structured files.

This is where it gets interesting.

Transform CSV Files Into SQL Tables

Got CSV files? Turn them into queryable tables automatically:

surveilr orchestrate transform-csv -d my-files.db

Every CSV file you ingested becomes a SQL table. If you scanned a directory with sales-2024.csv, you can now query it:

SELECT * FROM sales_2024 WHERE revenue > 10000;

No manual schema definition. No import scripts. Just automatic table creation.

Extract Data from HTML Files with CSS Selectors

Ingested HTML files? Extract specific elements with CSS selectors:

# Extract all paragraphs inside divs
surveilr orchestrate transform-html -d my-files.db \
  --css-select "paragraphs:div > p"

# Extract all links
surveilr orchestrate transform-html -d my-files.db \
  --css-select "links:a[href]"

# Extract headers
surveilr orchestrate transform-html -d my-files.db \
  --css-select "headers:h1, h2, h3"

The extracted data goes into JSON format you can query:

SELECT
    file_path,
    json_extract(transformed_content, '$.paragraphs') AS paragraphs
FROM uniform_resource
WHERE file_extn = 'html';

Real-world use case: Scrape product data from saved HTML pages, extract structured content from documentation, parse scraped web pages.

Transform Markdown Files

Extract specific sections from Markdown files:

# Extract all headers
surveilr orchestrate transform-markdown -d my-files.db \
  --md-select "headers:# *"

# Extract code blocks
surveilr orchestrate transform-markdown -d my-files.db \
  --md-select "code:```
{% endraw %}
*
{% raw %}
```"

Real-world use case: Index documentation, extract code examples from README files, parse technical notes.

Transform XML Files

XML files get automatically parsed into queryable JSON:

surveilr orchestrate transform-xml -d my-files.db

Then query XML content with SQLite's JSON functions:

SELECT
    file_path,
    json_extract(transformed_content, '$.root.item[0].name') AS item_name
FROM uniform_resource
WHERE file_extn = 'xml';

Real-world use case: Parse configuration files, process exported data from tools, analyze XML logs.

Why This Matters

Most file indexing tools just capture metadata.

surveilr extracts and transforms content so you can query inside your files.

This means you can:

Query values within CSV files, not just the file existence
Search for specific HTML elements across hundreds of pages
Extract code snippets from all Markdown docs
Parse XML configs and query their structure

Example workflow:

# 1. Ingest a directory of mixed files
surveilr ingest files -r ~/project-docs -d docs.db

# 2. Transform CSVs into tables
surveilr orchestrate transform-csv -d docs.db

# 3. Extract headers from Markdown
surveilr orchestrate transform-markdown -d docs.db --md-select "headers:# *"

# 4. Query across file types
surveilr shell -d docs.db

-- Find all CSV files with revenue data
SELECT * FROM sales_data WHERE revenue > 50000;

-- Find all Markdown headers mentioning "API"
SELECT file_path, json_extract(transformed_content, '$.headers')
FROM uniform_resource
WHERE file_extn = 'md'
  AND json_extract(transformed_content, '$.headers') LIKE '%API%';

You just turned a directory of mixed files into a queryable knowledge base.

Watch Mode: Continuous Indexing

Want surveilr to automatically re-index when files change?

surveilr ingest files -r ~/Documents -d my-files.db --watch

surveilr monitors the directory and updates the database in real-time.

Press Ctrl+C to stop.

Export Results

Need to share findings? Export as JSON, CSV, or Markdown:

surveilr shell -d my-files.db --cmd "
SELECT file_path, size_bytes, last_modified
FROM files
WHERE extension = 'pdf'
" --output json > pdfs.json

Or save queries as reusable SQL files:

cat > find-large-files.sql <<'EOF'
SELECT file_path, size_bytes / 1024 / 1024 AS size_mb
FROM files
WHERE size_bytes > 10485760
ORDER BY size_bytes DESC
LIMIT 20;
EOF

surveilr shell -d my-files.db find-large-files.sql

Why Not Just Use Scripts?

You could write a Python script to walk directories and output JSON.

But:

Scripts have no schema.
Next month, you won't remember what fields you captured or what format you used.

surveilr creates stable SQL tables.
The schema is self-documenting.

Scripts don't track history.
You get a point-in-time snapshot, then it's gone.

surveilr accumulates sessions over time.
Run it weekly and query across months of history.

Scripts don't compose.
How do you join your file scan with your email data?

surveilr stores everything in one database.
Cross-domain joins just work.

Forensic Investigation Workflows

Once your filesystem is queryable, you can ask surprising questions:

Find files that disappeared

-- Files that existed in session 1 but not session 2
SELECT file_path
FROM files WHERE session_id = 'session_1'
EXCEPT
SELECT file_path
FROM files WHERE session_id = 'session_2';

Track when specific files were modified

SELECT file_basename, last_modified_at, size_bytes
FROM files
WHERE file_basename LIKE '%contract%'
ORDER BY last_modified_at DESC;

Find files created during a specific time window

-- Files created during a deployment window
SELECT file_path, last_modified
FROM files
WHERE last_modified BETWEEN '2024-05-10 14:00' AND '2024-05-10 15:00';

These queries are impossible with find and grep.

Real-World Uses

For Developers

Track which files changed before/after a bug appeared
Find orphaned build artifacts
Identify large files bloating repos
Build custom file search tools

For Operations

Audit log file growth over time
Track config file changes
Find stale backups
Monitor disk usage trends

For Security Teams

Establish filesystem baselines
Detect unauthorized file modifications
Track file access patterns
Identify anomalous file creation

For Compliance (Oh, By the Way)

Document where regulated data lives (HIPAA PHI, GDPR PII)
Prove file retention policies
Provide audit evidence
Respond to data access requests

But the real power is permanent queryability.

Open the Database in Other Tools

Because surveilr outputs standard SQLite, you can use any SQLite tool:

DB Browser for SQLite — Visual database inspector
DBeaver — Universal database client
Datasette — Instant web UI for your database
Python pandas — pd.read_sql_query(sql, 'sqlite:///my-files.db')
DuckDB — Fast analytical queries on SQLite files
VS Code SQLite extensions — Query in your editor

You own the .db file. Use whatever tools you want.

Troubleshooting

"Permission denied" errors?
Run with appropriate permissions or scan a different directory.

Database getting too large?
Use --dry-run to preview:

surveilr ingest files -r ~/Documents --dry-run

Want to see what happened?
Add --stats to show ingestion summary:

surveilr ingest files -r ~/Documents -d my-files.db --stats

What's Next?

You just turned your filesystem into a queryable database.

Now combine it with other data sources:

Turn Email Into SQL — Query Gmail/Outlook alongside files
Extract API Data — Add GitHub, Jira, or 600+ other sources

Or explore more:

surveilr.com — Full documentation
Installation Guide — Other install methods

The Bottom Line

Your filesystem should be queryable, not just searchable.

find and grep are great for one-off questions.

But when you need:

Historical tracking
Complex queries
Structured output
Composability
Permanence

...you want SQL.

surveilr turns your filesystem into a SQLite database you own forever.

No scripts. No parsing. Just queries.

Ready to query your filesystem? Install surveilr and run your first scan.

Get Started →

Turn Anything Into a Queryable SQLite Database

Mukhtar — Sun, 17 May 2026 17:02:38 +0000

grep, jq, ETL, and forensic indexing collapsed into one local SQL primitive

Your audit trail should be a database you own, not a SaaS UI you rent.

surveilr turns your files, emails, and APIs into standard SQLite databases you can query forever—with any tool, offline, on your machine.

No cloud. No dashboards. Just SQL.

See It Work in 2 Minutes

Install surveilr and immediately query your filesystem:

# Install (macOS/Linux)
brew tap surveilr/tap && brew install surveilr

# Create a database and scan your Documents folder
surveilr admin init -d my-data.db
surveilr ingest files -r ~/Documents -d my-data.db

# Open SQL shell and query
surveilr shell -d my-data.db

Now run surprisingly powerful queries:

-- Find every PDF modified after a specific date
SELECT file_path, last_modified, size_bytes
FROM files
WHERE extension = 'pdf'
  AND last_modified > '2024-05-01';

-- Track file changes over time
SELECT file_basename, COUNT(*) as versions
FROM files
GROUP BY file_basename
HAVING versions > 1;

-- Find orphaned large files
SELECT file_path, size_bytes / 1024 / 1024 AS size_mb
FROM files
WHERE size_bytes > 10485760
ORDER BY size_bytes DESC;

That's it. You just turned your filesystem into a queryable database you can keep investigating.

What is surveilr?

surveilr is a local-first universal SQL layer that ingests operational data and outputs standard SQLite databases.

It's not a platform. It's not a dashboard. It's an ingestion layer that speaks SQL.

Core Capabilities

📂 File system indexing — Turn any directory into queryable metadata
🔄 Content transformation — CSV→SQL tables, HTML→JSON (CSS selectors), Markdown/XML→queryable data
📧 Email ingestion — IMAP to SQLite (Gmail, Outlook, any server)
🔌 API extraction — 600+ Singer taps (GitHub, Jira, Salesforce, databases)
🔍 Standard SQL — No custom DSL, just SQLite
🏠 Local-first — Everything runs on your machine, offline-capable
🔒 You own the data — Portable .db files you control forever

Why This Matters

Most operational data ends up in one of three dead ends:

SaaS dashboards you can't query your way
One-off scripts that become unmaintainable
CSV exports that lose context over time

surveilr gives you a different option: permanent, queryable SQLite databases.

Years from now, you'll still be able to open that .db file with any SQLite client and run new queries you haven't thought of yet.

The SQLite Advantage

surveilr doesn't invent a new database format. It uses SQLite—the world's most deployed database.

You already trust SQLite. It's in your phone, your browser, your laptop's OS.

What surveilr adds is disciplined ingestion patterns that turn messy operational data into clean, queryable tables.

Inspectable: Open any .db file with DB Browser, VS Code, or sqlite3
Durable: SQLite files last decades
Interoper able: Works with Datasette, DuckDB, pandas, Observable, Grafana
Portable: One file, zero dependencies
Permanent: Your data doesn't disappear when a vendor shuts down

This is the opposite of SaaS data lock-in.

Queries That Make You Go "Wait, I Can Do THAT?"

The power comes from cross-domain queries you can't run anywhere else.

Find documents mentioned in emails after they were modified

SELECT f.file_path, e.subject, e.date, f.last_modified
FROM files f
JOIN emails e ON e.subject LIKE '%' || f.file_basename || '%'
WHERE f.last_modified < e.date
ORDER BY e.date DESC;

Track all GitHub commits made within 24 hours of a production incident

SELECT c.commit_sha, c.author, c.message, c.timestamp
FROM github_commits c
WHERE c.timestamp BETWEEN
  (SELECT incident_time - interval '24 hours' FROM incidents WHERE id = 'INC-123')
  AND
  (SELECT incident_time FROM incidents WHERE id = 'INC-123');

Find invoices discussed in email without matching purchase orders in Jira

SELECT e.subject, e.from, e.date
FROM emails e
WHERE e.subject LIKE '%invoice%'
  AND NOT EXISTS (
    SELECT 1 FROM jira_issues j
    WHERE j.summary LIKE '%' || SUBSTR(e.subject, INSTR(e.subject, 'INV-'), 10) || '%'
  );

These aren't theoretical examples. These are real forensic workflows you can build.

Three Practical Guides

Guide 1: Query Your File System

Read the full guide →

Scan directories and query file metadata with SQL. Plus: Transform CSVs into SQL tables, extract data from HTML with CSS selectors, parse Markdown and XML into queryable JSON.

2-minute win: Find all PDFs modified in the last 30 days
Bonus: Turn CSV files into queryable SQL tables automatically

Guide 2: Turn Email Into SQL

Read the full guide →

Ingest Gmail/Outlook via IMAP. Query conversations, track threads, extract attachments, search across years of email history.

2-minute win: Find all emails from a specific sender mentioning "invoice"

Guide 3: Extract API Data

Read the full guide →

Use Singer taps to pull data from GitHub, Jira, GitLab, Salesforce, or 600+ other sources. Join across platforms.

2-minute win: Query all GitHub commits from the last 7 days

Why Not Just Write a Script?

You could. But:

Scripts become dead ends.
Six months later, you can't remember what format you used or where you saved the output.

surveilr outputs permanent, standard SQLite databases.
Open them years later with any SQL tool and keep querying.

Scripts don't compose.
You can't easily join your email script's output with your file scan script's output.

surveilr stores everything in one queryable database.
Cross-domain joins just work.

Scripts have no schema.
You're parsing JSON with jq and hoping the structure doesn't change.

surveilr normalizes data into stable SQL tables.
Query with confidence.

Ecosystem Integration

surveilr isn't the destination. It's the ingestion layer.

Once your data is in SQLite, you can use any tool in the SQLite ecosystem:

Datasette — Instant web UI for your databases
DuckDB — Fast analytical queries on SQLite files
DB Browser for SQLite — Visual database inspector
VS Code SQLite extensions — Query in your editor
Python pandas — Read SQLite directly into dataframes
Grafana — Build dashboards from SQLite
Observable — Interactive notebooks with SQL

You own the database. Use whatever tools you want.

Local-First. No Cloud. You Own It.

This is a big deal in 2026.

Most "compliance platforms" hide your own data behind proprietary dashboards.

surveilr gives you raw SQL access to everything.

✅ All data stays on your machine — No upload, no sync, no cloud dependency
✅ Works completely offline — Internet not required
✅ Inspectable — Open the .db file with any SQLite client
✅ Portable — Copy the file, query it anywhere
✅ No vendor lock-in — Standard SQLite format, not proprietary
✅ Privacy-first — Your data never leaves your control

If you're tired of SaaS platforms that:

Charge per seat
Lock your data behind APIs
Disappear when the startup shuts down
Require constant internet connectivity

...then surveilr is for you.

Oh, By the Way: Compliance Teams Love This Too

If you work in healthcare, finance, or regulated industries, surveilr happens to be perfect for:

HIPAA audits — Track where PHI files are stored
SOX compliance — Maintain 7-year email records with queryable evidence
GDPR requests — Respond to "right to access" with SQL queries
SOC 2 audits — Show complete change management history

But that's a side effect of the real value: permanent, queryable operational data you control.

Forensic Curiosity

Once you start using surveilr, you'll find yourself asking new questions:

What files were deleted but are still referenced in emails?
Which documents were modified right before a deployment?
What code changes correlate with customer support tickets?
Which team members touched files in a specific directory over the last year?
What attachments were sent externally that match internal file hashes?

These questions are impossible to answer with dashboards.

But with SQL, they're just queries.

Installation

macOS / Linux

brew tap surveilr/tap && brew install surveilr

Verify

surveilr --version
surveilr doctor  # Check environment

For other platforms, see the installation guide.

Run This Now

Pick your 2-minute win:

Option 1: Query your filesystem

surveilr admin init -d fs.db
surveilr ingest files -r ~/Documents -d fs.db

Option 2: Query your email

surveilr admin init -d email.db
surveilr ingest imap -u you@gmail.com -p "app-password" -a imap.gmail.com -d email.db

Option 3: Query GitHub (Ingest Singer taps)

surveilr admin init -d github.db
surveilr ingest files -r ./github-tap-script.py -d github.db
surveilr orchestrate adapt-singer (convert data to views)

Then open the .db file in any SQLite tool and start exploring.

Learn More

surveilr.com — Official website and documentation
Installation Guide — Download and install
Report Issues — Bug reports and feature requests

The Bottom Line

Your operational data—files, emails, API responses—should be:

Queryable (SQL, not grep)
Permanent (SQLite, not CSVs)
Yours (local, not SaaS)
Composable (join across domains)
Inspectable (open in any tool)

That's what surveilr gives you.

It's not compliance software. It's not enterprise governance.

It's grep, jq, ETL, and forensic indexing collapsed into one local SQL primitive you can build on forever.

Ready to own your data? Install surveilr and query something surprising.

Get Started →

Implementing OpenDAL with Filesystem (FS) In Rust

Mukhtar — Sat, 19 Jul 2025 18:55:47 +0000

Introduction to OpenDAL with SQLite Virtual Tables

OpenDAL is a powerful and unified data access layer that provides an abstraction for different storage backends such as local filesystems, cloud storage, and object stores. It simplifies file and metadata operations by offering a unified API, allowing seamless interaction with different storage solutions.

This guide explains the concepts behind integrating OpenDAL with SQLite virtual tables, allowing you to query filesystem metadata using SQL. The code examples demonstrate key concepts rather than complete implementations.

Core Concepts

1. OpenDAL Operator

The foundation of any OpenDAL integration is the Operator - your interface to the storage backend.

Concept: Create a configured operator for your storage type

// Conceptual example - actual implementation needs error handling
let fs_builder = Fs::default().root("/");
let operator = Operator::new(fs_builder)?.finish();

Key Ideas:

The operator abstracts away storage-specific details
Different services (Fs, S3, Azure, etc.) use the same Operator interface
Configuration happens through service-specific builders

2. SQLite Virtual Table Architecture

Virtual tables in SQLite allow you to present non-SQL data as queryable tables.

Concept: Bridge between OpenDAL and SQLite

// Conceptual structure - real implementation much more complex
struct FileSystemTable {
    base: sqlite3_vtab,           // Required SQLite base
    operator: Rc<Operator>,       // Your OpenDAL operator
}

Key Ideas:

Virtual tables implement a specific SQLite interface
They translate SQL queries into storage operations
The table schema defines what data is queryable

3. Schema Design

Design your virtual table schema to expose useful file metadata.

Concept: Map file attributes to SQL columns

CREATE TABLE filesystem_view(
    name TEXT,                    -- File name
    path TEXT,                    -- Full path
    last_modified TEXT,           -- When file was changed
    content BLOB,                 -- File contents (lazy-loaded)
    size INTEGER,                 -- File size in bytes
    content_type TEXT,            -- MIME type
    digest TEXT,                  -- File hash/checksum
    arg_path HIDDEN              -- Query parameter (hidden column)
);

Key Ideas:

Hidden columns can accept query parameters
BLOB columns can hold binary data
Schema should balance utility with performance

4. Query Translation Flow

The magic happens in translating SQL queries into OpenDAL operations.

Concept: SQL WHERE clause becomes OpenDAL list operations

// Conceptual flow - real implementation involves cursor management
fn handle_query_with_path_filter(path: &str) -> Vec<FileMetadata> {
    // 1. Use OpenDAL to list files in the specified path
    let lister = operator.lister_with(path)
        .metakey(Metakey::ContentLength | Metakey::LastModified)
        .call()?;

    // 2. Convert OpenDAL entries to your table format
    let mut results = Vec::new();
    for entry in lister {
        if entry.metadata().mode() == EntryMode::FILE {
            results.push(convert_to_table_row(entry));
        }
    }

    results
}

Key Ideas:

SQL filters become OpenDAL query parameters
Metadata is fetched on-demand to avoid performance issues
Results are converted to SQLite-compatible format

5. Lazy Loading Strategy

For performance, expensive operations (like reading file contents) should be lazy.

Concept: Only load data when specifically requested

// Conceptual approach to lazy loading
fn get_column_value(&self, column: i32) -> SqliteValue {
    match column {
        0 => SqliteValue::Text(self.file_name.clone()),
        1 => SqliteValue::Text(self.file_path.clone()),
        2 => SqliteValue::Integer(self.file_size),
        3 => {
            // Expensive operation - only do when column is requested
            let content = self.operator.read(&self.file_path)?;
            SqliteValue::Blob(content)
        }
        _ => SqliteValue::Null,
    }
}

Key Ideas:

Separate metadata operations from content operations
Use OpenDAL's stat() for metadata, read() for content
Only perform expensive operations when columns are actually selected

6. Error Handling Patterns

Bridge OpenDAL errors to SQLite errors appropriately.

Concept: Convert storage errors to SQL errors

// Conceptual error mapping
fn map_opendal_error(err: OpenDalError) -> SqliteError {
    match err.kind() {
        ErrorKind::NotFound => SqliteError::NotFound,
        ErrorKind::PermissionDenied => SqliteError::Auth,
        ErrorKind::Unexpected => SqliteError::Internal,
        _ => SqliteError::Internal,
    }
}

Key Ideas:

OpenDAL errors need translation to SQLite error codes
Some errors should be handled gracefully (missing files)
Others should propagate up to the SQL layer

Usage Patterns

Once implemented, your virtual table enables powerful queries:

-- List all files in a directory
SELECT name, size FROM filesystem_view WHERE arg_path = '/documents';

-- Find large files
SELECT path, size FROM filesystem_view 
WHERE arg_path = '/media' AND size > 1000000;

-- Search by file type (if you implement content_type detection)
SELECT name, path FROM filesystem_view 
WHERE arg_path = '/projects' AND content_type LIKE 'text/%';

Benefits of This Approach

Why combine OpenDAL with SQLite virtual tables?

Unified Query Interface: Use SQL to query any storage backend
Flexibility: Same queries work across local files, S3, Azure, etc.
Integration: Easy to embed in existing SQLite-based applications
Performance: SQLite's query optimization benefits your file operations
Ecosystem: Leverage SQLite's rich ecosystem of tools and extensions

Next Steps

To implement this approach:

Study SQLite's virtual table interface documentation
Explore OpenDAL's service implementations for your storage needs
Design your schema based on your specific use cases
Implement the virtual table interface with proper error handling
Add performance optimizations like caching and pagination

This pattern opens up powerful possibilities for unified data access across different storage systems while maintaining the familiar SQL interface.

Conclusion

This guide demonstrated how to use OpenDAL for filesystem operations in Rust and integrate it with SQLite virtual tables. By leveraging OpenDAL, you can seamlessly interact with local storage while maintaining a structured query interface via SQLite.

For more details, check out OpenDAL's documentation.

Getting Started with Rust in 2025: Why Now Is a Great Time to Learn Rust 🦀

Mukhtar — Sun, 13 Jul 2025 20:48:12 +0000

Getting Started with Rust in 2025 🦀

Rust has been growing steadily over the years, and in 2025, it's more relevant than ever. If you've been on the fence about learning it, now's the perfect time to jump in.

🧠 Why Learn Rust?

Rust is a modern systems programming language focused on:

Performance (like C/C++)
Safety (no nulls, no data races)
Modern developer experience (great tooling, compiler messages)

Companies like Microsoft, Amazon, Cloudflare, and Dropbox use Rust in production. It's built to help you write fast, reliable, and maintainable software.

🌟 What Makes Rust Special?

Memory safety without garbage collection
Fearless concurrency – build multi-threaded apps without race conditions
Helpful compiler – the compiler actually teaches you!
Powerful tooling – cargo, clippy, rustfmt, and more
Zero-cost abstractions – safety and performance, no tradeoffs

If you've dealt with C/C++ bugs or hit performance walls in Python or JavaScript, Rust is the best of both worlds.

🆚 Rust vs Other Languages

Rust sits in a unique position compared to other programming languages:

Performance – As fast as C/C++ but much safer
Memory Management – Manual control without the crashes
Type Safety – Catches bugs at compile time, not runtime
Concurrency – Built-in support for safe multi-threading
Learning Curve – Steeper initially, but pays off long-term

Unlike garbage-collected languages, Rust gives you control. Unlike manual memory management languages, Rust prevents common bugs.

🚀 Code Examples

Hello World

Every Rust program starts with a main() function. This is your entry point!

// This is the main function - where your program starts
fn main() {
    // println! is a macro that prints to the console
    println!("Hello, world!");
}

Variables and Types

Rust is statically typed but can often infer types. Variables are immutable by default - you need mut to change them.

fn main() {
    // Variables are immutable by default
    let name = "Alice";           // string slice (&str)
    let age: u32 = 30;           // unsigned 32-bit integer
    let mut score = 100;         // 'mut' makes it mutable (changeable)

    // We can change mutable variables
    score += 50;

    // {} are placeholders for variables in println!
    println!("{} is {} years old with a score of {}", name, age, score);
}

Functions

Functions in Rust are defined with fn. The last expression (without semicolon) is the return value.

// Function that takes two i32 parameters and returns an i32
fn add(a: i32, b: i32) -> i32 {
    a + b  // no semicolon = this is the return value
}

fn main() {
    // Call the function and store the result
    let result = add(5, 3);
    println!("5 + 3 = {}", result);
}

Ownership (Rust's Superpower)

Ownership is Rust's way of managing memory safely. Each value has one owner, and when the owner goes out of scope, the value is dropped.

fn main() {
    // Create a String on the heap
    let s1 = String::from("hello");

    // This MOVES s1 to s2. s1 is no longer valid!
    let s2 = s1;

    // This would cause a compile error because s1 was moved
    // println!("{}", s1); 

    // Only s2 is valid now
    println!("{}", s2);
}

Borrowing

Borrowing lets you use a value without taking ownership. Think of it like borrowing a book - you can read it, but you have to give it back!

// This function borrows a String reference (&String)
// It doesn't take ownership, just looks at the data
fn print_length(s: &String) {
    println!("Length: {}", s.len());
}

fn main() {
    let my_string = String::from("hello");

    // Pass a reference (&) to the function
    print_length(&my_string);

    // We can still use my_string because we only borrowed it
    println!("{}", my_string);
}

Structs & Methods

Structs are like classes in other languages - they group related data together. You can add methods to them using impl blocks.

// Define a struct (like a class in other languages)
struct Person {
    name: String,
    age: u32,
}

// Implementation block - where we define methods
impl Person {
    // Associated function (like a constructor)
    fn new(name: String, age: u32) -> Person {
        Person { name, age }  // shorthand for name: name, age: age
    }

    // Method that takes &self (borrows the instance)
    fn greet(&self) {
        println!("Hi, I'm {}", self.name);
    }
}

fn main() {
    // Create a new Person instance
    let person = Person::new("Bob".to_string(), 25);

    // Call the method
    person.greet();
}

Error Handling

Rust doesn't have exceptions. Instead, it uses Result to handle errors explicitly. This forces you to deal with potential failures.

// Function returns Result<T, E> - either Ok(value) or Err(error)
fn divide(a: f64, b: f64) -> Result<f64, String> {
    if b == 0.0 {
        // Return an error
        Err("Cannot divide by zero".to_string())
    } else {
        // Return the successful result
        Ok(a / b)
    }
}

fn main() {
    // Use match to handle both success and error cases
    match divide(10.0, 2.0) {
        Ok(result) => println!("Result: {}", result),
        Err(error) => println!("Error: {}", error),
    }
}

🛠️ Getting Started

Install Rust: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Create new project: cargo new my_project
Build and run: cargo run
Read "The Rust Book" online for free

🔥 What You Can Build

Web servers (Actix, Rocket frameworks)
Command-line tools (ripgrep, bat)
Game engines (Bevy)
Blockchain projects (Solana, Polkadot)
WebAssembly applications
Operating systems (Redox OS)

💡 Why 2025 is Perfect

Mature ecosystem – tons of quality crates
Industry adoption – major companies using it in production
Great learning resources – excellent documentation and community
Job market – high demand, good salaries
Future-proof – language designed for next-generation computing

Rust's combination of performance, safety, and growing ecosystem makes it an excellent choice for modern development in 2025!