Build a Data Science Query Language in Python using Lark
What if you could write something like this:
DATA [1, 2, 3, 4, 5]
SUM
MEAN
STD
…and have it behave like a mini data science engine?
In this tutorial, we’ll build a **Domain-Specific Language (DSL)** for data analysis using:
- Python
- Lark (parser library)
- NumPy
---
# What Are We Building?
We are creating a **custom query language** that:
- Accepts a dataset
- Runs statistical commands
- Prints results
---
# Step 1: Install Dependencies
```bash
pip install lark numpy
Step 2: Define the Grammar
The grammar defines how our language looks.
from lark import Lark, Transformer
import numpy as np
grammar = """
start: data command+
data: "DATA" list
command: "SUM" -> sum
| "MEAN" -> mean
| "STD" -> std
| "MAX" -> max
| "MIN" -> min
list: "[" NUMBER ("," NUMBER)* "]"
%import common.NUMBER
%import common.WS
%ignore WS
"""
Explanation
start: data command+
- Program must start with
DATA - Followed by one or more commands
data: "DATA" list
- Defines dataset input
- Example:
DATA [1, 2, 3]
Commands
SUM → sum
MEAN → mean
STD → std
MAX → max
MIN → min
- These map text → function names
-
-> summeans callsum()in Transformer
List Rule
list: "[" NUMBER ("," NUMBER)* "]"
-
Accepts:
[1][1, 2, 3]
(, NUMBER)*means repeat
Ignore Spaces
%ignore WS
- Allows flexible formatting
⚙️ Step 3: Build the Interpreter
Now we convert parsed text into execution.
class DLangInterpreter(Transformer):
def data(self, items):
self.data = np.array([float(x) for x in items[0]])
return self.data
Explanation
-
items[0]→ list of numbers - Convert to NumPy array
- Store in
self.datafor reuse
Step 4: Add Operations
SUM
def sum(self, _):
print(np.sum(self.data))
MEAN
def mean(self, _):
print(np.mean(self.data))
STD
def std(self, _):
print(np.std(self.data))
MAX
def max(self, _):
print(np.max(self.data))
MIN
def min(self, _):
print(np.min(self.data))
Explanation
- Each function matches grammar rule
-
_= unused input - Uses NumPy for computation
- Prints result immediately
Step 5: Parse List
def list(self, items):
return items
Explanation
- Returns list of numbers
- Passed to
data()method
Step 6: Create the Parser
parser = Lark(grammar, parser="lalr", transformer=DLangInterpreter())
Explanation
-
lalr→ fast parsing algorithm -
transformer→ auto-executes logic
Step 7: Read Input File
with open("example.dl") as f:
code = f.read()
parser.parse(code)
Example example.dl
DATA [10, 20, 30, 40]
SUM
MEAN
MAX
✅ Output
100
25.0
40
How It Works (Flow)
Text Input
↓
Parser (Lark)
↓
Grammar Rules Match
↓
Transformer Methods Trigger
↓
NumPy Executes
↓
Output Printed
✨ Why This Is Powerful
- You built a mini programming language
-
Clean separation of:
- Syntax (grammar)
- Execution (Transformer)
Easily extensible
Next Features You Can Add
1. Filtering
FILTER > 10
2. Sorting
SORT ASC
3. CSV Support
DATA file.csv
4. Chaining
DATA [1,2,3,4]
FILTER > 2
MEAN
Final Thought
This is how real systems like:
- SQL
- Pandas query engine
- Spark
…start at a basic level.
You just built the foundation of a data query engine
If You Liked This
Drop a like ❤️
Follow for more AI + Systems content
And try extending this DSL yourself!
Top comments (2)
nice
Thank you.