Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every commit. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product.
Vector graphics are the holy grail of modern UI/UX design, CAD systems, and digital illustration. Unlike raster images (JPEGs, PNGs) composed of fixed pixel grids, Scalable Vector Graphics (SVGs) use geometric shapes—Bézier curves, polygons, and fill attributes. They render crisply at 8K resolution and weigh only a few kilobytes.
However, automating SVG creation has always been a major pain point for developers:
- Optimization methods (like DiffVG) treat vectorization as an iterative guessing game. They generate cluttered, uneditable SVG code with thousands of overlapping anchor points, consuming massive GPU power for every single image.
- LLM code-generation methods (like Chat2SVG or StarVector) try to write raw XML tags directly. Because standard AI tokenizers split XML tags and coordinate numbers haphazardly, models suffer from "coordinate hallucination" and hit strict character limits whenever designs get complex.
A team of researchers from Fudan University and StepFun introduced OmniSVG—an end-to-end framework built on native Vision-Language Models (Qwen2.5-VL) that treats vector generation not as raw text writing, but as structured token sequence prediction.
1. High-Level Intuition: Tokenizing Geometry Instead of Text
Standard Large Language Models (LLMs) view an SVG as a long string of raw text: <path d="M 10 20 C 30 40 ..."/>. To a typical AI tokenizer, <path, d=", and 10 are disconnected sub-word chunks. The AI has to learn XML syntax rules, floating-point math, and spatial geometry all at the same time.
OmniSVG changes the paradigm:
- Flatten the Structure: All complex grouping tags, transformations, and inline styles are stripped away into simple, flat drawing instructions.
- Atomic Commands: Any vector graphic is simplified into five core path commands:
- M (Move To)
- L (Line To)
- C (Cubic Curve)
- A (Arc)
Z (Close Path)
Explicit Fill Token (F): Color information is mapped directly into hex fill commands (F) right inside the drawing order, treating color as just another drawing step rather than a messy CSS attribute.
+-----------------------------------------------------------------------+
| OmniSVG Pipeline |
| |
| [ Input Instruction ] |
| "A cute red panda icon" |
| | |
| v |
| +-----------------------+ +-----------------------------------+ |
| | Qwen2.5-VL VLM | --> | Flattened & Parameterized Tokens | |
| | Autoregressive Core | | <SOP> M [2000] C [2045] F #FF0000 | |
| +-----------------------+ +-----------------------------------+ |
| | |
| v |
| [ Clean, Editable SVG ] |
+-----------------------------------------------------------------------+
By separating spatial geometry from low-level XML syntax, the underlying model can focus entirely on visual layout and drawing order.
2. The Math Explained in Plain English: Coordinate Compression & Economics
Spatial Token Compression
Traditional methods tokenized horizontal ($X$) and vertical ($Y$) coordinates as separate numbers. If an SVG path has 100 points, it takes at least 200 tokens just to specify where those points go.
OmniSVG fits every SVG onto a standardized 200 by 200 pixel canvas grid. Instead of sending two separate numbers for every point, OmniSVG merges the two coordinates into a single number index using simple grid math:
Combined Token Index = (X Coordinate * Canvas Width) + Y Coordinate
For a canvas that is 200 pixels wide and 200 pixels tall, this formula translates every possible $(X, Y)$ spot on the canvas into a single unique ID number between 0 and 39,999.
To convert that single ID number back into 2D coordinates on the fly:
- X Coordinate: Divide the Token ID by the Canvas Width (200) and round down.
- Y Coordinate: Take the remainder after dividing the Token ID by the Canvas Width (200).
Back-of-the-Envelope Token Savings
Imagine writing a single smooth curve segment with a start point, two control handles, an end point, and a red fill color:
-
Raw XML Text String:
<path d="M 120 45 C 130 50 140 85 150 90 Z" fill="#FF5733"/> Standard Tokenizer Count: 22 to 28 tokens.
OmniSVG Approach:
Commands: M, C, Z, F (4 tokens)
Start point (Combined 120 and 45): 1 token
Curve handle points and end point: 3 tokens
Red Color Code (#FF5733): 1 token
Total Token Count: 9 tokens (A 65% reduction in total tokens!).
System Economics & Operational Efficiency
When AI models generate content, their computational work and latency scale rapidly with the total length of the token sequence. Shorter sequences mean significantly faster rendering and lower server costs.
Token Sequence Length vs. Inference Latency (Conceptual)
Latency
^
| / (Raw XML: ~32k Tokens)
| /
| /
| /
| /
| /----------------------/ (OmniSVG: ~4.8k Tokens)
|_________________/________________________________________>
Sequence Length
| Metric | DiffVG / Live Optimization | Raw XML LLM Generation | OmniSVG (4B/8B Models) |
|---|---|---|---|
| Method Type | Repeated Gradient Guesswork | Standard Text LLM | Structured Vision-Language Model |
| Tokens / Steps | 50,000 to 300,000 steps | ~18,500 raw text tokens | ~3,800 to 5,800 tokens |
| Time per Image | 45 to 180 seconds | 12 to 25 seconds | 1.5 to 4 seconds |
| Output Structure | Messy, overlapping anchor points | Clean, but prone to syntax errors | Hierarchical, clean, fully editable |
3. How OmniSVG Learns to Draw
OmniSVG fine-tunes the Qwen2.5-VL Vision-Language Model using standard "next-token prediction"—the exact same technique LLMs use to write text, but applied to vector commands.
- Input Context: The model receives the user's instructions (such as a text prompt like "a blue bird icon" or a reference image).
- Sequential Prediction: Based on what it has drawn so far, the model predicts the single most likely next drawing command or coordinate token.
- Training Objective: During training, the model is shown millions of completed vector graphs and adjusts its internal weights to minimize the difference between its predicted tokens and the actual target tokens.
4. Scalability: MMSVG-2M & MMSVG-Bench
To train OmniSVG, the authors created MMSVG-2M, a dataset containing 2 million high-quality SVG assets across three core categories:
- MMSVG-Icon (1.1 Million Assets): Web icons, interface symbols, UI elements.
- MMSVG-Illustration (0.5 Million Assets): Richly colored flat graphics, web banners, vector artwork.
- MMSVG-Character (0.4 Million Assets): Detailed vector anime art and character designs.
+-------------------------------------------------------------------+
| MMSVG-2M Dataset Breakdown |
| |
| [ MMSVG-Icon ] [ MMSVG-Illustration ] [ Character ] |
| 1.1M Assets 0.5M Assets 0.4M Assets |
| (Monochrome/UI) (Rich Flat Graphics) (Complex Anime) |
+-------------------------------------------------------------------+
Benchmark Results (MMSVG-Bench)
OmniSVG was tested against existing tools across two main tasks: generating SVGs from text descriptions and converting raster images to SVGs.
Text-to-SVG Task
- Visual Quality (Lower FID is better): OmniSVG scored 137.40, vastly outperforming older text models like VectorFusion (250.77) and Chat2SVG (190.87).
- Text Alignment: OmniSVG maintained high prompt accuracy while generating outputs in less than 4,000 tokens, compared to optimization tools that required over 60,000 iterations.
Image-to-SVG Vectorization Task
- Visual Similarity (Higher SSIM is better): OmniSVG achieved a structural similarity score of 0.950, matching expensive optimization tools like DiffVG (0.954) while generating clean, organized vector paths instead of messy shapes.
- Accuracy vs. Large Models: OmniSVG drastically outperformed GPT-4o's direct SVG output (0.792 SSIM) and StarVector (0.881 SSIM) in vectorizing detailed images.
5. Tokenization Ablation Insights
To prove that combining coordinates and colors into custom tokens was the secret to success, the authors compared four model variants:
| Configuration | Visual Error (Lower is better) | Output Accuracy (Higher is better) | Average Token Count |
|---|---|---|---|
| Raw XML Text (No Tokenization) | High Error (218.76 FID) | Low Accuracy (0.718 SSIM) | 18,500 tokens |
| Without Coordinate Tokens | Medium Error (193.42 FID) | Medium Accuracy (0.809 SSIM) | 10,200 tokens |
| Without Color Tokens | Low-Medium Error (167.28 FID) | Good Accuracy (0.879 SSIM) | 6,300 tokens |
| OmniSVG (Full Tokenization) | Lowest Error (145.89 FID) | Highest Accuracy (0.928 SSIM) | 4,800 tokens |
Key Takeaway: Combining $X$ and $Y$ coordinates into single scalar tokens reduced file size and sequence lengths by 74% while making the resulting graphics twice as visually accurate.
6. Hands-On Example: Tokenizer Implementation
Below is a Python script showing how OmniSVG compresses 2D coordinates and drawing commands into discrete tokens:
import re
class OmniSVGTokenizer:
def __init__(self, canvas_width=200, canvas_height=200):
self.w = canvas_width
self.h = canvas_height
# Command mapped tokens
self.cmd2token = {'M': '<M>', 'L': '<L>', 'C': '<C>', 'A': '<A>', 'Z': '<Z>', 'F': '<F>'}
self.token2cmd = {v: k for k, v in self.cmd2token.items()}
def encode_point(self, x: int, y: int) -> int:
"""Flattens 2D coordinates into a single 1D scalar token ID."""
x = max(0, min(self.w - 1, x))
y = max(0, min(self.h - 1, y))
return (x * self.w) + y
def decode_point(self, token_id: int) -> tuple[int, int]:
"""Restores 2D coordinates from a single 1D scalar token ID."""
x = token_id // self.w
y = token_id % self.w
return x, y
def tokenize_path(self, path_str: str, fill_hex: str = None) -> list:
"""Converts raw SVG path string into OmniSVG discrete tokens."""
tokens = ["<SOP>"]
if fill_hex:
tokens.extend([self.cmd2token['F'], fill_hex])
# Parse commands and coordinates using regex
pattern = r'([MLCAZ])|(-?\d+\.?\d*)'
matches = re.findall(pattern, path_str)
current_cmd = None
coords_buffer = []
for cmd, num in matches:
if cmd:
if current_cmd and coords_buffer:
tokens.extend(self._process_coords(coords_buffer))
coords_buffer = []
current_cmd = cmd
tokens.append(self.cmd2token[cmd])
elif num:
coords_buffer.append(int(float(num)))
if coords_buffer:
tokens.extend(self._process_coords(coords_buffer))
tokens.append("<EOS>")
return tokens
def _process_coords(self, coords: list) -> list:
"""Helper to pair raw coordinates into scalar tokens."""
encoded = []
for i in range(0, len(coords), 2):
if i + 1 < len(coords):
encoded.append(f"<COORD_{self.encode_point(coords[i], coords[i+1])}>")
return encoded
# Example Usage
tokenizer = OmniSVGTokenizer(canvas_width=200, canvas_height=200)
raw_path = "M 10 20 C 30 40 50 60 70 80 Z"
fill_color = "#FF0000"
token_sequence = tokenizer.tokenize_path(raw_path, fill_hex=fill_color)
print("Tokenized Sequence:")
print(token_sequence)
# Demonstrate Coordinate Reconstruction
sample_token_id = tokenizer.encode_point(120, 45)
decoded_x, decoded_y = tokenizer.decode_point(sample_token_id)
print(f"\nCoordinate Check: Original (120, 45) -> Combined ID: {sample_token_id} -> Reconstructed: ({decoded_x}, {decoded_y})")
7. What's Next for Vector Graphics?
OmniSVG proves that AI vector generation works best when we treat geometry as a structured sequence of drawing actions rather than raw code text. By combining Vision-Language Models with smart spatial tokenization, we can generate clean, editable vector graphics in seconds instead of minutes.
How do you handle SVG vectorization in your current development workflows—do you rely on traditional image processing libraries, optimization scripts, or manual design tools? What challenges do you run into most with AI-generated SVG code? Share your thoughts in the comments below!
*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.
git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*
Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.
HexmosTech
/
git-lrc
Free, Micro AI Code Reviews That Run on Git Commit
| 🇩🇰 Dansk | 🇪🇸 Español | 🇮🇷 Farsi | 🇫🇮 Suomi | 🇯🇵 日本語 | 🇳🇴 Norsk | 🇵🇹 Português | 🇷🇺 Русский | 🇦🇱 Shqip | 🇨🇳 中文 | 🇮🇳 हिन्दी |
git-lrc
Free, Micro AI Code Reviews That Run on Commit
GenAI today is a race car without brakes. It accelerates fast -- you describe something, and large blocks of code appear instantly. But AI agents silently break things: they remove logic, relax constraints, introduce expensive cloud calls, leak credentials, and change behavior -- without telling you. You often find out in production.
git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.
In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen
At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…
Top comments (0)