DEV Community: SandGrid

Bringing Notes, WeChat Reading, and Zhihu into Obsidian: My LLM-Wiki Knowledge Hub

SandGrid — Sat, 27 Jun 2026 03:39:10 +0000

A while back, I came across Andrej Karpathy's llm-wiki concept and felt an instant sense of resonance. I've always enjoyed writing things down, but the problem was that everything ended up scattered across different places, and I never had the energy to manage it properly. When I discovered llm-wiki, I realized — all that stuff I'd been writing over the years was finally going to pay off.

Why a Knowledge Hub

First, what is llm-wiki?

llm-wiki is a concept recently proposed by Andrej Karpathy: take all the written material you've accumulated over the years — notes, blog posts, reading highlights, work logs — treat it as a "corpus," and let an LLM automatically extract concepts, create pages, and weave cross-references into a structured, continuously evolving personal wiki.

The core premise is simple: everyone produces a substantial amount of structured, insightful writing in their daily work and learning — it's just scattered everywhere with no connections. llm-wiki uses an LLM-driven process to string these scattered pearls together. You keep producing and collecting content; the LLM handles the organization and management.

Unlike traditional manual wiki maintenance — creating pages, writing summaries, adding links, tedious and hard to sustain — llm-wiki brings the organizational cost down to nearly zero. You just tell the LLM the structure and rules of your knowledge base (an AGENTS.md file), and it can repeatedly execute ingestion, updating, and auditing operations. My personal experience: watching an AI turn scattered notes into a structured network of cross-references feels like clearing out a long-overdue debt.

Data Ingestion

The first thing I did was export all my Notion notes — development knowledge, investing insights, and countless miscellaneous records — and move them into Obsidian.

How to Import Notion Content into Obsidian

The process is straightforward:

Export your Notion data: Go to "Settings & members → Settings" in Notion, select "Export all workspace content," and choose Markdown & CSV as the format. You'll get a ZIP file; after unzipping, each Notion page becomes a .md file, and databases come with additional CSV files. Note: Notion's free plan exports one workspace at a time — if you have multiple workspaces, export them separately.
Install the Obsidian Importer plugin: Search for "Importer" in the Obsidian community plugin marketplace. It supports one-click import from Notion, Bear, Evernote, OneNote, and more, handling image attachments and internal links automatically. After enabling the plugin, press Cmd+P, search for "Importer: Open Importer," select the Notion format, and point it to the unzipped folder.
Manual import (fallback): If you prefer not to use the Importer plugin, just drop the unzipped folder directly into your Obsidian vault. Obsidian natively supports [[wiki-link]] internal links, and Notion's exported Markdown typically already converts links to this format.
Post-import handling: I recommend placing the original files in a dedicated subdirectory (e.g., raw/notion-export/) and marking them as "read-only." This preserves the integrity of the original data — a key tenet of the llm-wiki methodology: raw materials are never modified; the LLM builds structured knowledge on top of them. If your Notion had databases, keep the CSV files for reference; embedded Notion-specific blocks (calendars, kanban boards) will lose interactivity after export, but the text content remains.

After the whole process, thousands of scattered notes were consolidated into Obsidian, becoming the first batch of "raw materials" for my knowledge hub.

Next, I fed Karpathy's gist to an AI and had it generate the project's AGENTS.md document. The AI naturally figured out the ingestion and auditing operations required for llm-wiki.

Then it was time to execute. Watching the AI continuously generate wiki content, categorizing years of accumulated material — it was genuinely satisfying.

After that, I did a few more things: bringing in my Zhihu writings and WeChat Reading notes. I've written over a thousand answers on Zhihu, and over the years I've read more than a hundred books on WeChat Reading. Beyond just highlights, these are significant components of my knowledge system. Coincidentally, around that time, WeChat Reading released their official skill, so I put it to use.

Importing Zhihu Content

How to Sync Zhihu Writings to Obsidian

Zhihu doesn't provide an official data export API, so I used Playwright for browser automation.

Steps:

Install Playwright: pip install playwright && playwright install chromium
Run the script for the first time, log in by scanning a QR code or entering your password in the opened Chromium browser
Once logged in, the script automatically crawls your profile to capture all answers, articles, and status updates
Login state is persisted locally; subsequent runs use --reuse for silent execution without re-login

Features: Incremental sync — only fetches new content, existing files are never reprocessed. Files are organized by content type (answers/articles/pins).

Syncing WeChat Reading Notes

How to Sync WeChat Reading Notes to Obsidian

WeChat Reading provides an Agent API Gateway — apply for an API key and you're good to go.

Steps:

Call the /user/notebooks endpoint to get the list of books with notes
For each new book, fetch highlights and annotations separately
Group content by chapter and output as well-formatted Markdown files

Output format: Book title and author as the heading, each chapter's highlights in blockquote format (with dates), personal annotations placed below the corresponding highlights.

Features: Fully incremental — the script maintains a state file of synced book IDs, only processing new additions on each run. Over 150 books' worth of notes silently flowed into Obsidian, becoming one of the richest sources of raw material for my knowledge hub.

Intelligent Retrieval with LLM Wiki

At this point, the content layer was essentially ready. Then I started thinking: since most of my knowledge and creative output is here, could I start distilling... myself?

I built a simple first version: a "personal" pipeline parallel to the wiki pipeline, with similar ingestion and linting operations. The key difference: wiki focuses on knowledge, while personal focuses on who I am as an individual.

Knowledge Base vs. Personality Distillation: Two Different AI Processing Approaches

Here it's worth explaining the difference — they share the same set of raw materials but have entirely different goals and outputs.

Knowledge Base (Wiki): Answers "What Do I Know?"

Extracts objective knowledge from notes, blogs, and reading highlights, generating concept pages (e.g., "distributed consensus"), entity pages (e.g., "Raft algorithm"), and source summary pages (e.g., "Designing Data-Intensive Applications — reading notes"), with dense cross-references between them. The goal: make knowledge queryable and reusable — an externalized second brain.

Personality Distillation (Personal Model): Answers "Who Am I?"

Reverse-engineers cognitive patterns, expressive styles, and value orientations from your writing and reading. For example, analyzing technical blog posts might reveal a "thesis-first, case-driven" style; analyzing Zhihu answers might uncover recurring traits like "first-principles reduction" and "quantitative thinking." The output isn't knowledge entries — it's a cognitive map of a person: what you're good at, how you approach problems, what you value.

Comparison

Dimension	Knowledge Base	Personality Distillation
Core question	What do I know?	Who am I?
Input	Notes, blogs, reading highlights	All personal writing and reading records
Output	Concept/entity/source pages + cross-references	Domain depth, cognitive traits, expressive style, values
Direction	Outward: structuring external knowledge	Inward: modeling personal cognition
Workflow	Ingest → Query → Lint → Audit	Ingest → Query → Lint → Audit (isomorphic)

Both processes are structurally similar, but one looks outward, structuring and organizing the knowledge you possess; the other looks inward, distilling and modeling your cognitive traits as an individual. This "two sides of the same coin" design is, I think, the most fascinating part of the entire system.

Lately I've been looking at projects like Nüwa online to see if there are better approaches to personality distillation.

Results and Reflections

That's the recent story of my knowledge hub. If you have thoughts or ideas, I'd love to hear them.

Frequently Asked Questions

Q: Is Obsidian suitable for programmers doing knowledge management?

Absolutely. Obsidian's core philosophy — local Markdown files, bidirectional links, graph visualization — naturally aligns with how programmers work. You already know Markdown syntax. Local file storage means full data control and Git versioning. Bidirectional links let you manage knowledge references just like code dependencies. Layer on the llm-wiki approach, and AI can automatically extract concepts from scattered notes, create pages with cross-references, transforming loose documents into a structured knowledge network.

Q: Does LLM Wiki require a GPU?

No local GPU deployment is needed. The core idea of LLM Wiki is letting an LLM process your text, not running a model yourself. You simply call a cloud LLM API, feed it your Markdown files, and let it extract concepts, generate pages, and build cross-references. The entire "hardware" requirement is just Obsidian plus any tool that can call an LLM API (like WorkBuddy or another Agent).

Q: What's the difference between llm-wiki and a traditional wiki?

A traditional wiki requires you to manually create pages, write summaries, and add internal links — high maintenance cost that's hard to sustain. llm-wiki reduces the organizational cost to nearly zero — you just keep producing and collecting written content, and the LLM reads your AGENTS.md rules, repeatedly running ingestion, updating, and auditing operations to generate a structured cross-reference network. In short: a traditional wiki means "you organize knowledge"; llm-wiki means "AI organizes knowledge for you."

Q: What's the difference between knowledge distillation and personality distillation?

Both share the same set of raw materials, but their goals and outputs are entirely different. Knowledge Base (Wiki) answers "What do I know?" — extracting objective knowledge from notes and reading highlights, generating concept pages and cross-references. Personality Distillation (Personal Model) answers "Who am I?" — reverse-engineering your cognitive patterns, expressive style, and value orientations from your writing and reading records. One looks outward (structuring knowledge), the other looks inward (modeling personal cognition). The workflows are similar, but the direction is opposite.

Quick Start Guide

Install Obsidian: Download the client from obsidian.md and create a local Vault — just a local folder.
Configure LLM Wiki: Create AGENTS.md in the Vault root, following Karpathy's llm-wiki approach to define maintenance rules, including ingestion, updating, and auditing workflows. Let AI tools read this file to automatically extract concepts and build cross-references from raw notes.
Import Notion notes: Export from Notion settings as Markdown + CSV, use the Obsidian Importer plugin for one-click import, or drop the extracted Markdown folder directly into the Vault.
Connect WeChat Reading: Apply for a WeChat Reading API key, call the /user/notebooks endpoint to fetch books, pull highlights and notes, group by chapter, and output as Markdown files into the Vault.
Import Zhihu and blog: Use a Playwright script to auto-scrape Zhihu answers and articles; copy blog Markdown source files into the Vault. Once done, run a full wiki ingestion via AI to generate the complete cross-reference network.

Source: https://lichuanyang.top/en/posts/18804/

Free AI Video Generator: How I Built a Zero-Cost Tool for Narrated Multi-Scene Videos

SandGrid — Sat, 27 Jun 2026 03:37:26 +0000

"The solution is not to suppress AI, but to make it a more equitable capability, so that everyone knows how to create more with AI. This is a very important vision for our company — to make world-class AI belong to everyone."

This is something Bruce Yang, the founder of Agnes AI, said in an interview.

Many Chinese AI companies today — DeepSeek, Zhipu, and others — are driving down the price of AI. To be fair, the cost of text and code processing has already been pushed remarkably low. But video is different. Making AI videos today has an absurdly high barrier — overseas services like Runway and Pika charge tens of dollars monthly, domestic platforms like Jimeng and Keling charge by the second once free quotas run out, and running open-source models locally requires a GPU costing over ten thousand RMB.

Objectively speaking, video generation is genuinely expensive right now. Making industrial-grade video generation available to everyone isn't realistic. But ordinary people should still have ways to experiment and create. Thanks to Agnes for opening up their video model and giving us this opportunity. This project is just a small contribution toward that goal. Agnes Video Generator (official website) — it's a free AI video generator. Not "free trial" or "free for 3 generations," but the whole thing: script writing, image synthesis, video rendering, voiceover, subtitles, all at zero cost. You just need a free API key from Agnes AI.

Agnes's video model isn't perfect yet, to be honest. But I want to use this project to grow alongside Agnes, and contribute in my own small way toward AI equity.

Multiple Ways to Use It

Give it a text prompt, get a video back. A few different modes:

Simple Video. A straightforward API wrapper — good for testing. Most API parameters are exposed as config options.

Creative Video. You write a story idea, like "dark version of The Frog Prince," and the AI handles everything: expand story → generate character references → split into scenes → write shot prompts → generate per-scene video → narration → subtitles → final output. Ten steps, all automatic. By pre-generating end frames, it ensures the best possible visual continuity between scenes.

Manuscript Video & Digital Anchor. Paste a long article or script — it auto-splits by speech duration and generates video per segment, or puts a digital anchor there to read it. Everything stitched with a unified TTS narration + subtitle track. Great for explainers and course content.

For detailed parameters and usage guides for each mode, check the official website.

Getting It Running

git clone https://github.com/lcy362/agnes-video-generator.git
cd agnes-video-generator
./start.sh

That's it. start.sh creates a virtual environment, installs dependencies, and starts the server.

Once it's running, open http://localhost:8765, paste in your Agnes AI API key at the top, pick a mode, write your idea, and wait patiently for the results.

If you're using an AI coding assistant like Cursor or Claude, I've included a dedicated guide for AI Agents. Just tell your agent to read the Agents.md file in the project — it'll handle the whole setup on its own.

Demos

I made a few demos — check them out:

The Frog Prince — no narration — 5 scenes, keyframes chaining, fully auto-generated
Same story, with voiceover and subtitles — AI narration + auto subtitles, see the subtitle effect
Manuscript video — pasted a long article, auto-split with different visuals per segment

That's About It

Going back to Bruce Yang's words — "making world-class AI belong to everyone."

This project isn't some grand mission. It's just about keeping the door to AI video creation open. No subscription, no fancy GPU, no cost at all — just a free API key and a machine that can run Python.

Code on GitHub, official website at video.lichuanyang.top. Bug reports welcome.

Source: https://lichuanyang.top/en/posts/22470/

I build an open-source ,completely free AI video generator

SandGrid — Wed, 24 Jun 2026 09:47:11 +0000

Thanks to Agnes AI, for that they made all their models free. And I build a pipeline to make their video model easy to use.

So everybody can generate AI video with this, no need to pay for anything, no tokens or GPU.

Agnes’s video model isn’t perfect yet, to be honest. But I want to use this project to grow alongside Agnes, and contribute in my own small way toward AI equity.

It's easy to use. For example, the Creative Video mode, You write a story idea and the AI takes over: expand story → generate character references → split into scenes → write shot prompts → generate per-scene video → narration → subtitles → final output. Ten steps, all automatic.

clone it and try from github

lcy362 / agnes-video-generator

Open-source, self-hosted AI video generator — completely free. Text to multi-scene video with narration, subtitles, and digital anchor via Web UI, powered by Agnes AI.

🎬 Agnes Video Generator — Completely Free AI Video Generator

Completely free AI video generator — Built on Agnes AI's free models, no subscription, no high-end GPU, no usage limits. Type in a text idea and automatically generate multi-scene AI videos with narration and subtitles. Supports text-to-video, image-to-video, keyframes animation, digital anchor, and more. All AI compute runs in the cloud — a regular laptop is all you need.

"The solution is not to suppress AI, but to make it a more equitable capability, so that everyone knows how to create more with AI. This is a very important vision for our company — to make world-class AI belong to everyone. What we can do may be insignificant, but this vision is very long-term and enduring."

— Bruce Yang, Founder of Agnes AI

🌐 Official Website | 📝 Blog (中文) | 📝 Blog (English)

🎥 Demo

1. Creative Video —

…

View on GitHub

or the website: https://video.lichuanyang.top