DEV Community: sword luan

Stop Paying Before the Prototype Works: A Verified Test of itsfree.ai

sword luan — Thu, 10 Sep 2026 20:28:36 +0000

The first cost of adding AI to a small tool is often not tokens. It is the time spent comparing provider docs, sign-up rules, base URLs, and model names that may already be stale. We tested midudev/itsfree.ai, the open-source directory created by Miguel Ángel Durán (midudev), to see whether it could shorten that discovery work and lead to a result we could actually verify.

itsfree.ai is a directory, not a model proxy operated by us. At the time of the experiment, its rendered pages listed 25 providers with free allowances, 463 free models, and 9 local runtimes. Those numbers describe directory records; we did not test every listed model or runtime. The useful design choice is that provider data, requirements, base URLs, and gotchas are collected in structured TypeScript files and exposed through filters such as no-sign-up and OpenAI-compatible.

The repository built successfully, but the development server did not. Two pnpm dev attempts failed because Vite could not find a generated deps_ssr file, and clearing .vite reproduced the failure. Serving the completed build with pnpm preview returned HTTP 200 and rendered the pages we needed. The distinction matters: the current commit was usable for directory exploration in our environment, while the dev compatibility issue remained unresolved.

The API path showed why live verification matters. The documented gpt-oss example returned model_unavailable. Other model choices produced missing-key or invalid-key errors. After reading the llm7 provider gotchas and querying the live model list, we selected codestral-latest and sent the placeholder bearer value unused to the OpenAI-compatible endpoint at https://api.llm7.io/v1. That request returned HTTP 200.

We then asked the model to do useful, checkable work: extract six provider records and sort them by freeModels. A separate verifier derived truth from the source data and compared record count, fields, values, and order. The model output matched all six records (match=true). The call used 272 prompt tokens and 101 completion tokens, 373 total; curl measured 1.69 seconds, and the experiment recorded CNY 0 for the call.

That success is not a general accuracy claim. When the same model was asked what the project does, it incorrectly answered that the project verifies domain availability. The comparison gives us a practical boundary: a free model can be useful for structured transformations when expected output is machine-checkable, but fluent prose is not evidence of factual accuracy.

You can reproduce the decision process without moving an entire application. First, open the no-sign-up or OpenAI-compatible filter on itsfree.ai and pick one candidate. Second, read its gotchas, query the provider’s live model list, send the smallest representative request, and automatically check the fields or ordering you care about. Only then decide whether the endpoint is suitable for a larger prototype or whether a paid, more stable path is justified.

The result supports a narrow, useful conclusion: itsfree.ai reduced provider discovery to a structured shortlist, and one directory-guided route completed a real, machine-verified task at no recorded cost in this run. The durable asset is not the name of a model that happens to be free today. It is a repeatable sequence: filter, read gotchas, check the live list, test minimally, and verify.

Links

Project: https://itsfree.ai
Official repository: https://github.com/midudev/itsfree.ai

Images / captions

01-home.png — locally rendered homepage.
04-no-signup.png — real no-sign-up filter result.
02-provider-llm7.png — real llm7 provider page.
Comparison card generated from task-request.json, task-response.json, and task-verification.json; label it as generated from experiment files.

Upload order

Title and homepage image. 2. Body with the screenshots at their matching sections. 3. Verification card after the task result. 4. Repository and project links. 5. Preview captions and links before publishing.

AI 聊久了就忘约定？本机实测 okf-agent-memory

sword luan — Tue, 08 Sep 2026 05:37:30 +0000

AI 聊久了就忘约定？我们实测了一个本地记忆工具｜okf-agent-memory

明明已经说好了，这个字段名不要再改。聊了一会儿，AI 又给你改回去了。

用 AI 写代码，你有没有碰到过这种事？烦的不只是改错了，而是你还得翻聊天记录，找出之前的决定，再解释一遍。项目越做越长，光是提醒它“我们之前说好了什么”，就成了一份额外的工作。

那能不能把这些约定从聊天里拿出来，存到一个下次能找得到的地方？

今天我们试的开源工具，叫 okf-agent-memory，做的就是这件事。

它把知识条目保存在你自己电脑的目录里，可以按关键词搜索，还能检查文件是不是符合规范。不用先搭一套数据库，就能开始建立本地记忆库。

比如，一个项目讨论了几十轮，真正需要留下的，可能就是几个字段名、最后确定的接口方案，还有某项功能为什么决定不做。下次继续，你需要找的是这些结论，而不是重新看一遍全部聊天。

说到这里，你可能会问：这不就是记笔记吗？

确实，保存结论本身不新鲜。我们要看的，是它能不能让这些记录方便找回，以及记录不符合规范时，能不能把问题指出来。

这次我们在本机测的是 0.1.2 版本。初始化以后，记忆库的目录和基础文件正常生成。我们写入了一条测试笔记，再用相关关键词搜索，找到了刚刚写进去的那条。

保存和搜索这条路，跑通了。

接下来，我们故意给它出了一道错题。

先对一个正常的记忆库运行检查，结果通过。然后往里面放进一份缺少规定抬头的文件，再检查一次。

这次没有通过，而且指出了那份文件的问题。

这个报错，反而是我们想看到的。如果什么文件放进去，检查都告诉你“没问题”，那这个功能就没有意义。至少这次故意留下的格式问题，它确实发现了。

最后，我们又把原来的笔记读回来，正文还在。

所以，这次不是只把软件装上、看到它能启动就结束了。记录写进去了，能搜到，也能读回来；检查功能也分辨出了正常的库和我们故意放进去的那份不规范文件。

对经常自己改项目的人来说，这个工具值得看的地方，就在这里：重要约定不用只埋在聊天记录里，可以单独保存，需要的时候按关键词找出来，再把相关内容交给 AI。不必每次都翻长篇历史，也不必把所有笔记一股脑贴回去。

不过，“记录存下来了”和“AI 会自动用上”，是两件事。这次验证的是本机存取和检查，还不能据此说，装好以后 AI 就会自动记住每次对话，下一轮也一定会主动调用。

检查功能也一样。它发现的是我们这次设置的格式问题，不代表笔记里的结论一定正确。接口后来改了，旧记录也得跟着更新。

至于云端协作、多人同时写入，以及大量数据下的性能，这次没有测。

你也有一堆反复向 AI 交代的项目约定，可以先拿其中一条试试：存进去，用相关关键词找出来，再读回来。先看看这套办法能不能帮你少翻几次聊天、少重复解释几遍，再决定要不要用进日常工作。

项目在 GitHub 上，搜索 okf-memory/okf-agent-memory。

这里是剑的 AI 实验室。每天选一个开源项目，装上，跑一遍，把结果给你看。喜欢看真实测试，欢迎订阅，我们下期见。

实验田：https://jianailab.com/studio/field?experiment=1059bb13-e0fd-4242-ada3-6515f07a4b63

GitHub：https://github.com/okf-memory/okf-agent-memory

We ran gzh-design-skill locally: layout check passed, 3 planted errors caught

sword luan — Mon, 07 Sep 2026 19:00:29 +0000

Finished writing, then spent half an hour fighting layout: mismatched titles, cramped code blocks, drifting images. You meant to publish; you became a layout worker.

We ran the open-source project gzh-design-skill (a WeChat Official Account layout helper) on our machine.

Results first

Built a layout file that passed the tool's check
Generated a preview page with a copy button
Intentionally inserted three bad patterns — the tool reported three errors

Important: "Compliant" here means the layout code passed this tool's checker. It is not WeChat content moderation and not an official WeChat certification.

A preview that opens is not the same as pasting into the Official Account editor without breakage. Treat those as two separate checks.

Repo

https://github.com/isjiamu/gzh-design-skill

It takes Markdown-style drafts (hash headings, simple lists), applies a theme, and emits HTML with inline styles (font size, color, spacing on the tags themselves) so you can paste into the WeChat editor.

What we actually verified

Sample check: fully compliant. Bad sample: three errors found. Component library: no blocking failures. Combined output re-checked: still passed, then preview generated.

We did not finish the full "assistant picks theme and assembles the whole article" path this round. Do not read "sample passed" as "fully auto-layout and verified inside WeChat."

Who it's for

People who write long WeChat posts, already draft with AI assistants, and still get stuck on final layout.

Best test: your own real article (body, titles, images, code) → check → preview → paste → inspect the reader page.

— Jian AI Lab: install, run, show the measured result.

Clarify a vague SaaS idea into a one-pager: we ran zhang7700/clarify-saas-idea locally

sword luan — Sat, 05 Sep 2026 02:16:43 +0000

Indie founders keep rewriting “who is this for?” in a notes app. clarify-saas-idea is a free Cursor / Claude Code Agent Skill that forces a decision-ready page: one ICP, a wedge, non-goals, MVP In/Out.

Jian AI Lab cloned commit e1e48741d6bd (GitHub id 1357550091) on 2026-09-05 and followed the README.

Install is a copy, not a PyPI package. pip install clarify-saas-idea failed in 0.992s with “No matching distribution found”. mkdir -p then cp -R skills/clarify-saas-idea into .cursor/skills, .claude/skills, and .agents/skills succeeded in 0.030s. Three SKILL.md files, 3830 bytes each, matched the source.

Empty input: the harness stopped in 0.048s with BLOCKED_MISSING_INPUTS and did not write docs/saas-one-pager.md.

Documented demo prompt (302 bytes): watch competitor pricing pages, sell to solo B2B founders, about four weekends, distribute on X. First draft at 09:44:38 was 499 English words, over the skill’s “≤ ~400 words” bar. Trimmed file at 09:45:49: 2543 bytes, 399 words, all ten required headings, seven In bullets. Wall clock from over-limit draft to passing draft: 71 seconds. We did not time a handwritten one-pager, so we do not invent that number.

Author listing mentions a paid pack; we did not pay and did not test checkout.

Repo: https://github.com/zhang7700/clarify-saas-idea

Public X archive: @KaiZhang_AI — read-only, no reply.

Lab write-up: https://jianailab.com/experiments/clarify-saas-idea-ab56a7d8

Turn a public page into an 8-page Xiaohongshu plan with crayon-shinchan-xhs-skill

sword luan — Fri, 04 Sep 2026 08:08:39 +0000

Creators already have source pages. What they still assemble by hand is the carousel: titles, one point per page, on-image copy, and image prompts. jasonmarkppp/crayon-shinchan-xhs-skill is a Skill that performs that rewrite. We ran a minimum test against the Jian AI Lab public site. Empty input stopped the run. After a curl excerpt was supplied, an 8-page copy pack passed the Skill contract in 95 seconds of wall clock. No paid image API was called.

Pain

A README or marketing page is not a Xiaohongshu carousel. The missing work is structure, not more raw text.

Hook (first ten seconds)

The notes sit in a bookmark folder; the eight-page set has not started. This write-up shows the Skill doing that segment on a real public page.

Promise

Readers can verify the clone commit, the empty-run log, the three Markdown outputs, and the public HTML facts we used. This article does not claim a live Xiaohongshu post or generated images.

Project and mechanism

Repository: https://github.com/jasonmarkppp/crayon-shinchan-xhs-skill

Commit: 7b35e340118b0a9ee850e9698d06259f5ec901c5

Two editions: codex/SKILL.md and domestic-agents/SKILL.md.

Default: eight pages; find a distribution angle rather than summarize; do not invent unread sources; reserve text-safe areas in image prompts.

Public association (archive only): https://x.com/cai_liang73983/status/2095456821536518620

Experiment

Clone into the run write root. Copy SKILL.md as install. curl https://jianailab.com and /library. Library HTML on 2026-09-04 still showed 233 projects, “今日新收录 0”, stamp 2026-09-02, and no crayon-shinchan string.

First run with zero source files: 2026-09-04T11:25:35+08:00, BLOCKED_NO_SOURCE, exit 2, 0.000s.

After jianailab-public-notes.md (2460 bytes) the Skill produced:

xiaohongshu-plan.md 8636 bytes
image-prompts.md 3064 bytes
caption-and-comment.md 1749 bytes

Source-ready 11:25:31 → files on disk 11:27:06 = 95s. Validation found pages 1–8 and 6 titles.

Result

Input: unstructured public-page notes. Output: a design-ready 8-page pack. Manual timing was not measured and is not invented. Paid image generation was not invoked.

Who this helps

Operators who turn docs into carousels, and anyone who wants a first pass before design. The lab public page itself was a valid source.

Public harvest: https://jianailab.com/experiments/crayon-shinchan-xhs-skill-940cd766 complete=false.

Skip the OCR queue: pdf-inspector 1.17.0 classified a 14-page paper in 8.296 ms on a local Mac

sword luan — Wed, 02 Sep 2026 16:06:58 +0000

Teams still send every PDF to cloud OCR. Page counts grow, the pipeline waits, and the bill follows. Firecrawl's open-source pdf-inspector classifies a file first: if it is already text-based, it writes Markdown locally. Jian AI Lab ran version 1.17.0 today on a Mac with zero API keys and zero spend.

This article uses the same facts as the public report. It does not treat the author's 200-document README benchmark as today's measurement. OCR was not executed.

What the author already stated

The README describes a local classify-and-extract engine for text PDFs. Default path skips OCR. The public post by Nicolas Camara (status/2083295265793212827) points the same way: do not stall an agent on OCR before you know the file needs it.

What actually ran on this Mac

uv venv --python 3.11 .venv
uv pip install --python .venv/bin/python pdf-inspector

The environment contained pdf-inspector==1.17.0 only.

File 1 — a one-page text PDF generated for an invoice-like layout:

classify_pdf: 0.901 ms, text_based, confidence 1.0, pages_needing_ocr = []
process_pdf: 5.671 ms → 181 characters of Markdown, including Amount: 0 CNY

File 2 — public TraceMonkey paper PDF, 1,016,315 bytes:

classify: 8.296 ms, still text_based, confidence 1.0, 0 OCR pages
extract: 51.342 ms → 83,804 characters of Markdown
library page_count=14; system file reported 6 pages. Public number used: 14

Limits we actually hit

A planted not-a-pdf.txt raised ValueError: Not a PDF: file appears to be plain text on three calls, each under 1 ms. Switching back to a real PDF restored classify and extract. We did not measure scanned pages.

Project: firecrawl/pdf-inspector

Measured report: https://jianailab.com/experiments/pdf-inspector-cf28f675

OpenMAIC Offline Test: 7,039 Passed, 45 Failed, and a Real Editable PPTX

sword luan — Mon, 31 Aug 2026 11:28:46 +0000

OpenMAIC reached the top of GitHub Trending, so I tested it as today's single main experiment instead of writing a feature summary.

The scope was deliberately narrow and reproducible:

install the pinned source,
run the full test suite,
run a production build,
call the project's own PPTX export path,
keep every failure and limitation in the record.

The first install failed

The first installation exited with code 1. The workspace compiled, but pnpm 11 stopped third-party build scripts that had not been explicitly approved. I kept that failure.

In a second isolated run, I allowed only the esbuild step required by the export path. The installation then exited with code 0.

The full test suite was not green

The complete test run produced:

7,039 passed
45 failed
79 skipped

Most failures came from five-second timeouts in concurrent browser-storage tests, with another timeout in a remote-image export test. This is not a “fully green” result.

The export path worked

The focused PPTX export checks were stronger: three files and five tests all passed.

The Next.js production build also exited with code 0 and generated 52 static pages.

For the harvest artifact, I called OpenMAIC's production buildPptxBlob function directly. It produced a four-slide, 79,985-byte editable Chinese PPTX. I unpacked the file and verified that all four slide XML files and the expected preset text were present.

No paid model API was used.

What this experiment verified

installation after a documented permission adjustment,
production build,
focused PPTX export,
a real downloadable and editable output file.

What it did not verify

multi-agent course generation,
TTS,
video generation,
real-time classroom interaction,
end-to-end course quality using a live model.

Full report: https://jianailab.com/reports/openmaic-offline-classroom-export-test

Harvest package: https://jianailab.com/experiments/openmaic/openmaic-harvest-pack.zip

Repository: https://github.com/THU-MAIC/OpenMAIC

GitHub is the land. We are the experimental field.

I Tested Two Open-Source Projects to Narrow Down an AI Side-Hustle Direction

sword luan — Sun, 30 Aug 2026 15:07:49 +0000

I started with a practical question: if someone has no customers, product, audience, or stable traffic, cannot code, does not want to appear on camera, has two hours a day, and can spend about RMB 1,000, which AI-enabled business direction should they test first?

Project 1: a map of possible directions

The first repository was XiaomingX's AI Money Maker Handbook. It collects four side-hustle principles, one starting strategy, ten technical monetization ideas, six technical business models, and twenty-five AI content directions.

I used it as a map. A beginner does not need to start with model training, a SaaS product, or a large investment. The more realistic path is to find a small demand, test it through public content, and keep reusable assets such as templates, scripts, images, and workflows.

The problem is that a map still contains too many roads. I needed a second project to compare those directions against one specific profile.

Project 2: a fit check based on real constraints

The second repository was muzhi-888's AI Side Hustle Checker.

I installed the Skill and entered the complete test profile:

basic computer, search, AI chat, image, and editing skills;
no programming experience;
two hours available per day;
RMB 1,000 in starting capital;
no customer base, product, audience, private community, or stable traffic;
no on-camera appearance;
a preference for selling through public platforms instead of finding service clients first.

The first recording only showed the installation and did not capture all three outputs. I rejected it as incomplete evidence. A second uninterrupted 112-second recording captured installation, input, scoring, and the complete result.

The result

The project scored each route on cash-flow speed, repeatability, and payback period. Each category had a maximum of ten points, for a total of thirty.

Niche digital products — 27/30
Small, lightweight physical products — 24/30
Content-assisted product sales — 22/30

The score measures fit with this test profile. It is not a success probability or revenue forecast.

The strongest route was a niche digital product: templates, checklists, spreadsheets, asset packs, printable files, or a focused information package for one concrete situation. These products are inexpensive to test, easy to revise, and can be sold repeatedly without inventory.

What this experiment actually proved

The test did not select a final product, and I will not pretend that it did. It narrowed a long list of industries into three product routes that match the stated constraints.

The next experiment will move from industry selection to product selection. I will use open-source projects to compare public demand, competing offers, keywords, production cost, and after-sales effort before choosing one specific digital or lightweight physical product.

Repositories:

Full experiment report and uninterrupted screen recording:

https://jianailab.com/reports/ai-side-hustle-checker-industry-selection

I Ran AI-Trader Through a Paper-Trading Cycle. It Passed Locally, Not in the Real Market.

sword luan — Sat, 29 Aug 2026 13:05:02 +0000

Can an AI agent actually participate in a trading workflow, or does it only produce a confident-looking buy/sell sentence?

I tested HKUDS/AI-Trader as today's only experiment for Jian AI Lab. The project is designed to give AI agents a place to register, publish strategies and signals, join challenges, keep records, and be followed or copied by other users.

The short result is clear: the local paper-trading path completed successfully. That does not prove real-market profitability, exchange connectivity, or fund safety.

What AI-Trader is trying to build

AI-Trader brings several pieces into one workflow:

AI agents can register and participate in trading challenges.
Strategies can publish signals and leave a history of activity.
Users can compare strategies through challenge results and rankings.
A paper-trading route lets a user follow the data flow before considering a real account.

The practical idea is not “ask an LLM for a prediction.” It is to make the agent's action part of a trackable system: signal, position, valuation, exit, settlement, and comparison.

Test environment

The run used a temporary SQLite database and mock prices. No exchange account, API key, real order, or real market feed was used.

Environment:

Python 3.12.13
pytest 9.1.1
Temporary SQLite database
Background tasks disabled for the controlled run

The local service health check was also available during the test.

What passed

The test groups produced these results:

Price-fetching tests: 8 passed
Experiment assignment, event records, agent recovery, and real-time price guards: 12 passed
Other local backend tests: 103 passed
One dependency deprecation warning was reported

These results show that the tested local paths can run in the selected environment. They are not evidence that live market data or a real exchange is connected.

The paper-trading cycle

I created a challenge and a simulated account, then ran this sequence:

Set the mock BTC price to 100.
Bought 2 BTC.
The account cash balance became 800.
Moved the mock price to 110.
The position value became 1020 and the displayed return was 2%.
Sold at 110.
Confirmed that the position was empty and cash was 1020.

The final record showed:

Status: settled
Trades: 2 simulated trades
Ranking: 1

The result is also recorded in the Jian AI Lab experiment report.

The important caveat about the 2%

The 2% figure is a reproducible calculation under fixed mock prices. It is not investment income and it cannot show that AI-Trader can make money in a live market.

What the run actually demonstrated is narrower and more useful: the records for joining a challenge, opening a position, marking its value, closing the position, and settling the challenge can line up in one controlled local flow.

What I did not claim

I did not run test_challenges.py because it may exercise external Hyperliquid or Polymarket paths. I also did not verify:

live market data;
a real exchange connection;
a real user account;
token or API-key handling in production;
live signal writing;
long-running background workers;
real order execution or withdrawal safety.

Copy trading adds its own risks: stale signals, slippage, service outages, leaked credentials, bad position sizing, and uncontrolled exposure. These risks remain even when a local paper-trading test passes.

Why this project is worth testing

The interesting part is the workflow boundary. AI-Trader tries to connect agent behavior to a stateful trading system instead of leaving the output as untracked text. That could make experiments, comparisons, and failure analysis more concrete for developers building financial agents.

The threshold is also higher than installing a Python package. A serious deployment would need a safe account model, reliable market data, clear authorization boundaries, operational monitoring, and a strong separation between simulation and real funds.

What should be tested next?

This is a public record from Jian AI Lab, not an endorsement. If you maintain AI-Trader, please tell us:

Which real-environment path should be tested first?
What user scenario best represents the intended product?
Which setup or safety requirements are missing from this report?

You are welcome to correct, supplement, respond to, or claim the project page. The full experiment, evidence, and boundary notes are available at jianailab.com.

Reproduction notes

The authoritative commands and full run record are maintained with the experiment materials. Reproduce the local path only with mock data or a disposable environment. Do not connect real funds while using this report as a starting point.

—

This article is part of the daily open-source testing workflow at 剑的 AI 实验室 / Jian AI Lab, where projects are explained in Chinese, tested with evidence, and published with failures and limits included.

GitHub Trending #1 Installed Cleanly, but the Main UI Still Would Not Start

sword luan — Fri, 28 Aug 2026 13:30:21 +0000

Yesterday I tested bilawalsidhu/gods-eye-view, the repository that reached the top of GitHub Trending for the global Today view. The project presents a 3D globe with aircraft, ships, satellites, earthquakes, fires, traffic, and public cameras. The demo is visually convincing. The useful question is whether a new user can install it, verify it, and reach the main experience.

The test was pinned to one commit

I pinned commit b22573a9db28e47c324821ebdd4c67bdb241c0e1 and used Linux, Node.js 24.19.0, and npm 11.9.0. The package requirement accepts Node 24.14.x through 24.x or Node 26.x.

The repository is a real application rather than a static demo. The source tree contains 315 files, the scripts directory contains 51 testing and QA files, Vite provides the browser and local proxy layer, and Cesium provides the globe.

Installation and build

I first ran npm ci --ignore-scripts to inspect the dependency installation boundary. After that completed successfully, I ran the official npm ci. Both runs installed 201 packages successfully.

The root project does not define preinstall, install, or postinstall. Several transitive packages still run installation scripts for platform binaries, browser support, or image components. That is normal for this dependency set, but it means the install should not be described as script-free.

npm audit --omit=dev --json reported zero known production-dependency vulnerabilities. A scan for common Google, OpenAI, GitHub, AWS, and private-key patterns found no hard-coded real credentials. These checks reduce obvious risk; they are not a complete security audit.

The production build exited with code 0. Vite finished in 6.67 seconds and generated about 31 MB across 419 files. A few bundles were larger than the configured warning threshold, including the Earth gravity model. The warnings did not block the build.

The test count needs an honest sentence

The official test command reported 2,587 passing assertions. A separate focus-allocation check also passed, bringing the visible total to 2,588 with zero failures.

The process then stayed alive for more than 90 seconds. I had to interrupt it manually, producing exit code 130. The accurate statement is that all visible assertions passed, while the official command did not exit cleanly in this environment. Those two facts belong together in a report.

The main UI is gated by a billable key

With Google, OpenAI, AISStream, Cesium, TomTom, and FIRMS credentials removed, the HTML shell still returned HTTP 200. Several endpoints returned explicit missing-key responses. FIRMS and TomTom status endpoints remained available, and TomTom can fall back to simulated traffic.

The main application follows a different path. Its initialization checks GOOGLE_MAPS_API_KEY and throws when the key is missing. The core 3D globe therefore cannot start without a Google Maps key. The key requires billing and is exposed to the browser bundle, so a real deployment should restrict its referrers, APIs, budget, and quota.

The repository README makes zero-key operation sound broader than the fixed commit actually allows. Zero keys are enough for installation, logical tests, the production build, and selected endpoint checks. They are not enough to enter the main 3D experience.

Browser QA was blocked by the environment

I also ran the track regression command. It found the Puppeteer Chrome binary, but Chrome could not create its process singleton socket in the container and exited with code 2. That is an environment permission failure, not evidence that the map is broken.

It does mean this run did not verify WebGL rendering, map tiles, drag behavior, target focus, or real-data overlays. Those checks remain open until the same application is run in a desktop environment where Chrome and WebGL can start.

Verdict

God’s Eye View is installable and buildable. Its visible test assertions passed, its production dependencies had no known audit findings, and its documentation makes several useful distinctions between live, modeled, reconstructed, and simulated data.

The core user experience still has a hard first-run condition. Without a Google Maps key with billing enabled, the main globe does not initialize. The browser regression is also still pending because of the test container’s socket restriction.

The complete Chinese experiment record, evidence, and author-claim invitation are collected at Jian AI Lab. The source repository is bilawalsidhu/gods-eye-view.

This is a stage-blocked result, not a failure to collect evidence. A useful open-source project can be worth studying even when the first real test ends with a missing credential and an uncompleted browser check.

I Built GitHub Trending #1. The Code Passed, but the Main UI Still Would Not Start

sword luan — Fri, 28 Aug 2026 00:32:00 +0000

God’s Eye View was the top project on GitHub Trending when we selected it for Jian AI Lab’s daily experiment.

The pitch is immediately compelling. It brings aircraft, vessels, satellites, earthquakes, wildfire data, traffic, CCTV sources, and other feeds into one 3D globe. The repository also makes a serious effort to label data as live, modeled, reconstructed, or simulated.

We tested commit b22573a9db28e47c324821ebdd4c67bdb241c0e1 on Linux with Node.js 24.19.0 and npm 11.9.0.

Installation and security checks

We first ran npm ci --ignore-scripts, reviewed the install-script sources, and then ran the normal npm ci. Both installations succeeded with 201 packages.

The root project has no preinstall, install, or postinstall hook. Transitive install scripts come from esbuild, fsevents, Puppeteer, and sharp.

npm audit --omit=dev reported no known vulnerabilities in production dependencies. A common secret-pattern scan did not find hard-coded live credentials. This is a limited check, not a full source audit.

The project talks to many external services, including Google Maps, OpenAI, OpenSky, AISStream, NASA FIRMS, TomTom, CelesTrak, OSM, Open-Meteo, GDELT, and Radio Browser. It is local-first, but it is not offline.

Server-side keys such as OpenAI and AISStream are read by the local Vite proxy. Google Maps and Cesium tokens are intentionally delivered to the browser. Users must restrict referrers and APIs and set provider budgets and quotas.

2,588 visible assertions passed

The main test suite reported 2,587 passing assertions and zero failures. A separate focus-allocation check added one more passing assertion.

The visible total was 2,588 passes and zero failures.

The process did not exit after the summary. We waited more than 90 seconds and interrupted it manually. The final exit code was 130.

The precise result is that all visible assertions passed, while the official test command did not complete with a clean exit in this environment. This may indicate an open handle, connection, or child process that remains alive after the tests finish.

The production build succeeded

npm run build completed with exit code 0 in 6.67 seconds.

The dist directory contained 419 files and was about 31 MB. Vite reported browser externalization warnings for two Node modules and warned about several bundles larger than 1.5 MB. The warnings did not block the build, but initial-load performance deserves a separate measurement.

The core 3D interface requires a Google Maps key

We cleared the Google, OpenAI, AIS, Cesium, TomTom, and FIRMS environment variables before starting the local server.

The HTML shell returned HTTP 200. Missing-key API routes behaved clearly. OpenAI, Google Places, and AISStream returned explicit 503 responses. FIRMS and TomTom status routes remained available, and traffic can fall back to a simulation without a TomTom key.

The main UI is different. src/main.js checks GOOGLE_MAPS_API_KEY during initialization and throws when it is absent. The interactive globe never becomes available.

The key must have the required Google Maps service enabled and billing turned on. Because it is visible in the browser, it also needs strict referrer and API restrictions plus a budget and quota policy.

Without that key, a user can install the repository, run logic tests, build the production bundle, and inspect several status endpoints. They cannot enter the core photorealistic 3D experience.

Browser and WebGL QA remain incomplete

The repository includes a test:track regression command. After starting the server, the script found Puppeteer’s downloaded Chrome. Chrome then crashed because the current container could not create its process-singleton socket.

The command exited with code 2. This is an execution-environment restriction, not an application assertion failure. It also means we did not verify WebGL, real map tiles, dragging, target focus, or live data accuracy.

Completing that stage requires a desktop environment that can run Chrome and WebGL, plus a billing-enabled Google Maps key with strict restrictions.

Verdict

God’s Eye View has a substantial engineering foundation. Installation and production build succeeded, and 2,588 visible assertions passed with zero failures. The repository is unusually explicit about live, modeled, reconstructed, and simulated data.

The first-run experience still has a significant barrier. A user must configure a billable browser-visible Google Maps key before the core interface appears. A functional OSM first-run mode would let people explore the basic globe and data layers before deciding whether they need photorealistic 3D tiles.

Our status for this experiment is a staged result blocked by external conditions. The code-level evidence is complete. End-to-end visual and live-data validation is not.

Sources

I Put GitHub Trending #1 Through a Real Architecture-Test Workflow

sword luan — Thu, 27 Aug 2026 19:54:08 +0000

GitHub Trending is useful for discovery. It tells you where attention is moving today. It does not tell you whether a project installs cleanly, survives a real task, diagnoses its own failures, or produces an artifact another person can inspect.

That distinction matters because many open-source recommendations stop at the same point. Someone finds a fast-growing repository, repeats the README, runs the official demo, and calls the project “tested.” A demo proves that the happy path exists. It does not prove that the tool can handle your system.

So I took Archify, which ranked first on GitHub Trending's global Today list that day, and put it through the full workflow at Jian AI Lab.

The task was not invented for the test. Archify had to map the real architecture of Jian AI Lab, pass its own validation gates, and deliver an interactive result that a reader could open without an online diagramming service.

The final result passed 9 of 9 showcase checks with zero errors and zero warnings. It took four validation rounds to get there.

What Archify actually does

Archify is a Node.js rendering and validation system designed for coding agents including Codex CLI, Claude Code, Cursor, and OpenCode. An agent produces a typed JSON intermediate representation. Archify then compiles that structured source into deterministic HTML and SVG.

It supports five diagram families.

Architecture diagrams
Workflow diagrams
Sequence diagrams
Data-flow diagrams
Lifecycle diagrams

The output can be a self-contained HTML file with light and dark themes, search, node focus, route tracing, guided views, and export controls. The repository is published under the MIT license.

The important part is the delivery gate. Archify does not treat “an HTML file was generated” as success. Its validators inspect schema, layout, routes, labels, projected text size, and other measurable conditions. A failing candidate returns structured diagnostics instead of silently replacing the last valid artifact.

That was the part I wanted to test.

The real task

Jian AI Lab is a Chinese platform built around a simple workflow. We discover open-source projects, verify their claims, test one project against a real problem, preserve failures and fixes, and turn the result into reusable public material.

The website includes a project library, a live activity arena, personal experiment fields, a harvest area for completed artifacts, member profiles, authentication, community APIs, a database, and media storage.

For this experiment, the architecture model contained 12 components and 11 relationships. The diagram had to communicate the main path through the product without turning into a wall of boxes and crossing lines.

The expected deliverables were also explicit.

A typed architecture JSON file
A machine-readable validation receipt
A self-contained interactive HTML diagram
A repeatable record of every failed and successful run
A public experiment report linked to the final artifact

Environment and first checks

The experiment ran with Node.js v24.19.0.

I began with Archify's environment diagnostic.

node bin/archify.mjs doctor

All 16 checks passed. The official demo also generated successfully.

Those results established that the tool and runtime were available. They did not count as completion because neither command tested the Jian AI Lab architecture.

The real work started when the actual components and relationships were written into the architecture model and validated with the showcase quality profile.

Validation round one — only 5 of 9 checks passed

The first real candidate looked plausible at a glance. The validator rejected it.

Only five of the nine showcase checks passed. Four concrete problems remained.

A route crossed through a node.
One route segment was only 7 pixels long.
A label overlapped another visual element.
Another label had no safe clearance from its route.

This was the most useful moment in the experiment. A conventional image generator could have delivered that diagram because it looked approximately correct. Archify identified the exact local failures and refused to release it as a showcase artifact.

The distinction is practical. If a diagram is used in a product page, architecture review, onboarding document, or technical presentation, a crossing line can imply the wrong dependency. A label placed against the wrong route can change the reader's interpretation. Visual defects can become factual defects.

Rounds two and three — local repairs, not a full rewrite

The next two rounds focused on the diagnosed subjects. I adjusted route geometry, endpoint direction, and label clearance while preserving the rest of the model.

That matters because uncontrolled regeneration is a poor repair strategy. If an agent rebuilds the entire diagram after every warning, unrelated nodes move, valid relationships change, and reviewers lose the ability to compare versions.

Archify's diagnostics made narrower correction possible. Each run reduced the remaining failures without discarding the valid structure.

After the routing and label problems were repaired, one issue remained. The projected minimum font size measured 5.8125 pixels. The showcase threshold was 6 pixels.

The difference was less than two-tenths of a pixel, but the candidate still failed. The validator did not round the result up or quietly treat it as acceptable.

Round four — 9 of 9

The final repair widened the canvas to 1380 pixels. That gave the labels enough projected space without shrinking the content below the readability threshold.

The fourth validation completed with the result we needed.

9 of 9 checks passed
0 errors
0 warnings

The final self-contained HTML file was 713,036 bytes. The JSON source and validation receipt were preserved beside it.

The diagram can be opened locally or from the experiment page. It supports node search, focus, route inspection, view changes, and both light and dark presentation themes.

You can open the interactive Jian AI Lab architecture diagram here.

One check was skipped, and it stayed skipped

Archify also provides an automated visual-check path that requires Chrome or Chromium. That browser was not available in the test environment, so the automatic visual check did not run.

I recorded it as skipped.

I did not convert “could not run” into “passed.” A separate manual browser review checked the light theme, dark theme, labels, controls, and basic interaction. The manual review passed, but it remained separate from the unavailable automated check.

This sounds like a minor reporting choice. It is one of the easiest places for an AI-assisted experiment to become dishonest. If skipped, inferred, and manually inspected results are all displayed with the same green check mark, the report stops being evidence.

The tool test also tested our own website

Running Archify through a real experiment exposed six weaknesses in the Jian AI Lab workflow.

Project states were too vague. A saved project could look similar to a verified or tested project.
Completed artifact files did not have a strong, permanent entry from the experiment record.
Command evidence and validation results were stored too loosely.
The harvest area did not summarize the experiment records automatically enough.
The long-form article could drift away from the evidence collected during the test.
The video package could become detached from the same experiment and repeat unverified claims.

We repaired the workflow so that a project now moves through explicit states.

Discovered → Verified → Tested → Produced

The artifact, report, commands, failures, screenshots, and publication package remain connected to the same experiment. A failed attempt stays in the timeline instead of disappearing after the successful run.

This is why the Archify test became more than a software review. The project revealed defects in the system used to review it.

What I learned from Archify

Archify's main value is not that it can draw a diagram quickly. Many tools can generate a polished diagram from a prompt.

Its stronger idea is validation before delivery.

The typed JSON source makes the diagram reproducible. Deterministic checks make failures measurable. Local diagnostics allow focused repairs. Atomic delivery prevents a broken candidate from replacing the last valid result. The final HTML keeps the artifact portable.

There are still limits.

The agent must model the real system accurately. Validation cannot prove facts that were never supplied.
Automated layout checks do not replace human judgment about emphasis and clarity.
The visual-check feature needs Chrome or Chromium.
Archify is not a WYSIWYG editor or a hosted sharing platform.

For architecture reviews, technical documentation, product explanations, and agent-generated system maps, those limits are reasonable. The project is especially useful when “looks good” is not an acceptable definition of done.

Reproducible outcome

The public experiment includes the failed rounds, repair history, final result, and links back to the original repository.

GitHub Trending helped us find the project. The experiment determined what it could actually do.