DEV Community: MT

hink - A short link system with just 10 lines of code.

MT — Sun, 31 Aug 2025 04:50:55 +0000

I’d like to share a small tool I recently created—hink, a short link system built with fewer than 10 lines of code.

This tool leverages Git and Serverless platforms to enable short link generation and access analytics.

Core Concept: The Clever Use of Git Commit Hashes

The core idea behind hink is simple: it uses the hash value of an empty Git commit as a unique identifier for the short link, while storing the original long URL in the commit message. When a short link is accessed, the system retrieves the commit message via GitHub’s .patch file interface, extracts the long URL, and redirects to it. Paired with a cloud platform’s WAF (Web Application Firewall) analytics dashboard, it also provides access statistics.

This solution has been successfully tested on the following platforms:

Cloudflare Workers/Snippets + Cloudflare WAF (Pro)
Tencent Cloud EdgeOne (Free)
Alibaba Cloud ESA (Free)

Code: Minimalist Implementation

The code for hink is extremely concise, with the core logic boiled down to just a few lines. Below are the implementations for different platforms.

Cloudflare Workers / Alibaba Cloud ESA Version

const GIT_REPO = "https://github.com/ccbikai/hink"
export default {
  async fetch(request) {
    const { pathname } = new URL(request.url)
    const gitPatch = `${GIT_REPO}/commit${pathname}.patch`
    const patch = await fetch(gitPatch, { cf: { cacheEverything: true, cacheTtlByStatus: { '200-299': 86400 } }}).then(res => res.text())
    const url = pathname === '/' ? GIT_REPO : patch.match(/^Subject:\s*\[PATCH\](.*)$/m)?.[1]?.trim()
    return Response.redirect(url || GIT_REPO)
  }
}

Tencent Cloud EdgeOne Version

const GIT_REPO = "https://github.com/ccbikai/hink"
addEventListener("fetch", async (event) => {
  const { pathname } = new URL(event.request.url)
  const gitPatch = `${GIT_REPO}/commit${pathname}.patch`
  const patch = await fetch(gitPatch).then(res => res.text())
  const url = pathname === '/' ? GIT_REPO : patch.match(/^Subject:\s*\[PATCH\](.*)$/m)?.[1]?.trim()
  event.respondWith(new Response(null, { status: 302, headers: { Location: url || GIT_REPO } }))
});

Once deployed to a Serverless platform and bound to a domain, you’ll have your own short link service.

Why Build hink?

Short link services aren’t new—there are plenty of tools out there. However, hink’s goal was to explore a minimalist and creative approach. By using Git commit hashes, it eliminates the need for database management, relies on GitHub for storage, and utilizes cloud platform WAF features for easy access analytics. For me, this was a technical experiment to solve a real problem with the least amount of code.

Demo: Performance Across Three Platforms

I’ve deployed hink on Cloudflare Workers, Alibaba Cloud ESA, and Tencent Cloud EdgeOne, and monitored access stats via their WAF dashboards. Below are screenshots of the performance on each platform:

Cloudflare Workers

Alibaba Cloud ESA

Tencent Cloud EdgeOne

Project Repository

Demystifying SSH AI Chat: How It Works

MT — Fri, 01 Aug 2025 14:07:03 +0000

Hello everyone, I'm MT, and today I'd like to share my recent project - SSH AI Chat.

Project Introduction

SSH AI Chat is an AI chat application that you can connect to directly via SSH. Using it is incredibly simple:

ssh username@chat.aigc.ing

Note⚠️: Replace username with your GitHub username

That's right, it's that simple! You don't need to install any client or open a browser. Just an SSH client is all you need to chat with AI.

As a developer with a strong interest in TUI applications, I've always felt that chatting in the terminal is a really cool concept. I was initially amazed by itter.sh - a social network you could access via SSH! This made me realize that SSH isn't just for connecting to servers; it can be used for many interesting things.

That's when I had this idea: how cool would it be to chat with AI via SSH! No software installation, no browser needed, just type ssh yourname@chat.aigc.ing in your terminal to start chatting.

Project Architecture

Core Technology Stack

SSH Server: Node.js + ssh2 module
UI Framework: React + Ink (for terminal rendering)
Database: PostgreSQL / PGLite (optional)
Cache: Redis / ioredis-mock (optional)
AI Integration: Vercel AI SDK

System Architecture Diagram

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   SSH Client    │    │   SSH Server    │    │   React App     │
│                 │    │                 │    │                 │
│  ssh username@  │───▶│  Node.js +      │───▶│  Ink UI +       │
│  chat.aigc.ing  │    │  ssh2           │    │  React Hooks    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                               │
                               ▼
                       ┌─────────────────┐
                       │   AI Services   │
                       │                 │
                       │  OpenAI API     │
                       │  Gemini API     │
                       │  DeepSeek API   │
                       └─────────────────┘

Core Module Analysis

1. SSH Server Module

This is the core of the entire application, responsible for handling SSH connections and authentication. The system automatically handles key verification, GitHub public key authentication, login restrictions, and rate limiting.

2. Authentication System

The most clever design is the use of GitHub public key authentication. Users don't need to register; they can log in directly using their GitHub SSH keys. The system retrieves the user's GitHub public key for verification, caching it every 6 hours, making it both secure and efficient.

3. Terminal UI System

Using the Ink framework to render React components in the terminal. Imagine the React components you usually write, now rendering not in a browser but displaying in the terminal! It supports multilingual interfaces, real-time chat, history, model selection, and responsive layouts.

4. Chat System

Using Vercel AI SDK to handle AI conversations. When you type a message in the terminal, the system receives the message, loads conversation history, selects a model, displays streaming responses in real-time, and saves conversation records. It supports streaming responses, multi-model support, chain-of-thought display, and conversation history management.

Technical Challenges and Solutions

1. Terminal Rendering Challenges

The biggest challenge was implementing complex UI interfaces in the terminal. Using the Ink framework to render React components to the terminal and implementing a virtual PTY to handle terminal I/O. Displaying AI responses in Markdown in the terminal requires a dedicated worker process to handle the conversion to display bold, italic, and code blocks.

2. SSH Session Management

Managing multiple SSH sessions and states requires creating independent React application instances for each connection, using Context API to manage global state, and implementing session lifecycle management.

3. Real-time Streaming Responses

AI responses are streamed, and if the interface refreshes with every byte received, the terminal would freeze. Using Vercel AI SDK's streamText with throttled updates, refreshing every 300ms, ensures smooth performance without lag.

4. Flexibility in Data Storage

The project supports both PostgreSQL and PGLite databases, as well as Redis and in-memory caching, allowing the project to run independently or be deployed in production environments.

Interesting Design Details

1. GitHub Authentication

The coolest design! Users don't need to register; they can log in directly using their GitHub SSH keys, which is both convenient and secure.

2. Multi-model Support

Supports multiple AI models including DeepSeek-V3/DeepSeek-R1, Gemini-2.5-Flash/Gemini-2.5-Pro, including chain-of-thought display. It requires handling differences between various model APIs while providing a unified interface.

3. Internationalization Support

Complete i18n support, automatically detecting user language preferences through the LANG environment variable, with support for Chinese and English switching.

4. Keyboard Shortcuts

Ctrl+C to exit the application, N for new conversation, I to focus on input box, ? to view help. There are also some Easter egg features.

Development Insights

1. Possibilities of Terminal Applications

This project showed me the enormous potential of terminal applications. Through the Ink framework, we can implement complex interactive interfaces in the terminal.

2. Creative Uses of SSH

SSH is not just a remote management tool; it's also a powerful application platform. Through SSH, we can create cross-platform client applications without users needing to install any additional software.

3. Modern Technology Stack

Although this is a terminal application, we used the most modern technology stack: React, TypeScript, Vercel AI SDK, etc. This proves that terminal applications can also be "modern."

4. Pitfalls Encountered

SSH terminal compatibility issues, performance problems with streaming output, and performance challenges in conversation history management were all difficulties that needed to be overcome during development.

Future Outlook

Plans include supporting more AI models, including local models like Ollama, and supporting the MCP (Model Context Protocol) to allow users to extend functionality through plugins.

Conclusion

SSH AI Chat is an innovative project that integrates multiple technologies. It demonstrates:

The modern possibilities of terminal applications
Flexible applications of the SSH protocol
React's adaptability across different platforms
The popularization of AI technology

This project made me realize that technology isn't just for solving problems; it can also be fun. Combining SSH and AI created an unexpected experience.

I hope this project brings some inspiration to everyone, and let's explore the boundaries of technology together!

Try It Out

If you want to try it, you can use:

ssh username@chat.aigc.ing

Project URL: https://github.com/ccbikai/ssh-ai-chat

If you have any questions or suggestions about this project, feel free to discuss them on GitHub. You're also welcome to Star this project - your support is my motivation to continue developing!

Run MCP Server in a Docker sandbox

MT — Wed, 23 Apr 2025 14:01:48 +0000

MCP is a hot protocol in the AI development industry this year, but its Client/Server (C/S) architecture requires users to run the MCP Server locally.

Common ways to run MCP Server include stdio methods like npx (NPM ecosystem), uvx (Python ecosystem), Docker, and HTTP (SSE/Streaming) methods. However, running commands with npx and uvx carries significant risks. Accidentally executing a malicious package could lead to sensitive data exposure, posing a major security threat. For details, you can refer to Invariant's article MCP Security Notification: Tool Poisoning Attacks.

As a software industry professional, I have a high degree of concern for security. I asked ChatGPT to compile a list of NPM and PyPI supply chain attack incidents from the past 5 years, and it was chilling.

Time	Event	Summary and Scope of Impact
February 2021	"Dependency Confusion" Vulnerability Disclosure	Security researcher Alex Birsan utilized the Dependency Confusion technique to upload packages to NPM/PyPI with the same names as internal libraries used by multiple companies, successfully infiltrating the internal servers of 35 major companies including Apple and Microsoft (PyPI flooded with 1,275 dependency confusion packages). This demonstration sparked high concern within the industry regarding supply chain risks.
October 2021	UAParser.js Library Hijacked	The popular library ua-parser-js on NPM, with over 7 million weekly downloads, was compromised by attackers via the maintainer's account to publish malicious versions (A Timeline of SSC Attacks, Curated by Sonatype). Infected versions implanted password-stealing trojans and cryptocurrency miners upon installation, affecting a large number of developer systems.
October 2021	Poisoning via Fake Roblox Libraries	Attackers uploaded multiple packages impersonating Roblox API on NPM (e.g., noblox.js-proxy), containing obfuscated malicious code. These packages would implant trojans and ransomware payloads after installation (A Timeline of SSC Attacks, Curated by Sonatype). These packages were downloaded thousands of times, demonstrating attackers used typosquatting to trick game developers.
November 2021	COA and RC Libraries Successively Hijacked	Popular libraries on NPM, coa (millions of weekly downloads) and rc (14 million weekly downloads), were successively compromised to publish malicious versions. The affected versions executed credential-stealing trojans similar to the UAParser.js case, at one point causing build pipelines to break for numerous projects globally using frameworks like React (A Timeline of SSC Attacks, Curated by Sonatype) (A Timeline of SSC Attacks, Curated by Sonatype). Official investigations determined the cause in both cases was compromised maintainer accounts.
January 2022	Colors/Faker Open Source Libraries "Suicide"	The authors of the famous color formatting library colors.js and test data generation library faker.js, out of protest, injected destructive code like infinite loops in the latest versions, causing thousands of projects, including those at companies like Meta (Facebook) and Amazon, to crash (A Timeline of SSC Attacks, Curated by Sonatype) (While not an external attack, it falls within the scope of supply chain poisoning).
January 2022	PyPI: 1,275 Malicious Packages Deployed in Bulk	A single user frantically published 1,275 malicious packages to PyPI in one day on January 23rd (A Timeline of SSC Attacks, Curated by Sonatype). Most of these packages impersonated the names of well-known projects or companies (e.g., xcryptography, Sagepay, etc.). After installation, they collected fingerprint information like hostname, IP, etc., and exfiltrated it to the attackers via DNS/HTTP (PyPI flooded with 1,275 dependency confusion packages) (PyPI flooded with 1,275 dependency confusion packages). PyPI administrators took down all related packages within an hour of receiving the report (PyPI flooded with 1,275 dependency confusion packages).
March 2022	Node-ipc "Protestware" Incident	The author of node-ipc, a commonly used front-end build library, added malicious code in versions v10.1.1–10.1.3: when detecting client IPs belonging to Russia or Belarus, it would wipe the file system and overwrite files with heart emojis ([Corrupted open-source software enters the Russian battlefield
October 2022	LofyGang Large-Scale Poisoning Campaign	Security companies discovered a group named "LofyGang" distributed nearly 200 malicious packages on NPM (LofyGang Distributed ~200 Malicious NPM Packages to Steal Credit Card Data). These packages implanted trojans through typosquatting and by impersonating common library names, stealing developers' credit card information, Discord accounts, and game service login credentials, accumulating thousands of installations (LofyGang Distributed ~200 Malicious NPM Packages to Steal Credit Card Data). This was an organized cybercrime activity that lasted over a year.
December 2022	PyTorch-nightly Dependency Chain Attack	Well-known deep learning framework PyTorch disclosed that its nightly version suffered a dependency confusion supply chain attack between December 25-30 ([Malicious PyTorch dependency ‘torchtriton’ on PyPI
March 2023	"W4SP Stealer" Trojan Rampant on PyPI	Security researchers successively discovered a large number of malicious packages carrying the W4SP Stealer information-stealing trojan appearing on PyPI (W4SP Stealer Discovered in Multiple PyPI Packages Under Various Names). These trojans have many aliases (e.g., ANGEL Stealer, PURE Stealer, etc.) but essentially all belong to the W4SP family, specifically designed to steal information like user passwords, cryptocurrency wallets, and Discord tokens (W4SP Stealer Discovered in Multiple PyPI Packages Under Various Names). A single report revealed 16 such malicious packages (e.g., modulesecurity, easycordey, etc.) (W4SP Stealer Discovered in Multiple PyPI Packages Under Various Names). PyPI initiated a cleanup targeting such trojans and strengthened upload detection.
August 2023	Lazarus Group Attacks PyPI	ReversingLabs reported that a branch of the North Korean hacking group Lazarus published over two dozen (more than 24) malicious packages disguised as popular libraries on PyPI (codenamed "VMConnect" operation) (Software Supply Chain Attacks: A (partial) History). These packages attempted to target users in specific industries (e.g., finance) to implant remote access trojans. It is claimed this attack is linked to previous similar activities targeting NuGet, showing state-sponsored hackers' interest in the open-source supply chain.
2024 and Beyond	Ongoing Supply Chain Threats	Since 2024, new poisoning incidents continue to emerge on NPM and PyPI. For example, in early 2024, fake VS Code-related NPM packages were found to contain remote control spyware (A Timeline of SSC Attacks, Curated by Sonatype), and PyPI packages impersonating Solana libraries to steal crypto wallet keys (A Timeline of SSC Attacks, Curated by Sonatype) were discovered. This indicates that supply chain attacks have become a normalized threat, requiring the ecosystem to continuously raise vigilance and defense capabilities.

I complained a bit on Twitter, and while complaining, I saw a tweet from a friend who had just encountered a supply chain attack incident.

Fortunately, @TBXark recommended his MCP Proxy project, which makes it very convenient to run MCP Server in Docker. His initial goal was to run MCP Server on a server to reduce client load and facilitate mobile client calls. However, Docker's inherent isolation features perfectly aligned with my requirement for a sandbox.

MCP Proxy runs MCP Servers in Docker and converts the protocol to MCP SSE, allowing users to make all calls via the SSE protocol from the MCP client. This can significantly reduce the risk of arbitrary file reading caused by directly running npx and uvx. If deployed on an overseas server, it can also help solve network issues.

However, it is currently still possible to read the /config/config.json configuration file of MCP Proxy, but the risk is manageable. I have also raised a feature request with the developer to configure the config file with 400 permissions and run the npx and uvx commands as the nobody user. If this can be implemented, it will perfectly solve the arbitrary file reading issue.

Running MCP Proxy

If you have your own VPS with Docker deployed, you can use the following command to run MCP Proxy.

docker run -d -p 9090:9090 -v /path/to/config.json:/config/config.json ghcr.io/tbxark/mcp-proxy:latest

If you don't have your own VPS, you can use the free container service provided by claw.cloud ($5 credit per month, GitHub registration must be older than 180 days).

Since Claw has container size limitations, we need to use the following environment variables to configure the cache directories for npx and uvx to prevent container crashes.

UV_CACHE_DIR=/cache/uv
npm_config_cache=/cache/npm

Simultaneously mount 10GB of storage under the /cache path. Refer to my configuration: 0.5c CPU, 512M Memory, 10G Disk.

The final configuration is as follows:

Configuring MCP Proxy

The configuration file needs to be mounted at the /config/config.json path. For the complete configuration, please refer to https://github.com/TBXark/mcp-proxy?tab=readme-ov-file#configurationonfiguration.

Below is my configuration, for your reference.

{
    "mcpProxy": {
        "baseURL": "https://mcp.miantiao.me",
        "addr": ":9090",
        "name": "MCP Proxy",
        "version": "1.0.0",
        "options": {
          "panicIfInvalid": false,
          "logEnabled": true,
          "authTokens": [
            "miantiao.me"
          ]
        }
    },
    "mcpServers": {
        "github": {
            "command": "npx",
            "args": [
                "-y",
                "@modelcontextprotocol/server-github"
            ],
            "env": {
                "GITHUB_PERSONAL_ACCESS_TOKEN": "<YOUR_TOKEN>"
            }
        },
        "fetch": {
            "command": "uvx",
            "args": [
                "mcp-server-fetch"
            ]
        },
        "amap": {
            "url": "https://mcp.amap.com/sse?key=<YOUR_TOKEN>"
        }
    }
}

Calling MCP proxy

Taking ChatWise calling fetch as an example, just configure the SSE protocol directly.

Isn't it simple? When ChatWise releases its mobile version, calling it this way will also be fully usable.

Use Cloudflare Workers to concat audio files

MT — Sat, 19 Apr 2025 11:09:12 +0000

I recently updated the Hacker News Chinese Podcast to use a dual-speaker format. Since current speech synthesis models don't handle two-person dialogues very well, I needed a way to merge the audio files for each speaker.

The project runs on the Cloudflare Workers runtime, which lacks many Node.js features and cannot call C++ extensions. Furthermore, Cloudflare Containers aren't generally available yet. This meant I had to use the Browser Rendering API for the audio merging task.

FFmpeg is the standard tool for merging audio files, and fortunately, it can now run in the browser via WASM. So, the overall technical approach is:

Use a Worker Binding to launch a browser instance (via the Browser Rendering API).
Have the browser navigate to an audio merging page, perform the merge operation on the audio files, and return the result as a Blob.
Receive the Blob back in the Worker and upload it to R2 storage.

The overall code footprint for this isn't large, but debugging was tricky because Browser Rendering runs remotely.

Here's the final implementation code:

Browser-Side Audio Merging Code

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Audio</title>
  </head>
  <body>
    <script>
      const concatAudioFilesOnBrowser = async (audioFiles) => {
        const script = document.createElement('script')
        script.src = 'https://unpkg.com/@ffmpeg/ffmpeg@0.11.6/dist/ffmpeg.min.js'
        document.head.appendChild(script)
        await new Promise((resolve) => (script.onload = resolve))

        const { createFFmpeg, fetchFile } = FFmpeg
        const ffmpeg = createFFmpeg({ log: true })

        await ffmpeg.load()

        // Download and write each file to FFmpeg's virtual file system
        for (const [index, audioFile] of audioFiles.entries()) {
          const audioData = await fetchFile(audioFile)
          ffmpeg.FS('writeFile', `input${index}.mp3`, audioData)
        }

        // Create a file list for ffmpeg concat
        const fileList = audioFiles.map((_, i) => `file 'input${i}.mp3'`).join('\n')
        ffmpeg.FS('writeFile', 'filelist.txt', fileList)

        // Execute FFmpeg command to concatenate files
        await ffmpeg.run(
          '-f',
          'concat',
          '-safe',
          '0',
          '-i',
          'filelist.txt',
          '-c:a',
          'libmp3lame',
          '-q:a',
          '5',
          'output.mp3',
        )

        // Read the output file
        const data = ffmpeg.FS('readFile', 'output.mp3')

        // Create a downloadable link
        const blob = new Blob([data.buffer], { type: 'audio/mp3' })

        // Clean up
        audioFiles.forEach((_, i) => {
          ffmpeg.FS('unlink', `input${i}.mp3`)
        })
        ffmpeg.FS('unlink', 'filelist.txt')
        ffmpeg.FS('unlink', 'output.mp3')

        return blob
      }
    </script>
  </body>
</html>

Worker Codes

export async function concatAudioFiles(audioFiles: string[], BROWSER: Fetcher, { workerUrl }: { workerUrl: string }) {
  const browser = await puppeteer.launch(BROWSER)
  const page = await browser.newPage()
  await page.goto(`${workerUrl}/audio`)

  console.info('start concat audio files', audioFiles)
  const fileUrl = await page.evaluate(async (audioFiles) => {
    // JS runs here in the browser.
    // @ts-expect-error Objects in the browser
    const blob = await concatAudioFilesOnBrowser(audioFiles)

    const result = new Promise((resolve, reject) => {
      const reader = new FileReader()
      reader.onloadend = () => resolve(reader.result)
      reader.onerror = reject
      reader.readAsDataURL(blob)
    })
    return await result
  }, audioFiles) as string

  console.info('concat audio files result', fileUrl.substring(0, 100))

  await browser.close()

  const response = await fetch(fileUrl)
  return await response.blob()
}

const audio = await concatAudioFiles(audioFiles, env.BROWSER, { workerUrl: env.HACKER_NEWS_WORKER_URL })
return new Response(audio)

The above code is basically written by Cursor, and the final effect can be viewed at Hacker News Code Repository.

RSS.Beauty - Make Your RSS Beautiful!

MT — Tue, 31 Dec 2024 14:50:48 +0000

The tool that has been delayed for nearly half a year is finally completed.

RSS.Beauty is an RSS beautification tool based on XSLT technology that transforms ordinary RSS/Atom feeds into elegant reading interfaces.

Key Features

🎨 Beautiful reading interface
🔄 Support for RSS 2.0 and Atom 1.0
📱 Responsive design, mobile-friendly
🔌 One-click subscription to major RSS readers
🖥 Self-hosting support

Quick Start

Visit RSS.Beauty and enter any RSS feed URL to try it out.

Or visit https://rss.beauty/rss?url=https%3A%2F%2Fgithub.com%2Fccbikai%2FRSS.Beauty%2Freleases.atom to try it out.

Tech Stack

Deployment

Detailed deployment guide can be found in Deployment Guide.

Serverless

Support deployment to Cloudflare Pages, Vercel, Netlify, etc. After Fork this project, follow the platform tutorial to deploy.

Docker

docker pull ghcr.io/ccbikai/rss.beauty:main
docker run -d --name rss-beauty -p 4321:4321 ghcr.io/ccbikai/rss.beauty:main

Credits

Tailus UI

Sponsor

Run Python programs easily in the browser.

MT — Sat, 21 Dec 2024 09:57:34 +0000

Microsoft recently open-sourced MarkItDown, a program that converts Office files to Markdown format. The project quickly climbed to GitHub's trending list upon release.

However, since MarkItDown is a Python program, it might be challenging for non-technical users to use. To address this issue, I thought of using WebAssembly technology to run Python code directly in the browser.

Pyodide is an open-source program that runs Python in the browser, using WebAssembly to port CPython, so it supports all Python syntax. Cloudflare's Python Workers also use Pyodide.

Pyodide is a port of CPython to WebAssembly/Emscripten.

Pyodide makes it possible to install and run Python packages in the browser using micropip. Any pure Python package with wheels available on PyPI is supported.

Many packages with C extensions have also been ported for use with Pyodide. These include common packages like regex, PyYAML, lxml, and scientific Python packages including NumPy, pandas, SciPy, Matplotlib, and scikit-learn. Pyodide comes with a robust JavaScript ⟺ Python foreign function interface that allows you to freely mix these languages in your code with minimal friction. This includes comprehensive support for error handling, async/await, and more.

When used in the browser, Python has full access to the Web APIs.

Trying to run MarkItDown was surprisingly smooth, proving that WebAssembly is truly the future of browsers.

The main challenges faced and solutions:

File Transfer Issue: How to pass user-selected files to the Python runtime in the Worker?
Dependency Installation Issue: Limited access to PyPI in mainland China.

Eventually, we successfully implemented a MarkItDown tool that runs entirely in the browser. Feel free to try it out at Office File to Markdown.

Here's the core code for running Python in the Worker:

// eslint-disable-next-line no-undef
importScripts('https://testingcf.jsdelivr.net/pyodide/v0.26.4/full/pyodide.js')


async function loadPyodideAndPackages() {
  // eslint-disable-next-line no-undef
  const pyodide = await loadPyodide()
  globalThis.pyodide = pyodide

  await pyodide.loadPackage('micropip')

  const micropip = pyodide.pyimport('micropip')

  // micropip.set_index_urls([
  // 'https://pypi.your.domains/pypi/simple',  
  // ])

  await micropip.install('markitdown==0.0.1a2')
}

const pyodideReadyPromise = loadPyodideAndPackages()

globalThis.onmessage = async (event) => {
  await pyodideReadyPromise

  const file = event.data
  try {
    console.log('file', file)
    const startTime = Date.now()
    globalThis.pyodide.FS.writeFile(`/${file.filename}`, file.buffer)

    await globalThis.pyodide.runPythonAsync(`
from markitdown import MarkItDown

markitdown = MarkItDown()

result = markitdown.convert("/${file.filename}")
print(result.text_content)

with open("/${file.filename}.md", "w") as file:
  file.write(result.text_content)
`)
    globalThis.postMessage({
      filename: `${file.filename}.md`,
      content: globalThis.pyodide.FS.readFile(`/${file.filename}.md`, { encoding: 'utf8' }),
      time: Date.now() - startTime,
    })
  }
  catch (error) {
    globalThis.postMessage({ error: error.message || 'convert error', filename: file.filename })
  }
}

Use Cloudflare Snippets to set up a Docker Registry Mirror

MT — Sat, 21 Dec 2024 08:17:40 +0000

Using Cloudflare Workers to set up Docker image proxies works fine for personal use with low request volumes. However, if made public, high request volumes can incur significant costs.

Actually, Cloudflare has an even lighter JS Runtime called Cloudflare Snippets, though it comes with stricter limitations: 5ms CPU execution time, 2MB memory limit, and 32KB code size limit. Still, it's sufficient for request rewriting purposes.

Unfortunately, Cloudflare Snippets isn't currently available for Free plans, although their blog mentions that Free plans can create 5 Snippets.

If you have a Pro plan, you can slightly modify the Cloudflare Workers code to run it. It supports Docker Hub, Google Container Registry, GitHub Container Registry, Amazon Elastic Container Registry, Kubernetes Container Registry, Quay, and Cloudsmith.

Modified code:

// Raw Codes: https://github.com/ciiiii/cloudflare-docker-proxy/blob/master/src/index.js

const CUSTOM_DOMAIN = 'your.domains'
const MODE = 'production'

const dockerHub = 'https://registry-1.docker.io'

const routes = {
    // production
    [`docker.${CUSTOM_DOMAIN}`]: dockerHub,
    [`quay.${CUSTOM_DOMAIN}`]: 'https://quay.io',
    [`gcr.${CUSTOM_DOMAIN}`]: 'https://gcr.io',
    [`k8s-gcr.${CUSTOM_DOMAIN}`]: 'https://k8s.gcr.io',
    [`k8s.${CUSTOM_DOMAIN}`]: 'https://registry.k8s.io',
    [`ghcr.${CUSTOM_DOMAIN}`]: 'https://ghcr.io',
    [`cloudsmith.${CUSTOM_DOMAIN}`]: 'https://docker.cloudsmith.io',
    [`ecr.${CUSTOM_DOMAIN}`]: 'https://public.ecr.aws',

    // staging
    [`docker-staging.${CUSTOM_DOMAIN}`]: dockerHub,
}

async function handleRequest(request) {
    const url = new URL(request.url)
    const upstream = routeByHosts(url.hostname)
    if (upstream === '') {
        return new Response(
            JSON.stringify({
                routes,
            }), {
                status: 404,
            },
        )
    }
    const isDockerHub = upstream === dockerHub
    const authorization = request.headers.get('Authorization')
    if (url.pathname === '/v2/') {
        const newUrl = new URL(`${upstream}/v2/`)
        const headers = new Headers()
        if (authorization) {
            headers.set('Authorization', authorization)
        }
        // check if need to authenticate
        const resp = await fetch(newUrl.toString(), {
            method: 'GET',
            headers,
            redirect: 'follow',
        })
        if (resp.status === 401) {
            return responseUnauthorized(url)
        }
        return resp
    }
    // get token
    if (url.pathname === '/v2/auth') {
        const newUrl = new URL(`${upstream}/v2/`)
        const resp = await fetch(newUrl.toString(), {
            method: 'GET',
            redirect: 'follow',
        })
        if (resp.status !== 401) {
            return resp
        }
        const authenticateStr = resp.headers.get('WWW-Authenticate')
        if (authenticateStr === null) {
            return resp
        }
        const wwwAuthenticate = parseAuthenticate(authenticateStr)
        let scope = url.searchParams.get('scope')
        // autocomplete repo part into scope for DockerHub library images
        // Example: repository:busybox:pull => repository:library/busybox:pull
        if (scope && isDockerHub) {
            const scopeParts = scope.split(':')
            if (scopeParts.length === 3 && !scopeParts[1].includes('/')) {
                scopeParts[1] = `library/${scopeParts[1]}`
                scope = scopeParts.join(':')
            }
        }
        return await fetchToken(wwwAuthenticate, scope, authorization)
    }
    // redirect for DockerHub library images
    // Example: /v2/busybox/manifests/latest => /v2/library/busybox/manifests/latest
    if (isDockerHub) {
        const pathParts = url.pathname.split('/')
        if (pathParts.length === 5) {
            pathParts.splice(2, 0, 'library')
            const redirectUrl = new URL(url)
            redirectUrl.pathname = pathParts.join('/')
            return Response.redirect(redirectUrl, 301)
        }
    }
    // foward requests
    const newUrl = new URL(upstream + url.pathname)
    const newReq = new Request(newUrl, {
        method: request.method,
        headers: request.headers,
        redirect: 'follow',
    })
    const resp = await fetch(newReq)
    if (resp.status === 401) {
        return responseUnauthorized(url)
    }
    return resp
}

function routeByHosts(host) {
    if (host in routes) {
        return routes[host]
    }
    if (MODE === 'debug') {
        return dockerHub
    }
    return ''
}

function parseAuthenticate(authenticateStr) {
    // sample: Bearer realm="https://auth.ipv6.docker.com/token",service="registry.docker.io"
    // match strings after =" and before "
    const re = /(?<==")(?:\\.|[^"\\])*(?=")/g
    const matches = authenticateStr.match(re)
    if (matches == null || matches.length < 2) {
        throw new Error(`invalid Www-Authenticate Header: ${authenticateStr}`)
    }
    return {
        realm: matches[0],
        service: matches[1],
    }
}

async function fetchToken(wwwAuthenticate, scope, authorization) {
    const url = new URL(wwwAuthenticate.realm)
    if (wwwAuthenticate.service.length) {
        url.searchParams.set('service', wwwAuthenticate.service)
    }
    if (scope) {
        url.searchParams.set('scope', scope)
    }
    const headers = new Headers()
    if (authorization) {
        headers.set('Authorization', authorization)
    }
    return await fetch(url, {
        method: 'GET',
        headers
    })
}

function responseUnauthorized(url) {
    const headers = new(Headers)()
    if (MODE === 'debug') {
        headers.set(
            'Www-Authenticate',
            `Bearer realm="http://${url.host}/v2/auth",service="cloudflare-docker-proxy"`,
        )
    } else {
        headers.set(
            'Www-Authenticate',
            `Bearer realm="https://${url.hostname}/v2/auth",service="cloudflare-docker-proxy"`,
        )
    }
    return new Response(JSON.stringify({
        message: 'UNAUTHORIZED'
    }), {
        status: 401,
        headers,
    })
}

export default {
    fetch: handleRequest,
}

Cloudflare PyPI Mirror

MT — Wed, 18 Dec 2024 13:06:14 +0000

Pyodide is a library that runs Python in WebAssembly, using Micropip to install packages from PyPI. Due to WebAssembly's requirements for CORS and PEP 691 when running in browsers, and the fact that Tsinghua's TUNA mirror doesn't support CORS, this creates some challenges.

PyPI is not directly accessible in mainland China, but there are many mirrors available. Institutions like Tsinghua University, Alibaba Cloud, Tencent Cloud, and Huawei Cloud provide mirror services. However, except for Tsinghua's TUNA mirror, none of them support the JSON-based Simple API for Python (PEP 691).

Since WebAssembly requires both CORS support and PEP 691 compliance when running in browsers, and Tsinghua's TUNA mirror doesn't support CORS, there might not be any suitable PyPI mirrors available in mainland China for Micropip.

Given this situation, I've set up a Cloudflare-based mirror that supports both PEP 691 and CORS.

You can build this using either Workers or Snippets, each with their own advantages and disadvantages:

Workers

Pros: Available with the free plan.

Cons: Generates many Worker requests, which might exceed free plan limits and require payment or become unusable.

Snippets

Pros: Doesn't generate Worker requests, supports high usage volumes.
Cons: Currently only available for Pro plans and above, not available on Free tier.

Code

The corresponding code has been open-sourced and is available at:

https://github.com/ccbikai/cloudflare-pypi-mirror

Minimal Docker Image Packaging for Vite SSR Projects

MT — Sat, 31 Aug 2024 05:14:00 +0000

Recently, I've been preparing to migrate projects hosted on Cloudflare, Vercel, and Netlify to my own VPS to run via Docker. I revisited Docker image packaging. However, even a small project ended up being packaged into a 1.05GB image, which is clearly unacceptable. So, I researched minimal Docker image packaging for Node.js projects, reducing the image size from 1.06GB to 135MB.

The example project is an Astro project using Vite as the build tool, running in SSR mode.

Version 0

The main idea is to use a minimal system image, opting for the Alpine Linux image.

Following the Astro official documentation for Server-Side Rendering (SSR), I replaced the base image with node:lts-alpine, and switched from NPM to PNPM. The resulting image size was 1.06GB, which is the worst-case scenario.

FROM node:lts-alpine AS base

ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable

WORKDIR /app

COPY . .
RUN pnpm install --frozen-lockfile
RUN export $(cat .env.example) && pnpm run build

ENV HOST=0.0.0.0
ENV PORT=4321
EXPOSE 4321
CMD node ./dist/server/entry.mjs

docker build -t v0 .
[+] Building 113.8s (11/11) FINISHED                                                                                                                                        docker:orbstack
 => [internal] load build definition from Dockerfile                                                                                                                                   0.0s
 => => transferring dockerfile: 346B                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/node:lts-alpine                                                                                                                     1.1s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 89B                                                                                                                                                       0.0s
 => [1/6] FROM docker.io/library/node:lts-alpine@sha256:1a526b97cace6b4006256570efa1a29cd1fe4b96a5301f8d48e87c5139438a45                                                               0.0s
 => [internal] load build context                                                                                                                                                      0.2s
 => => transferring context: 240.11kB                                                                                                                                                  0.2s
 => CACHED [2/6] RUN corepack enable                                                                                                                                                   0.0s
 => CACHED [3/6] WORKDIR /app                                                                                                                                                          0.0s
 => [4/6] COPY . .                                                                                                                                                                     2.0s
 => [5/6] RUN pnpm install --frozen-lockfile                                                                                                                                          85.7s
 => [6/6] RUN export $(cat .env.example) && pnpm run build                                                                                                      11.1s
 => exporting to image                                                                                                                                                                13.4s
 => => exporting layers                                                                                                                                                               13.4s
 => => writing image sha256:653236defcbb8d99d83dc550f1deb55e48b49d7925a295049806ebac8c104d4a                                                                                           0.0s
 => => naming to docker.io/library/v0

Version 1

The main idea is to first install production dependencies, creating the first layer. Then install all dependencies, package to generate JavaScript artifacts, creating the second layer. Finally, copy the production dependencies and JavaScript artifacts to the runtime environment.

Following the multi-stage build (using SSR) approach, I reduced the image size to 306MB. This is a significant reduction, but the drawback is that it requires explicitly specifying production dependencies; if any are missed, runtime errors will occur.

FROM node:lts-alpine AS base

ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable

WORKDIR /app
COPY package.json pnpm-lock.yaml ./

FROM base AS prod-deps
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prod --frozen-lockfile

FROM base AS build-deps
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile

FROM build-deps AS build
COPY . .
RUN export $(cat .env.example) && pnpm run build

FROM base AS runtime
COPY --from=prod-deps /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist

ENV HOST=0.0.0.0
ENV PORT=4321
EXPOSE 4321
CMD node ./dist/server/entry.mjs

docker build -t v1 .
[+] Building 85.5s (15/15) FINISHED                                                                                                                                         docker:orbstack
 => [internal] load build definition from Dockerfile                                                                                                                                   0.1s
 => => transferring dockerfile: 680B                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/node:lts-alpine                                                                                                                     1.8s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 89B                                                                                                                                                       0.0s
 => [base 1/4] FROM docker.io/library/node:lts-alpine@sha256:1a526b97cace6b4006256570efa1a29cd1fe4b96a5301f8d48e87c5139438a45                                                          0.0s
 => [internal] load build context                                                                                                                                                      0.3s
 => => transferring context: 240.44kB                                                                                                                                                  0.2s
 => CACHED [base 2/4] RUN corepack enable                                                                                                                                              0.0s
 => CACHED [base 3/4] WORKDIR /app                                                                                                                                                     0.0s
 => [base 4/4] COPY package.json pnpm-lock.yaml ./                                                                                                                                     0.2s
 => [prod-deps 1/1] RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prod --frozen-lockfile                                                                           35.1s
 => [build-deps 1/1] RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile                                                                                 65.5s
 => [runtime 1/2] COPY --from=prod-deps /app/node_modules ./node_modules                                                                                                               5.9s
 => [build 1/2] COPY . .                                                                                                                                                               0.8s
 => [build 2/2] RUN export $(cat .env.example) && pnpm run build                                                                                                                       7.5s
 => [runtime 2/2] COPY --from=build /app/dist ./dist                                                                                                                                   0.1s
 => exporting to image                                                                                                                                                                 4.2s
 => => exporting layers                                                                                                                                                                4.1s
 => => writing image sha256:8ae6b2bddf0a7ac5f8ad45e6abb7d36a633e384cf476e45fb9132bdf70ed0c5f                                                                                           0.0s
 => => naming to docker.io/library/v1

Version 2

The main idea is to inline node_modules into the JavaScript files, ultimately copying only the JavaScript files to the runtime environment.

When I looked into Next.js, I remembered that node_modules could be inlined into JavaScript files, eliminating the need for node_modules. So, I researched and found that Vite SSR also supports this. Therefore, I decided to use the inlining method in the Docker environment, avoiding the need to copy node_modules, and only copying the final dist artifacts, reducing the image size to 135MB.

Changes to the packaging script:

vite: {
  ssr: {
    noExternal: process.env.DOCKER ? !!process.env.DOCKER : undefined;
  }
}

The final Dockerfile is as follows:

FROM node:lts-alpine AS base

ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable

WORKDIR /app
COPY package.json pnpm-lock.yaml ./

# FROM base AS prod-deps
# RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prod --frozen-lockfile

FROM base AS build-deps
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile

FROM build-deps AS build
COPY . .
RUN export $(cat .env.example) && export DOCKER=true && pnpm run build

FROM base AS runtime
# COPY --from=prod-deps /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist

ENV HOST=0.0.0.0
ENV PORT=4321
EXPOSE 4321
CMD node ./dist/server/entry.mjs

 docker build -t v2 .
[+] Building 24.9s (13/13) FINISHED                                                                                                                                         docker:orbstack
 => [internal] load build definition from Dockerfile                                                                                                                                   0.0s
 => => transferring dockerfile: 708B                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/node:lts-alpine                                                                                                                     1.7s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 89B                                                                                                                                                       0.0s
 => [base 1/4] FROM docker.io/library/node:lts-alpine@sha256:1a526b97cace6b4006256570efa1a29cd1fe4b96a5301f8d48e87c5139438a45                                                          0.0s
 => [internal] load build context                                                                                                                                                      0.3s
 => => transferring context: 240.47kB                                                                                                                                                  0.2s
 => CACHED [base 2/4] RUN corepack enable                                                                                                                                              0.0s
 => CACHED [base 3/4] WORKDIR /app                                                                                                                                                     0.0s
 => CACHED [base 4/4] COPY package.json pnpm-lock.yaml ./                                                                                                                              0.0s
 => CACHED [build-deps 1/1] RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile                                                                           0.0s
 => [build 1/2] COPY . .                                                                                                                                                               1.5s
 => [build 2/2] RUN export $(cat .env.example) && export DOCKER=true && pnpm run build                                                                                                15.0s
 => [runtime 1/1] COPY --from=build /app/dist ./dist                                                                                                                                   0.1s
 => exporting to image                                                                                                                                                                 0.1s
 => => exporting layers                                                                                                                                                                0.1s
 => => writing image sha256:0ed5c10162d1faf4208f5ea999fbcd133374acc0e682404c8b05220b38fd1eaf                                                                                           0.0s
 => => naming to docker.io/library/v2

In the end, the size was reduced from 1.06GB to 135MB, and the build time was reduced from 113.8s to 24.9s.

docker images
REPOSITORY                         TAG         IMAGE ID       CREATED          SIZE
v2                                 latest      0ed5c10162d1   5 minutes ago    135MB
v1                                 latest      8ae6b2bddf0a   6 minutes ago    306MB
v0                                 latest      653236defcbb   11 minutes ago   1.06GB

The example project is open-source and can be viewed on GitHub.

BroadcastChannel - Turn your Telegram Channel into a MicroBlog

MT — Sun, 11 Aug 2024 09:18:12 +0000

I have been sharing some interesting tools on X and also synchronizing them to my Telegram Channel. I saw that Austin mentioned he is preparing to create a website to compile all the shared content. This reminded me of a template I recently came across called Sepia, and I thought about converting the Telegram Channel into a microblog.

The difficulty wasn't high; I completed the main functionality over a weekend. During the process, I achieved a browser-side implementation with zero JavaScript and would like to share some interesting technical points:

The anti-spoiler mode and the hidden display of the mobile search box were implemented using the CSS ":checked pseudo-class" and the "+ adjacent sibling combinator." Reference
The transition animations utilized CSS View Transitions. Reference
The image lightbox used the HTML popover attribute. Reference
The display and hiding of the "back to top" feature were implemented using CSS animation-timeline, exclusive to Chrome version 115 and above. Reference
The multi-image masonry layout was achieved using grid layout. Reference
The visit statistics were tracked using a 1px transparent image as the logo background, an ancient technique that is now rarely supported by visit statistics software.
JavaScript execution on the browser side was prohibited using the Content-Security-Policy's script-src 'none'. Reference

After completing the project, I open-sourced it, and I was pleasantly surprised by the number of people who liked it; I received over 800 stars in just a week.

If you're interested, you can check it out on GitHub.

https://github.com/ccbikai/BroadcastChannel

How to Replace Google Safe Browsing with Cloudflare Zero Trust

MT — Sun, 07 Jul 2024 14:48:53 +0000

So, get this, right? I built the first version of L(O*62).ONG using server-side redirects, but Google slapped me with a security warning the very next day. Talk about a buzzkill! I had to scramble and switch to local redirects with a warning message before sending folks on their way. Then came the fun part – begging Google for forgiveness.

Now, the smart money would've been on using Google Safe Browsing for redirects. But here's the catch: Safe Browsing's got a daily limit – 10,000 calls, and that's it. Plus, no custom lists. And since I'm all about keeping things simple and sticking with Cloudflare, Safe Browsing was a no-go.

Fast forward to a while back, I was chewing the fat with someone online, and bam! It hit me like a bolt of lightning. Why not use a secure DNS server with built-in filters for adult content and all that shady stuff to check if a domain's on the up-and-up? Figured I'd give Family 1.1.1.1 a shot, and guess what? It actually worked! Problem was, no custom lists there either. Then I remembered messing around with Cloudflare Zero Trust Gateway back in my HomeLab days. Turns out, that was the golden ticket – a solution so good, it's almost criminal.

Here's the deal: Cloudflare Zero Trust's Gateway comes packing a built-in DNS (DoH) server and lets you set up firewall rules like a boss. You can block stuff based on how risky a domain is, what kind of content it has, and even use your own custom naughty-and-nice lists. And get this – it pulls data from Cloudflare's own stash, over 30 open intelligence sources, fancy machine learning models, and even feedback from the community. Talk about covering all the bases! Want the nitty-gritty? Hit up the official documentation.

So, I went ahead and blocked all the high-risk categories – adult stuff, gambling sites, government domains, anything NSFW, newly registered domains, you name it. Plus, I've got my own little blacklists and whitelists that I keep nice and tidy.

Once I was done tweaking the settings, I got myself a shiny new DoH address:

To hook it up to my project, I used this handy-dandy code:

async function isSafeUrl(
  url,
  DoH = "https://family.cloudflare-dns.com/dns-query"
) {
  let safe = false;
  try {
    const { hostname } = new URL(url);
    const res = await fetch(`${DoH}?type=A&name=${hostname}`, {
      headers: {
        accept: "application/dns-json",
      },
      cf: {
        cacheEverything: true,
        cacheTtlByStatus: { "200-299": 86400 },
      },
    });
    const dnsResult = await res.json();
    if (dnsResult && Array.isArray(dnsResult.Answer)) {
      const isBlock = dnsResult.Answer.some(
        answer => answer.data === "0.0.0.0"
      );
      safe = !isBlock;
    }
  } catch (e) {
    console.warn("isSafeUrl fail: ", url, e);
  }
  return safe;
}

And here's the kicker: Cloudflare Zero Trust's management panel has this sweet visualization interface that lets you see what's getting blocked and what's not. You can see for yourself – it's got the kibosh on some adult sites and those brand-spanking-new domains.

Oh, and if a domain ends up on the wrong side of the tracks, you can always check the log to see what went down.

Browser locally uses AI to remove image backgrounds

MT — Sun, 07 Jul 2024 13:28:51 +0000

Yo, so I've been digging into this whole AI thing for front-end development lately, and stumbled upon this cool Transformers.js example. Turned it into a sweet little tool, check it out!

Basically, it uses Transformers.js in a WebWorker to tap into WebGPU and run this RMBG-1.4 model. Long story short, you can now use AI to nuke image backgrounds right in your browser. And get this, it only takes half a second to process a 4K image on my M1 PRO!

Here's the link to the tool: https://html.zone/background-remover

Wanna build it yourself? Head over to https://github.com/xenova/transformers.js/tree/main/examples/remove-background-client for the source code. Oh, and heads up, you gotta be on Transformers.js V3 to mess with WebGPU.