DEV Community: Medcl

Create Your Own AI Assistant, Coco AI v0.1.0 Released

Medcl — Fri, 14 Feb 2025 09:12:03 +0000

Coco AI v0.1.0 Released

We’re excited to announce the official release of Coco AI v0.1.0 – the first preview version of our powerful, open-source, cross-platform unified AI search and productivity tool!

Coco AI allows you to seamlessly search through various data sources such as local applications, cloud storage (Google Drive, Notion, Youyan, Hugo), and enterprise knowledge bases. By leveraging advanced models like DeepSeek, Coco AI transforms personal knowledge management, enabling users to efficiently access information with enhanced privacy and security.

With Coco AI, users can:

Search multiple data sources in real-time, including both local and cloud data.
Integrate AI-powered personal knowledge management for smarter information access.
Chat with an AI assistant based on your personalized knowledge base, ensuring that information is always at your fingertips.

You can now create your own personal AI Assistant with Coco AI now~

Key Features:

Unified Search & Productivity Tool: Connect and search across a wide range of data sources including desktop apps, cloud data, and internal knowledge bases.
AI Assistant Mode: Chat with the AI assistant based on your knowledge base. The assistant understands your documents, personal data, and can offer intelligent summaries and responses.
Privacy & Security: Coco AI allows for private deployment, ensuring your data stays secure, with all personal knowledge hosted locally or on your own server.
Scalable & Open: Easily integrate your own data into Coco AI via APIs and extend functionality with custom data connectors.

Download & Get Started

Coco AI v0.1.0 is now available for MacOS 12 and above. You can start using it today by downloading from the following links:

Download Coco AI: - https://coco.rs/

Coco AI App’s Github: https://github.com/infinilabs/coco-app/

Coco AI Server’s Github: https://github.com/infinilabs/coco-server

Getting Started with INFINI Framework - Our homemade framework for building enterprise golang applications

Medcl — Mon, 16 Dec 2024 10:25:46 +0000

We recently open-sourced our homemade framework for building enterprise golang applications, called INFINI Framework.

The github repo is https://github.com/infinilabs/framework.

In this article, I will show you how to get started with INFINI Framework.

Create New Application

Let's use the NewAPP as the new project for example.

Create the project folder

Use the name new_app as the project id, and create the project folder as below:

cd ~/go/src/infini.sh/
mkdir new_app

Note: Ensure that new_app is located in the same directory as the framework folder. This structure is required for the Makefile to function correctly.

Create the main file

Create a empty main.go file, and paste the code as below:

package main

import (
        "infini.sh/framework"
        "infini.sh/framework/core/module"
        "infini.sh/framework/core/util"
        "infini.sh/framework/modules/api"
        "infini.sh/new_app/config"
)

func main() {

        terminalHeader := ("     __               _      ___  ___ \n")
        terminalHeader += ("  /\\ \\ \\_____      __/_\\    / _ \\/ _ \\\n")
        terminalHeader += (" /  \\/ / _ \\ \\ /\\ / //_\\\\  / /_)/ /_)/\n")
        terminalHeader += ("/ /\\  /  __/\\ V  V /  _  \\/ ___/ ___/ \n")
        terminalHeader += ("\\_\\ \\/ \\___| \\_/\\_/\\_/ \\_/\\/   \\/     \n\n")

        terminalFooter := ("Goodbye~")
        app := framework.NewApp("new_app", "Make a golang application is such easy~.",
                util.TrimSpaces(config.Version), util.TrimSpaces(config.BuildNumber), util.TrimSpaces(config.LastCommitLog), util.TrimSpaces(config.BuildDate), util.TrimSpaces(config.EOLDate), terminalHeader, terminalFooter)
        app.IgnoreMainConfigMissing()
        app.Init(nil)
        defer app.Shutdown()
        if app.Setup(func() {
                module.RegisterSystemModule(&api.APIModule{})
                module.Start()
        }, func() {
        }, nil) {
                app.Run()
        }
}

We use this online tool to generate a beauty ASCII based terminal header.

Create the config file

touch new_app.yml

Create the makefile

create a empty Makefile, and paste the code as below:

SHELL=/bin/bash

# APP info
APP_NAME := new_app
APP_VERSION := 1.0.0_SNAPSHOT
APP_CONFIG := $(APP_NAME).yml
APP_EOLDate ?= "2025-12-31T10:10:10Z"
APP_STATIC_FOLDER := .public
APP_STATIC_PACKAGE := public
APP_UI_FOLDER := ui
APP_PLUGIN_FOLDER := plugins
PREFER_MANAGED_VENDOR=fase

include ../framework/Makefile

Build the application

➜  new_app OFFLINE_BUILD=true make build
building new_app 1.0.0_SNAPSHOT main
/Users/medcl/go/src/infini.sh/new_app
framework path:  /Users/medcl/go/src/infini.sh/framework
fatal: not a git repository (or any of the parent directories): .git
update generated info
update configs
(cd ../framework/  && make update-plugins) || true # build plugins in framework
GOPATH=~/go:~/go/src/infini.sh/framework/../vendor/ CGO_ENABLED=0 GRPC_GO_REQUIRE_HANDSHAKE=off  GO15VENDOREXPERIMENT="1" GO111MODULE=off go build -a  -gcflags=all="-l -B"  -ldflags '-static' -ldflags='-s -w' -gcflags "-m"  --work  -o /Users/medcl/go/src/infini.sh/new_app/bin/new_app
WORK=/var/folders/j5/qd4qt3n55dz053d93q2mswfr0000gn/T/go-build435280758
# infini.sh/new_app
./main.go:17:9: can inline main.deferwrap1
./main.go:21:12: can inline main.func2
./main.go:18:22: func literal does not escape
./main.go:19:45: &api.APIModule{} escapes to heap
./main.go:21:12: func literal escapes to heap
restore generated info

Run the application

➜  new_app git:(main) ✗ ./bin/new_app
     __               _      ___  ___
  /\ \ \_____      __/_\    / _ \/ _ \
 /  \/ / _ \ \ /\ / //_\\  / /_)/ /_)/
/ /\  /  __/\ V  V /  _  \/ ___/ ___/
\_\ \/ \___| \_/\_/\_/ \_/\/   \/

[NEW_APP] Make a golang application is such easy~.
[NEW_APP] 1.0.0_SNAPSHOT#001, 2024-12-16 06:15:10, 2025-12-31 10:10:10, HEAD
[12-16 14:15:19] [INF] [env.go:203] configuration auto reload enabled
[12-16 14:15:19] [INF] [env.go:209] watching config: /Users/medcl/go/src/infini.sh/new_app/config
[12-16 14:15:19] [INF] [app.go:311] initializing new_app, pid: 64426
[12-16 14:15:19] [INF] [app.go:312] using config: /Users/medcl/go/src/infini.sh/new_app/new_app.yml
[12-16 14:15:19] [INF] [api.go:214] local ips: 192.168.3.17
[12-16 14:15:19] [INF] [api.go:312] api server listen at: http://0.0.0.0:2900
[12-16 14:15:19] [INF] [module.go:159] started module: api
[12-16 14:15:19] [INF] [module.go:184] all modules are started
[12-16 14:15:19] [INF] [instance.go:101] workspace: /Users/medcl/go/src/infini.sh/new_app/data/new_app/nodes/ctfs8hbq50kevmkb3m6g
[12-16 14:15:19] [INF] [app.go:537] new_app is up and running now.
^C
[NEW_APP] got signal: interrupt, start shutting down
[12-16 14:15:23] [INF] [module.go:213] all modules are stopped
[12-16 14:15:23] [INF] [app.go:410] new_app now terminated.
[NEW_APP] 1.0.0_SNAPSHOT, uptime: 4.13334s

Goodbye~

Conclusion

The demo code can be found here.

By leveraging the INFINI Framework, creating a Go application becomes significantly simpler and more efficient.
The framework provides built-in commands and modules, streamlining the development process and enabling you to focus on building your application's core functionality.

How We Do Documentation Engineering

Medcl — Mon, 16 Dec 2024 10:22:32 +0000

At INFINI Labs, we see product documentation as an integral part of the product development process. Effective documentation ensures that users understand, adopt, and get the most out of our offerings.

Take a look at our documentation site: https://docs.infinilabs.com/. It hosts detailed documentation for each of our products.

For each product, you can even switch between different versions effortlessly, like this:

Managing comprehensive documentation might seem like an enormous task, especially since we don’t have a dedicated documentation team. However, with limited resources and many competing priorities, we’ve streamlined a practical and efficient approach to product documentation engineering.

So, how do we do it?

The Tools We Use

Our documentation workflow relies on the following tools:

GitHub Pages: For hosting our documentation website.
GitHub Actions: To automate builds and deployments.
Hugo: A fast and flexible static site generator.
Markdown: To write clean and easily maintainable documentation.

We love GitHub's ecosystem for its reliability and developer-friendly features. Using GitHub Pages, we can host our documentation effortlessly. Once our documentation is compiled into static files using Hugo, it’s lightweight, fast, and easy to serve to users.

To enhance the user experience further, we integrate offline search functionality powered by our own pizza-searchbox. This ensures users can quickly find what they need, even without an internet connection.

How It All Works Together

Organizing Components Across Repositories

We maintain separate repositories for each part of the documentation workflow. The final compiled version of all product documentation lives here:

Compiled docs: https://github.com/infinilabs/docs
Hugo theme for docs: https://github.com/infinilabs/docs-theme
Product with docs: https://github.com/infinilabs/gateway

By separating concerns, we make it easier to manage updates, streamline collaboration, and keep everything organized.

As you can see the layout of compiled folder is looks like this:

Each products have folder for each different version, and the main is alwasy point to the latest version.

And how did these static docs coming from?

Checkout this specify product's repo for example:

As you can see, in the product’s repository, there’s a folder named docs, which contains all the documentation specific to that product.

Alongside it, there’s a config.yaml defines the basic configuration for the product:

# VERSIONS=latest,v1.0 hugo    --minify --baseURL="/product/v1.0/"  -d public/product/v1.0

title: INFINI Gateway
theme: book

# Book configuration
disablePathToLower: true
enableGitInfo: false

# Needed for mermaid/katex shortcodes
markup:
  goldmark:
    renderer:
      unsafe: true
  tableOfContents:
    startLevel: 1

# Multi-lingual mode config
# There are different options to translate files
# See https://gohugo.io/content-management/multilingual/#translation-by-filename
# And https://gohugo.io/content-management/multilingual/#translation-by-content-directory
defaultContentLanguage: en
languages:
  en:
    languageName: English
    contentDir: content.en
    weight: 3


menu:
  before: []
  after:
    - name: "Github"
      url: "https://github.com/infinilabs/gateway"
      weight: 10

...
EMITTED
...

Make sure you changed the right github's repo address and the product name.

And also there’s a Makefile that defines how we build the docs:

SHELL=/bin/bash

# Basic info
PRODUCT?= $(shell basename "$(shell cd .. && pwd)")
BRANCH?= main
VERSION?= $(shell [[ "$(BRANCH)" == "main" ]] && echo "main" || echo "$(BRANCH)")
CURRENT_VERSION?= $(VERSION)
VERSIONS?= "main"
OUTPUT?= "/tmp/docs"
THEME_FOLDER?= "themes/book"
THEME_REPO?= "https://github.com/infinilabs/docs-theme.git"
THEME_BRANCH?= "main"

.PHONY: docs-build

default: docs-build

docs-init:
    @if [ ! -d $(THEME_FOLDER) ]; then echo "theme does not exist";(git clone -b $(THEME_BRANCH) $(THEME_REPO) $(THEME_FOLDER) ) fi

docs-env:
    @echo "Debugging Variables:"
    @echo "PRODUCT: $(PRODUCT)"
    @echo "BRANCH: $(BRANCH)"
    @echo "VERSION: $(VERSION)"
    @echo "CURRENT_VERSION: $(CURRENT_VERSION)"
    @echo "VERSIONS: $(VERSIONS)"
    @echo "OUTPUT: $(OUTPUT)"

docs-config: docs-init
    cp config.yaml config.bak
    # Detect OS and apply the appropriate sed command
    @if [ "$$(uname)" = "Darwin" ]; then \
        echo "Running on macOS"; \
        sed -i '' "s/BRANCH/$(VERSION)/g" config.yaml; \
    else \
        echo "Running on Linux"; \
        sed -i 's/BRANCH/$(VERSION)/g' config.yaml; \
    fi

docs-build: docs-config
    hugo --minify --theme book --destination="$(OUTPUT)/$(PRODUCT)/$(VERSION)" \
        --baseURL="/$(PRODUCT)/$(VERSION)"
    @$(MAKE) docs-restore-generated-file

docs-place-redirect:
    echo "<!DOCTYPE html> <html> <head> <meta http-equiv=refresh content=0;url=main /> </head> <body> <p><a href=main />REDIRECT TO THE LATEST_VERSION</a>.</p> </body> </html>" > $(OUTPUT)/$(PRODUCT)/index.html

docs-restore-generated-file:
    mv config.bak config.yaml

Usually, there’s no need to make any changes—simply copy these files to your new product, and everything will work seamlessly.

Automation Makes Everything Easier

And we use .github/workflows/build-docs.yml to define how to build the documentation once someone pushed the code, or someone released a new version, take a look at the github actions:

name: Build and Deploy Docs

on:
  push:
    branches:
      - main
      - 'v*'
    tags:
      - 'v*'

jobs:
  build-deploy-docs:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Product Repo
        uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: Set Variables Based on Ref
        id: vars
        run: |
          PRODUCT_NAME=$(basename $(pwd))  # Get the directory name as the product name
          echo "PRODUCT_NAME=$PRODUCT_NAME" >> $GITHUB_ENV
          CURRENT_REF=${GITHUB_REF##*/}
          IS_SEMVER=false
          SEMVER_REGEX="^v([0-9]+)\.([0-9]+)\.([0-9]+)$"

          if [[ "${GITHUB_REF_TYPE}" == "branch" ]]; then
            if [[ "$CURRENT_REF" == "main" ]]; then
              echo "VERSION=main" >> $GITHUB_ENV
              echo "BRANCH=main" >> $GITHUB_ENV
            elif [[ "$CURRENT_REF" =~ $SEMVER_REGEX ]]; then
              IS_SEMVER=true
              echo "VERSION=$CURRENT_REF" >> $GITHUB_ENV
              echo "BRANCH=$CURRENT_REF" >> $GITHUB_ENV
            else
              echo "Branch '$CURRENT_REF' is not a valid semantic version. Skipping build."
              exit 0
            fi
          elif [[ "${GITHUB_REF_TYPE}" == "tag" ]]; then
            if [[ "$CURRENT_REF" =~ $SEMVER_REGEX ]]; then
              IS_SEMVER=true
              echo "VERSION=$CURRENT_REF" >> $GITHUB_ENV
              echo "BRANCH=main" >> $GITHUB_ENV  # Set BRANCH to 'main' for tags
            else
              echo "Tag '$CURRENT_REF' is not a valid semantic version. Skipping build."
              exit 0
            fi
          fi

          # Gather branches and tags, filter for semantic versions, sort, remove duplicates
          VERSIONS=$(git for-each-ref refs/remotes/origin refs/tags --format="%(refname:short)" | \
            grep -E "^v[0-9]+\.[0-9]+\.[0-9]+$" | sort -Vr | uniq | tr '\n' ',' | sed 's/,$//')
          echo "VERSIONS=main,$VERSIONS" >> $GITHUB_ENV

      - name: Install Hugo
        run: |
          wget https://github.com/gohugoio/hugo/releases/download/v0.79.1/hugo_extended_0.79.1_Linux-64bit.tar.gz
          tar -xzvf hugo_extended_0.79.1_Linux-64bit.tar.gz
          sudo mv hugo /usr/local/bin/

      - name: Checkout Docs Repo
        uses: actions/checkout@v2
        with:
          repository: infinilabs/docs
          path: docs-output
          token: ${{ secrets.DOCS_DEPLOYMENT_TOKEN }}

      - name: Build Documentation
        run: |
          (cd docs && OUTPUT=$(pwd)/../docs-output make docs-build docs-place-redirect)

      - name: Commit and Push Changes to Docs Repo
        working-directory: docs-output
        run: |
          git config user.name "GitHub Actions"
          git config user.email "actions@github.com"

          if [[ -n $(git status --porcelain) ]]; then
            git add .
            git commit -m "Rebuild $PRODUCT_NAME docs for version $VERSION"
            git push origin main
          else
            echo "No changes to commit."
          fi

      - name: Rebuild Docs for Latest Version (main), if not already on main
        run: |
          # Only rebuild the main branch docs if the current ref is not "main"
          if [[ "$CURRENT_REF" != "main" ]]; then
            echo "Switching to main branch and rebuilding docs for 'latest'"

            # Checkout the main branch of the product repo to rebuild docs for "latest"
            git checkout main

            # Ensure the latest changes are pulled
            git pull origin main

            # Build Docs for Main Branch (latest)
            (cd docs && OUTPUT=$(pwd)/../docs-output VERSION="main" BRANCH="main" make docs-build docs-place-redirect)

            # Commit and Push Latest Docs to Main
            cd docs-output
            git config user.name "GitHub Actions"
            git config user.email "actions@github.com"

            if [[ -n $(git status --porcelain) ]]; then
              git add .
              git commit -m "Rebuild $PRODUCT_NAME docs for main branch with latest version"
              git push origin main
            else
              echo "No changes to commit for main."
            fi
          else
            echo "Current ref is 'main', skipping rebuild for 'latest'."
          fi
        working-directory: ./  # Working in the product repo

If you look closely at the GitHub Actions workflow, it simplifies the entire process by automating key tasks:

Monitor Branches and Tags: Watches for branches or tags starting with v and validates them as semantic versioned branches or tags. Only valid versions trigger the documentation build process.
Fetch Theme: Pulls the documentation theme from a separate repository to ensure a consistent look and feel across all products.
Build and Deploy: Compiles the documentation into static files using Hugo, commits the changes with a clear and informative message, and pushes the updates to the docs repository.

Through GitHub Actions, the entire workflow becomes seamless:

Compile: Converts Markdown files into a polished static site using Hugo.
Deploy: Publishes the site effortlessly to GitHub Pages.

This automation reduces manual effort, ensures consistency across documentation, and allows us to focus on delivering quality content to users.

Why This Approach Works for Us

Efficiency: With automation, we can focus on content rather than operations.
Scalability: As our product offerings grow, this workflow scales effortlessly.
User Experience: A fast, searchable, and offline-ready documentation site means better support for our users.

Final Thoughts

Product documentation isn’t just a necessity—it’s a competitive advantage. By leveraging the right tools and automating where possible, we’ve built a process that delivers high-quality documentation without needing a dedicated team.

Want to see it in action? Visit our documentation site here: https://docs.infinilabs.com/.

Let us know what you think! Your feedback helps us improve not only our products but also the way we document and share them with the world.

Introducing Coco AI in two minutes - an open-source alternative to Glean

Medcl — Tue, 26 Nov 2024 16:21:21 +0000

Introducing Coco AI in two minutes. Coco AI unifies all your enterprise applications and data—Google Workspace, Dropbox, GitHub, and more—into one powerful search and Gen-AI chat platform. Instantly access and interact with your team’s unique knowledges.

http://coco.rs
https://github.com/infinilabs/coco-app/

// Detect dark theme var iframe = document.getElementById('tweet-1861590090985939294-771'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1861590090985939294&theme=dark" }

https://x.com/medcl/status/1861590090985939294

Benchmarking and Profiling Rust Applications on macOS: A Practical Guide

Medcl — Mon, 23 Sep 2024 10:47:26 +0000

Profiling Rust code has become part of my daily routine. As I primarily develop on macOS, I've noticed there aren't many tools that allow for easy and quick profiling of Rust applications. So, I’d like to share my daily profiling workflow, in case it helps others. If you have other approaches or tools that work well for you, feel free to share—I’d love to hear them!

Setting Up Micro-Benchmarks

I use micro-benchmark tests to track the performance of critical functions in my Rust application. For this, I rely on criterion, which is both powerful and easy to use. Here’s what my project setup looks like:

As you noticed, i organized my benchmark tests per module, and so i can easily include or exclude specify tests in the benches.rs, some times, they just take too much time, if i only want to profile specify tests, i can just comment out unrelated one. dirty but works.

If you have many similar tests, you may group them by use a customized name, like this:

Micro-benchmarking is a fundamental step that helps track performance changes when refactoring code or adding new features.

Profiling The Micro-Benchmark

What if some tests are slow, a quick way to profiling is to use:

sudo CARGO_PROFILE_BENCH_DEBUG=true cargo flamegraph --bench benches   -o find-baseline.svg -- --bench

You can run the benchmarks with a single command, but be sure to comment out any unrelated tests in benches.rs. Just remember to revert those changes or avoid checking them in later. that's tedious, yes, i know :(

Tracking Performance with Bencher

Now that you have several micro-benchmark tests, how can you continuously monitor performance? I’m glad I discovered a free service provided by bencher.dev. It helps track performance over time, making it easier to identify any regressions or improvements.

Here is my bencher command in my Makefile:

bencher-engine:
    if [ -z "${BENCHER_API_TOKEN:-}" ]; then \
        echo "Error: BENCHER_API_TOKEN environment variable is not set."; \
        exit 1; \
    fi

    bencher run \
        --project pizza-engine-bd8p44nc \
        --branch main \
        --testbed localhost \
        --adapter rust_criterion \
        "cd lib/engine && cargo bench"

Each time you run make bencher-engine, it executes all your benchmarks and sends the results to their database. This service is free to use for open-source projects, making it a great resource for ongoing performance tracking.

For example here is my dashboard:

You can add a CI action to your GitHub repository to automatically track performance changes with each commit. If your code is hosted on GitHub, this setup will record performance variations for every commit, helping you maintain a history of your application's performance over time.

Profiling on MacOs

Profiling on macOS can be slightly less convenient than on Linux, where there are many robust tools available. Here’s what I do to make the most of the profiling process, I use Dtrace along with two scripts:

➜ cat ~/start_profile.sh 
#!/bin/bash

# Check if PID argument is provided
if [ -z "$1" ]; then
  echo "Usage: $0 <pid>"
  exit 1
fi

pid=$1

# Run dtrace with the provided PID
sudo rm -rf target/out.user_stacks || true

sudo dtrace -x ustackframes=100 -n "profile-97 /pid == $pid/ { @[ustack()] = count(); } tick-60s { exit(0); }" -o target/out.user_stacks

➜ cat ~/end_profile.sh 
#!/bin/bash

# Clean up previous output files
rm -f target/stacks.folded target/flamegraph.svg

# Process the new profiling data
cat target/out.user_stacks | inferno-collapse-dtrace > target/stacks.folded
cat target/stacks.folded | inferno-flamegraph > target/flamegraph.svg

# Notify the user
echo "Flamegraph generated at target/flamegraph.svg"

And then start your target Rust application, make sure you set debug=true in Cargo.toml

And then execute the start script, wait for a while and Ctrl+C to capture profiling data:

➜  indexer git:(batch_indexing) ✗ ~/start_profile.sh 1494                                                      
dtrace: system integrity protection is on, some features will not be available

dtrace: description 'profile-97 ' matched 2 probes
^C%

And then you can generate the flamegraph:

➜  indexer git:(batch_indexing) ✗ ~/end_profile.sh 
Flamegraph generated at target/flamegraph.svg

Open it with your web browser and figure out what's the bottlenect, and rock with it.

That’s it! I hope this information helps you in your Rust development journey. If you have any questions or need further assistance, feel free to reach out!

References:

Adding Search Functionality to a Hugo Static Site Based on INFINI Pizza for WebAssembly

Medcl — Wed, 28 Aug 2024 13:21:42 +0000

INFINI Pizza is an upcoming search engine developed by INFINI Labs, written in Rust (soon to be fully open-sourced). The basic search capabilities have already been completed, and based on the core engine of INFINI Pizza, a WASM version of the ultra-lightweight kernel is provided. This can be easily embedded into various application systems, such as websites, especially static sites or small blog systems.

Currently, The website of INFINI Pizza have integrated INFINI Pizza for WebAssembly. The specific search results are shown in the image below:

Visit the website (http://pizza.rs), and by pressing the shortcut key 's', you can bring up the search box and experience the search capabilities provided by INFINI Pizza. Notably, during the search process, all of your actions are executed locally in the browser. Unlike traditional search implementations, where each query requires an interaction with a backend search server, INFINI Pizza for WebAssembly operates entirely offline. Even if you're disconnected from the internet, you can still enjoy a seamless search experience.

Without further ado, let's dive into how you can use INFINI Pizza for WebAssembly on your own site.

First, INFINI Pizza for WebAssembly is open source. You can find the GitHub repository here: https://github.com/infinilabs/pizza-wasm. The compiled WASM package is available for direct download here: https://github.com/infinilabs/pizza-wasm/tree/main/pkg.

➜  wasm git:(main) ✗ du -sh pkg/*
4.0K    pkg/README.md
4.0K    pkg/package.json
4.0K    pkg/pizza_wasm.d.ts
4.0K    pkg/pizza_wasm.js
 12K    pkg/pizza_wasm_bg.js
580K    pkg/pizza_wasm_bg.wasm
4.0K    pkg/pizza_wasm_bg.wasm.d.ts
256K    pkg/pizza_wasm_bg.wasm.gz

You'll notice that the WASM package is just over 500 KB, and after Gzip compression, it’s reduced to just over 200 KB, making it quite lightweight. You may have thinking there are some JavaScript based static search solution maybe more smaller, but please note they are not full featured search engine, and I will explain this when INFINI Pizza officially reveal sometime later.

Pizza-WASM is a WebAssembly interface wrapper for the core engine of INFINI Pizza, exposing only a few simple access interfaces. These are sufficient for current frontend search applications. You can find a very basic example of how to call the WASM methods in the following directory: https://github.com/infinilabs/pizza-wasm/tree/main/web.

However, just having the Pizza WASM isn't enough. If we want to add a search box to an existing static site, we also need to consider where the data comes from and how the results are displayed. To address this, we've wrapped up a project called Pizza-DocSearch, which further simplifies usage. This project is also open source, and you can find it on GitHub here: https://github.com/infinilabs/pizza-docsearch.

Since the example project has already uploaded the compiled code and samples, we can directly download the source code and preview the functionality locally.

➜  /tmp git clone https://github.com/infinilabs/pizza-docsearch.git
Cloning into 'pizza-docsearch'...
remote: Enumerating objects: 174, done.
remote: Counting objects: 100% (174/174), done.
remote: Compressing objects: 100% (112/112), done.
remote: Total 174 (delta 86), reused 147 (delta 59), pack-reused 0 (from 0)
Receiving objects: 100% (174/174), 941.94 KiB | 1.20 MiB/s, done.
Resolving deltas: 100% (86/86), done.
➜  /tmp cd pizza-docsearch/example/dist 
➜  dist git:(main) python3 -m http.server 8083    

Serving HTTP on :: port 8083 (http://[::]:8083/) ...

Open your browser and visit: http://localhost:8083, as shown below:

Observe the network requests in the browser, and you'll see that it loads the sample index.json data:

In practice, if it's our own static website or blog, simply ensuring that this file (index.json) with the appropriate format exists in the root directory of the site is enough to quickly integrate the search functionality you see into your own website. OK, with the functionality verified, let's start integrating it into our site.

The official website of Pizza/INFINI Labs is statically generated using Hugo, and the index.json file does not need to be manually created. First, we need to enable Hugo to generate content in JSON format, which is a built-in capability of Hugo. We'll need to modify the configuration of the Hugo project:

Add a JSON output option to the outputs parameter, and then define the JSON output format template in the theme's template files:

The text content is as follows for easy copying and pasting. Save the file with the name index.json:

{{- $index := slice -}}
{{- range where .Site.RegularPages.ByDate.Reverse "Type" "not in" (slice "page" "json") -}}
    {{- $index = $index | append (dict "title" (.Title | plainify) "url" .Permalink "tags" .Params.tags "category" .Params.category "subcategory" .Params.subcategory "summary" (.Params.Summary | markdownify | plainify) "content" (.Content | markdownify | plainify)) -}}
{{- end -}}
{{- $index | jsonify -}}

OK, next, we need to add the tags we’ve used above to the metadata of each article or blog post on the site:

OK, start the Hugo site:

hugo server --forceSyncStatic   --minify --theme book


                   | EN   
-------------------+------
  Pages            | 181  
  Paginator pages  |   5  
  Non-page files   |   0  
  Static files     | 110  
  Processed images |   0  
  Aliases          |  52  
  Sitemaps         |   1  
  Cleaned          |   0  

Built in 323 ms
Watching for changes in /Users/medcl/Documents/rust/pizza/website/{assets,content.en,static,themes}
Watching for config changes in /Users/medcl/Documents/rust/pizza/website/config.yaml
Environment: "development"
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop

Open the Hugo site address and try accessing http://localhost:1313/index.json. You should be able to access the JSON file:

At this point, the data preparation is complete. Next, we'll integrate the frontend search widget.

Do you remember the resource files we downloaded from Pizza-DocSearch earlier? We mainly use the three files in the assets folder:

/tmp/pizza-docsearch/example/dist
➜  dist git:(main) tree    
.
├── assets
  ├── index-C1z1vz3D.css
  ├── index-D_gOo737.js
  └── pizza_wasm_bg-BRCuviY_.wasm
├── index.html
└── index.json

1 directory, 5 files
➜  dist git:(main)

Open the index.html file, and you will see the following content:

In order to integrate with the docsearch, we just need to add the three highlighted sections of code into our own static website.

Copy the files from the assets directory to our Hugo site, in the following location:

Then, modify the Hugo theme templates by adding a segment of code to the html-head.html file in the header template of all pages to load our CSS stylesheet:

Next, continue modifying the Hugo theme template files by adding a segment of code to the footer template of all pages to load the JS script files:

Next, insert a Docsearch tag in the appropriate place within the page template to place the search box, as shown in the image:

With that, the task is complete!

Open your browser to see the final result:

Finally, to summarize, with the help of the 3 lines of code and copy 3 files from INFINI Pizza Docsearch, you can add a lightweight offline search functionality to your static site in just 5 minutes. Give it a try!

References:

How to leverage the state-of-the-art NLP models in Rust

Medcl — Tue, 01 Aug 2023 02:59:25 +0000

In this tutorial we will show you how to use rust-bert library to utilize the state-of-the-art natural language processing models in Rust, and we are specifically tested on macOS environments.

Rust crate rust_bert implementation of the BERT language model (https://arxiv.org/abs/1810.04805 Devlin, Chang, Lee, Toutanova, 2018). The base model is implemented in the bert_model::BertModel struct. Several language model heads have also been implemented, including:

Masked language model: bert_model::BertForMaskedLM
Multiple choices: bert_model:BertForMultipleChoice
Question answering: bert_model::BertForQuestionAnswering
Sequence classification: bert_model::BertForSequenceClassification
Token classification (e.g. NER, POS tagging): bert_model::BertForTokenClassification

Transformers

Transformers is a State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Before installing the transformers:

python3 -m venv .env
source .env/bin/activate

brew install cmake
brew install pkg-config
brew install sentencepiece

pip install sentencepiece
pip install transformers
pip install 'transformers[torch]'
pip install 'transformers[tf-cpu]'
pip install 'transformers[flax]'
pip install onnxruntime

Verify the installation:

(.env) ➜  transformers git:(main) python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))"
No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english and revision af0f99b (https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english).
Using a pipeline without specifying a model name and revision in production is not recommended.
Downloading (…)lve/main/config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 629/629 [00:00<00:00, 1.13MB/s]
Downloading model.safetensors: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 268M/268M [00:26<00:00, 10.0MB/s]
Downloading (…)okenizer_config.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 48.0/48.0 [00:00<00:00, 158kB/s]
Downloading (…)solve/main/vocab.txt: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 554kB/s]
Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers
pip install xformers.
[{'label': 'POSITIVE', 'score': 0.9998704195022583}]

Install the huggineface's transformers:

git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e .

🤗 Transformers is tested on Python 3.6+, PyTorch 1.1.0+, TensorFlow 2.0+, and Flax. Follow the installation instructions below for the deep learning library you are using:

PyTorch installation instructions.
TensorFlow 2.0 installation instructions.
Flax installation instructions.

So we will need to install these 3 dependend projects as follow.

Install PyTorch

pip3 install torch torchvision torchaudio

Install TensorFlow

# There is currently no official GPU support for MacOS.
python3 -m pip install tensorflow
# Verify install:
python3 -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

Install Flax

Flax delivers an end-to-end and flexible user experience for researchers who use JAX with neural networks. Flax exposes the full power of JAX. It is made up of loosely coupled libraries.

JAX is a project designed for High-Performance Array Computing,

python3.11 -m pip install --upgrade pip
pip install flax

Init Rust BERT

brew install libtorch
brew link libtorch
brew ls --verbose libtorch | grep dylib
export LIBTORCH=$(brew --cellar pytorch)/$(brew info --json pytorch | jq -r '.[0].installed[0].version')
export LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH

git clone https://github.com/guillaume-be/rust-bert.git
cd rust-bert
ORT_STRATEGY=system cargo run --example sentence_embeddings

Also better to add the following to bash/zsh environment in case you met exception like "libtch/torch_api_generated.cpp" with args "c++" did not execute successfully (status code exit status: 1).:

export LIBTORCH=$(brew --cellar pytorch)/$(brew info --json pytorch | jq -r '.[0].installed[0].version')
export LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH

In Rust's project

Import the example code into Pizza's module:

use log::*;
use rust_bert::pipelines::translation::{Language, TranslationModelBuilder};
pub(crate) fn translation() -> anyhow::Result<()> {
    info!("start translation:");
    let model = TranslationModelBuilder::new()
        .with_source_languages(vec![Language::English])
        .with_target_languages(vec![
            Language::Spanish,
            Language::French,
            Language::Italian,
        ])
        .create_model()
        .unwrap();

    let input_text = "Hello world!";
    let output = model
        .translate(&[input_text], None, Language::Spanish)
        .unwrap();

    for sentence in output {
        info!("Output: {}", sentence);
    }
    Ok(())
}

And we will get the result as follows:

   ___ _____  __________   _
  / _ \\_   \/ _  / _  /  /_\
 / /_)/ / /\/\// /\// /  //_\\
/ ___/\/ /_   / //\/ //\/  _  \
\/   \____/  /____/____/\_/ \_/

[PIZZA] The Next-Gen Real-Time Hybrid Search & AI-Native Innovation Engine.
[2023-06-03 19:00:32] [INFO] [pizza:96] PIZZA now starting.
[2023-06-03 19:00:32] [INFO] [pizza::modules::api:71] api listen at: http://0.0.0.0:2900
[2023-06-03 19:00:32] [INFO] [pizza::modules:37] started module [api]
[2023-06-03 19:00:32] [INFO] [pizza::modules::bert::translation:4] start translation:
[2023-06-03 19:00:32] [INFO] [actix_server::builder:200] starting 8 workers
[2023-06-03 19:00:32] [INFO] [actix_server::server:197] Tokio runtime found; starting in existing Tokio runtime
[2023-06-03 19:00:33] [INFO] [cached_path::cache:414] Cached version of https://huggingface.co/Helsinki-NLP/opus-mt-en-ROMANCE/resolve/main/vocab.json is up-to-date
[2023-06-03 19:00:33] [INFO] [cached_path::cache:414] Cached version of https://huggingface.co/Helsinki-NLP/opus-mt-en-ROMANCE/resolve/main/source.spm is up-to-date
[2023-06-03 19:00:34] [INFO] [cached_path::cache:414] Cached version of https://huggingface.co/Helsinki-NLP/opus-mt-en-ROMANCE/resolve/main/config.json is up-to-date
[2023-06-03 19:00:36] [INFO] [cached_path::cache:414] Cached version of https://huggingface.co/Helsinki-NLP/opus-mt-en-ROMANCE/resolve/main/rust_model.ot is up-to-date
[2023-06-03 19:00:36] [INFO] [pizza::modules::bert::translation:21] Output:  ¡Hola mundo!
[2023-06-03 19:00:36] [INFO] [pizza::modules:37] started module [bert]
[2023-06-03 19:00:36] [INFO] [pizza::modules:39] all modules are started
[2023-06-03 19:00:36] [INFO] [pizza:116] PIZZA is up and running now. PID: 37426

Wow, Hello World was successfully translated to ¡Hola mundo!, which is great!

As you can see, creating a translation application is incredibly simple just in a few lines of code. And this is just the beginning! By harnessing the power of these pre-trained language models, we can accomplish so much more.