DEV Community: Samy Fodil

Building a Serverless Web Game With WebAssembly and Three.js

Samy Fodil — Wed, 03 Sep 2025 13:51:12 +0000

In this tutorial, we'll build an online tower stacking game.

👉 Try the final result: Play Now

🛠️ Tech Stack

Frontend: Three.js + TypeScript
Backend: Go serverless functions on Taubyte
Local Dev: Dream (Taubyte's local cloud) + GitHub integration

⚡ Step 1: Run the Frontend Locally

Clone and run the game — at this stage, it has no leaderboard:

git clone https://github.com/taubyte/example-games-tower-blocks.git
cd example-games-tower-blocks

npm install
npm run dev

Open http://localhost:3000 — the game works, but scores aren't saved anywhere yet.

🔧 Step 2: Start Taubyte Local Cloud (Dream)

Install Dream and start a local cloud instance:

npm install -g @taubyte/dream
dream new multiverse

Keep this terminal open — it runs your local Taubyte cloud.

🌐 Step 3: Connect to Console

Go to console.taubyte.com
Log in with your email and GitHub

From the dropdown, select Dream/blackhole (your local cloud).

🖥️ Step 4: Create Applications

Create and open a new project , this will create a code and a config repositories to save your project's data .

You'll need two apps:

frontend → hosts the website
backend → handles leaderboard logic

Frontend Setup

Inside frontend app, add a Website resource

Name: tower_game
Repo: pick your GitHub fork (console fetches repos automatically)
Domain: use the default generated domain
Path: /

Push configuration to GitHub (green Push button).

In another terminal, trigger builds:

dream inject push-all

trigger a build for the website specifically:

dream inject push-specific --rid <ID> --fn <FULL_NAME>

Find rid and full_name in the YAML tab of the resource.

Add your Dream domain to /etc/hosts:

127.0.0.1 your-domain.blackhole.localtau

In the console, click the ⚡ button on the website resource → frontend should now load locally.

🗄️ Step 5: Implement Backend

Create Database

Inside backend app, add a Database resource:

Name: leaderboard
Matcher: /leaderboard

Create Library

Add a Library resource:

Name: api_leaderboard
Language: Go (empty template)

Use this reference implementation:
The Library's Code

Create Functions

In the Functions tab, create:

getAll
- Method: GET
- Path: /api/leaderboard
- Entry: getAll

get
- Method: GET
- Path: /api/score
- Query param: player_name
- Entry: get

set
- Method: POST
- Path: /api/score
- Entry: set

🚀 Step 6: Deploy and Test

Push builds:

dream inject push-all

(Use push-specific for just the library if needed.)

Wait for builds to complete in the Builds tab.

Test each function by clicking the ⚡ button in the console → you'll see live responses.

🌍 Step 7: Push to Production

All that is left now is to push to production.

It's a really simple process:

Log out of the local cloud's console.
Make sure you already have a Taubyte cloud online set up.
Log in using your email and domain name.
Instead of creating a project like we did the first time, import the local project from GitHub. Select it from the dropdown and click Import.

Open the imported project. Change the applications' local generated domain to a production one by toggling Generate Domain. Do this for both frontend and backend apps.

Push your changes → this will trigger a config build.

If your project has a Library or Website, you must register the resource with Auth before builds work. This can be done from the web console using the broken heart icon 💔 next to the resource.

Opening your game now in the production domain should work properly with all features up and running 🎉

✅ Summary

You now have:

A frontend game hosted via Taubyte
A backend with a database + Go library
Three serverless functions powering a leaderboard

All running locally with Dream, and ready to deploy globally once you push to production.

📚 Resources

Game Repository: Github
Backend Library: Github
Dream NPM Package: NPM
Taubyte Documentation: tau.how

Your Software, Your Cloud: Building Your Own Vercel Ain't That Hard!

Samy Fodil — Wed, 23 Jul 2025 20:49:34 +0000

Why pay Vercel when you can build better? This is a guide to creating your own distributed cloud that rivals the big players—for a fraction of the cost.

At Taubyte, we’ve spent the last few of years building an open-source cloud platform. This guide is here to help you deploy it manually, step-by-step. It's a solid walkthrough to get a working, self-hosted Tau setup running across multiple continents.

You'll build a globally distributed developer cloud with DNS-based load balancing, service discovery, and container orchestration—all with transparency and control, and with no vendor lock-in.

And by the way, if you'd rather watch a video:

Let’s dive in.

What You're Building (And Why It's Awesome)

We're not just spinning up a few servers here. You're building a genuinely distributed cloud platform. This platform will feature:

Global Edge Presence: Deploy across multiple continents for maximum performance and redundancy.
Zero-Downtime Deployments: Built-in redundancy means your applications stay online.
DNS-Based Load Balancing: No expensive, dedicated load balancers needed.
Automatic Service Discovery: Your services find each other like magic.

Think of it as your personal Vercel, but one that you actually understand and control completely.

Prerequisites

A Fully Qualified Domain Name (FQDN): This guide uses k8s.cyou as an example.
Advanced DNS Management: Your DNS provider must support custom A and NS records.

Infrastructure Plan

We will deploy across three continents for performance and redundancy:

host-001-spain
host-002-uae
host-003-algeria

Pro tip: Choose your hosting providers strategically. I recommend different providers in each region to avoid correlated failures.

Host Preparation (Repeat on all nodes)

1. Firewall Configuration

First, ensure SSH access is not blocked.

sudo ufw allow ssh

Then, open the necessary ports for Taubyte services.

Peer-to-Peer Communication:
```
sudo ufw allow 4242,4247,4252/tcp
```
Web & DNS Traffic:
```
sudo ufw allow 80,443/tcp
sudo ufw allow 53,953/tcp
sudo ufw allow 53,953/udp
```
Port 53 handles public DNS queries, while 953 is used for secure DNS control plane operations.
Enable Firewall:
```
sudo ufw enable
sudo ufw status
```

2. Install Tools & Free DNS Port

sudo apt update
sudo apt install -y curl nano

Configure systemd-resolved to use a public DNS resolver and free up port 53 for Taubyte.

sudo nano /etc/systemd/resolved.conf

Set the following values:

DNS=1.1.1.1
DNSStubListener=no

Why these specific changes matter:

Cloudflare DNS (1.1.1.1): A fast, privacy-focused public DNS resolver with excellent uptime.
DNSStubListener=no: Prevents conflicts with Seer service, which needs direct control over DNS resolution.

Apply the changes:

sudo systemctl restart systemd-resolved.service
sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf

3. Install Docker

Docker is used as a containerization engine by tau.

curl -fsSL https://get.docker.com | sh
docker --version

4. Install Tau

Tau is the core of Taubyte's Developer Cloud Platform Solution.

curl https://get.tau.link/tau | sh

Primary Node Configuration (Run on node 1 only)

1. Generate Master Configuration

Get your node's public IP address.

curl ifconfig.me -4

Generate the configuration that defines your cloud. This node will enable all core services and initiate Docker's clustering mode.

sudo tau config generate \
  -n k8s.cyou \
  -s compute \
  --services all \
  --ip YOUR_PUBLIC_IP \
  --dv \
  --swarm

2. Understanding Taubyte's Service Architecture

Your cloud runs on these core services, each with a specific purpose:

Service	Role	Why You Need It
Auth	Authentication & Authorization	Secures access to your cloud APIs and resources
Patrick	CI/CD Orchestration	Manages code builds and deployments automatically
Monkey	Job Execution Engine	Runs the actual work (builds, functions, etc.)
TNS	Taubyte Naming Service	Service registry and discovery system
Hoarder	Object Storage	Handles file storage, backups, and static assets
Seer	DNS Load Balancing	Intelligent traffic distribution via DNS
Substrate	Event Processing	The compute engine for serverless functions

3. Fine-Tuning Your Configuration (Optional)

You can customize your setup by editing the generated configuration:

sudo nano /tb/config/compute.yaml

4. Validate and Launch

Always validate before proceeding:

sudo tau conf validate -s compute

A successful validation will output a unique cryptographic ID for your node. Now, launch the services manually to test.

sudo tau start -s compute

As the logs scroll by, you are witnessing a distributed system come alive as services start and discover each other.

5. Test Your Cloud Node

Before moving on, verify the node is working. Use a DNS lookup tool to query your node directly.

# dig <domain> @<nameserver>
dig seer.tau.k8s.cyou @YOUR_NODE_IP

If you get your node's IP back in the ANSWER SECTION, your first node is live and responding correctly.

Making it Production-Ready with Systemd

Running manually is great for testing, but production demands reliability. systemd provides:

Automatic restarts: Services restart instantly after crashes.
Boot-time startup: Your cloud survives server reboots.
Centralized logging: Integrated logs via journald for easier debugging.

First, stop the manual process with Ctrl+C. Then, create the service file:

sudo nano /lib/systemd/system/tau@.service

Paste the following configuration:

[Unit]
Description=Tau %I
After=network.target

[Service]
Type=simple
User=root
Group=root
ExecStart=/usr/local/bin/tau start -s %i
Restart=always
RestartSec=3
LimitNOFILE=16384

[Install]
WantedBy=multi-user.target

Enable and start the service on your primary node.

sudo systemctl enable tau@compute.service
sudo systemctl start tau@compute.service
sudo systemctl status tau@compute.service

Look for "active (running)" to confirm success.

DNS Configuration

Configure your domain's DNS records to delegate traffic to Seer service for load balancing.

At your domain provider, create the following records:

A Record (for now, just the first node):

Type: A
Name: seer
Value: YOUR_NODE_1_IP

NS Record (delegation):
```
Type: NS  
Name: tau
Value: seer.k8s.cyou.
```
The trailing dot is required. This delegates the tau subdomain to your platform.

DNS Verification

Test that your cloud is discoverable via public DNS using a tool like dig or an online equivalent.

dig tns.tau.k8s.cyou @8.8.8.8

Success returns your node's IP, proving public DNS is working.

Scaling: Adding More Nodes (Run on nodes 2, 3...)

1. Export & Prepare Configuration

On your primary node, export the shared cluster configuration.

# On Node 1
sudo tau config export

Copy the YAML output. On your second node, paste this into a new file: compute.tmpl.yaml. Then, edit the file and remove the location section.

2. Join the Cluster

Get the multiaddress from your primary node.

# On Node 1
sudo tau config show

Get the public IP of your new node.

# On Node 2
curl ifconfig.me -4

On the new node, generate its final configuration. This command uses the node's IP, the config template from the first node, and the first node's multiaddress to join the cluster.

# On Node 2
sudo tau config gen \
  --ip SECOND_NODE_IP \
  --use compute.tmpl.yaml \
  --bootstrap FIRST_NODE_MULTIADDRESS

3. Start the Service

Set up and start the same systemd service on the new node.

# On Node 2
sudo systemctl enable tau@compute.service
sudo systemctl start tau@compute.service

Repeat this scaling process for all remaining nodes.

Last touches

1. Update DNS Records

Go back to your domain provider and add A records for seer pointing to all your node IPs.

seer IN A <host-001-spain IP>
seer IN A <host-002-uae IP>
seer IN A <host-003-algeria IP>

Your DNS provider will now perform round-robin load balancing for you.

2. Ensure Bootstrapping Persistence

For a fully functional cluster that can recover from a full shutdown, every node must have a persistent list of its peers.

Collect the multiaddress from all nodes with sudo tau config show.
On each node, edit /tb/config/compute.yaml.
Add all collected multiaddresses to the peers list under the p2p section.
Restart the service on each node to apply the changes: sudo systemctl restart tau@compute.service.

3. Global Load Balancing Verification

Test your global setup. You should see all node IPs in the response.

dig tns.tau.k8s.cyou

The Grand Test: Deploying Your First Application

Time to see your cloud in action by deploying a real application. Find detailed steps here: https://tau.how/02-platform-getting-started/12-try-the-cloud/.

Accessing Taubyte Console

Navigate to console.taubyte.com and enter:

Email: Your email address
Domain: Your domain (e.g., k8s.cyou)

If the console accepts your domain, your nodes are properly configured and discoverable.

Project Creation

Log in with GitHub and create a blank project. The console will automatically detect your nodes and show them on a global map at their correct geographic locations.

Function Deployment

Create a simple "ping-pong" function to test your cloud:

Choose your domain from the dropdown.
Choose a function from the templates that returns "PONG".
Push to GitHub to trigger the automatic build and deploy pipeline.
Monitor build progress in the console.
Test the function using the lightning bolt icon.

For more, see the documentation at tau.how.

Spore Drive: Building a Cloud Platform in a Few Lines of Code

Samy Fodil — Thu, 31 Oct 2024 02:04:02 +0000

Building your own cloud platform has many benefits, including cost savings, ownership, sovereignty, and most importantly, the ability to make your solution self-hostable. However, constructing a cloud computing platform is a long, complex, and expensive endeavor—a problem partially solved with Tau. Why only partially? Simply because, though Tau is a simple-to-deploy single binary, you still need to get that done, configure Tau, and ensure dependencies like Docker are installed. Not to mention, you have to repeat this on each host for every Tau update. If you ask me, it's not developer-friendly just yet!

Introducing Spore-Drive

One option is using Infrastructure as Code (IaC) tools like Terraform or Pulumi, which come with their own complexities and learning curves—not to say these aren't developer-friendly tools. What we need is a tool that's specific to Tau, much like macOS is specific to a MacBook, delivering the best results and user experience. This is why I built Spore-Drive, a tool that allows you to build, expand, and keep a cloud up to date with just a few lines of code.

The idea is that given a configuration—which is generated or edited with code—we can determine and execute a deployment strategy. The first part relies on the concept behind go-seer, which essentially treats a folder containing YAML files as a database. This helps keep the state in a human-readable format, and we can also commit it to a Git repository. The second part consumes the configuration and deployment parameters to build a deployment plan (called a course) and then executes it, resulting in the deployment of Tau.

Architecture

Let's consider our goals:

Support for multiple programming languages
Web-ready
Enterprise in mind
No full rewrite for each language

With these goals in mind, we need to have Spore-Drive run as a service, with clients written in many languages. Because it has to be web-ready, we'll use ConnectRPC to implement the API.

The core can be written in Go—not only is it my language of choice, but many of the components we'll be using, like go-seer and mycelium, are written in Go. Moreover, we intend to have Spore-Drive be part of Tau's CDK, which means it needs to be embedded in Tau as well.

Using Spore-Drive

As of this article, I have only managed to create a TypeScript client. You can use Spore-Drive directly with Go, but a client is next on my list. So for this article, I'm going to use the TypeScript package.

Context

I have created droplets on DigitalOcean, and I'd like to build a cloud with the main domain being pom.ac, hosted by Namecheap.

I need to fetch droplet information to create or update my configuration, deploy with Spore-Drive, and then update my DNS records on Namecheap.

Create a Project

Let's create a TypeScript project:

$ npm init -y
$ npm install typescript tsx --save-dev
$ npx tsc --init

Edit your tsconfig.json:

{
  "compilerOptions": {
    "target": "ES2020",
    "module": "ESNext",
    "moduleResolution": "node",
    "declaration": true,
    "outDir": "dist",
    "rootDir": ".",
    "strict": true,
    "esModuleInterop": true,
    "resolveJsonModule": true,
    "skipLibCheck": true,
    "lib": ["ES2020"]
  },
  "include": ["src/**/*.ts"],
  "exclude": ["node_modules"]
}

Add the following to package.json:

"scripts": {
    "displace": "tsx src/index.ts"
},

Finally, install Spore-Drive:

$ npm install @taubyte/spore-drive

Create a new file src/index.ts and import Spore-Drive:

import {
  Config,
  CourseConfig,
  Drive,
  TauLatest,
  Course,
} from "@taubyte/spore-drive";

Configuration

Start by creating a configuration:

const config: Config = new Config();

If you'd like to open or sync your configuration to disk, pass a path like so:

const config: Config = new Config(`${__dirname}/../config`);

Then, initialize it:

await config.init();

This will have the Spore-Drive service create an instance of your configuration.

A cloud built with Tau requires some minimal configuration:

Root domain: Identifies the cloud. For me, it's pom.ac.
Generated domain: Can also be a subdomain.
Domain validation key
Swarm key

Let's start with these:

export const createConfig = async (config: Config) => {
  const domain = config.Cloud().Domain();
  await domain.Root().Set("pom.ac");
  await domain.Generated().Set("g.pom.ac");
  await domain.Validation().Generate();
  await config.Cloud().P2P().Swarm().Generate();
}

We need to define a signer for SSH. Here, I'm using username and password, but you can also use an SSH key.

export const createConfig = async (config: Config) => {
  // Previous code...
  const mainAuth = config.Auth().Signer("main");
  await mainAuth.Username().Set("root");
  await mainAuth.Password().Set(process.env.DROPLET_ROOT_PASSWORD!);
}

Hosts can run multiple instances of Tau; each is called a node, and each node can run multiple services grouped under what we call a shape. Here, I'm defining a shape called all, which includes all services except gateway.

export const createConfig = async (config: Config) => {
  // Previous code...
  const all = config.Shapes().Shape("all");
  await all
    .Services()
    .Set(["auth", "tns", "hoarder", "seer", "substrate", "patrick", "monkey"]);
  await all.Ports().Port("main").Set(BigInt(4242));
  await all.Ports().Port("lite").Set(BigInt(4262));
}

Next, we're going to add our hosts. Here, I'm using a helper (see do.ts) that lists all my droplets. If the host does not exist, I add it:

const bootstrapers = [];

for (const droplet of await Droplets()) {
    const { hostname, publicIp, tags } = DropletInfo(droplet);
    if (!hosts.includes(hostname)) {
      const host = config.Hosts().Host(hostname);
      bootstrapers.push(hostname);
      await host.Addresses().Add([`${publicIp}/32`]);
      await host.SSH().Address().Set(`${publicIp}:22`);
      await host.SSH().Auth().Add(["main"]);
      await host.Location().Set("40.730610, -73.935242");
      if (!(await host.Shapes().List()).includes("all"))
        await host.Shapes().Shape("all").Instance().Generate();
    }
}

await config.Cloud().P2P().Bootstrap().Shape("all").Nodes().Add(bootstrapers);

Note that I'm adding new hosts to bootstrapers because the next step is making sure all these hosts can bootstrap each other. This is not a requirement—this example has two droplets—but in a larger setup, you'd like to have bootstrap nodes that do not run any service.

Finally, we commit the changes:

await config.Commit();

Add the call to the createConfig function after initialization:

await config.init();
await createConfig(config);

Drive

Now that we have our configuration loaded, we can create a Drive. It takes a reference to our configuration instance and an optional Tau version or binary. Here, I'm going to make sure hosts are running the latest Tau:

const drive: Drive = new Drive(config, TauLatest);
await drive.init();

The next step is to plot a course:

const course = await drive.plot(new CourseConfig(["all"]));

Spore-Drive will determine the best strategy to deploy the latest version of Tau, while installing dependencies and making sure configuration for each node is generated.

To start the deployment:

await course.displace();

This starts the deployment (or displacement, as I call it). The process is run on the service side and can be queued (in the future).

To get updates on the process, you can call progress, which will feed you a progress stream:

for await (const displacement of await course.progress()) {
  // Handle progress updates...
}

Each element of the stream is DisplacementProgress and has:

path: /<course name>/<host name>/<task name>
progress: 0 to 100
error: Error message, if any

Check out displayProgress if you'd like to display the progress as progress bars.

Use your cloud!

Your next step is to use your newly deployed cloud. Check tau.how for more details on that.

Full Code

The full code for the pom.ac cloud can be found here.

In action

What's Next?

In my effort to close the gap between software development and cloud computing, I'm planning to build a similar interface for taucorder. Ultimately, both will become part of the Tau CDK.

Building Ollama Cloud - Scaling Local Inference to the Cloud

Samy Fodil — Mon, 08 Jul 2024 16:43:56 +0000

Ollama is primarily a wrapper around llama.cpp, designed for local inference tasks. It's not typically your first choice if you're looking for cutting-edge performance or features, but it has its uses, especially in environments where external dependencies are a concern.

Local AI Development

When using Ollama for local AI development, the setup is straightforward but effective. Developers typically leverage Ollama to run inference tasks directly on their local machines. Here's a visual depiction of a typical local development setup using Ollama:

This configuration allows developers to test and iterate quickly without the complexities of remote server communications. It's ideal for initial prototyping and development phases where quick turnaround is critical.

From Local to Cloud

Transitioning from a local setup to a scalable cloud environment involves evolving from a simple 1:1 setup (one user request to one inference host) to a more complex many-to-many (multiple user requests to multiple inference hosts) configuration. This shift is necessary to maintain efficiency and responsiveness as demand increases.

Here's how this scaling looks when moving from local development to production:

Adopting a straightforward approach during this transition can significantly increase the complexity of applications, especially as sessions need to maintain consistency across various states. Delays and inefficiencies may arise if requests are not optimally routed to the best available inference host.

Moreover, the complex nature of distributed applications makes them challenging to test locally, which can slow down the development process and increase the risk of failures in production environments.

Serverless

Serverless computing abstracts server management and infrastructure details, allowing developers to focus solely on code and business logic. By decoupling request handling and consistency maintenance from the application, serverless architecture simplifies scaling.

This approach allows the application to remain concentrated on delivering value, solving many common scaling challenges without burdening developers with infrastructure complexities.

WebAssembly

WebAssembly (Wasm) addresses the challenge of dependency management by enabling the compilation of applications into self-contained modules. This makes apps easier to orchestrate and test both locally and in the cloud, ensuring consistency across different environments.

Tau

Tau is a framework to build low-maintenance and highly scalable cloud computing platforms. It excels in simplicity and extendibility. Tau makes deployment straightforward and supports running a local cloud for development, allowing for end-to-end (E2E) testing of both the cloud infrastructure and the applications running on it.

This approach, referred to by Taubyte as "Local Coding Equals Global Production," ensures that what works locally will work globally, significantly easing the development and deployment processes.

Integrating Ollama into Tau with the Orbit Plugin System

Tau’s plugin system, known as Orbit, significantly simplifies turning services into manageable components by wrapping them into WebAssembly host modules. This approach allows Tau to take over the orchestration duties, streamlining the deployment and management process.

Exporting Functions in Ollama

To make Ollama functions accessible within Tau’s ecosystem, we utilize the Orbit system to export Ollama’s capabilities as callable endpoints. Here’s how you can export an endpoint in Go:



func (s *ollama) W_pull(ctx context.Context, module satellite.Module, modelNamePtr uint32, modelNameSize uint32, pullIdptr uint32) Error {
    model, err := module.ReadString(modelNamePtr, modelNameSize)
    if err != nil {
        return ErrorReadMemory
    }

    id, updateFunc := s.getPullId(model)

    if updateFunc != nil {
        go func() {
            err = server.PullModel(s.ctx, model, &server.RegistryOptions{}, updateFunc)
            s.pullLock.Lock()
            defer s.pullLock.Unlock()
            s.pulls[id].err = err
        }()
    }

    module.WriteUint64(pullIdptr, id)

    return ErrorNone
}

For a straightforward example of exporting functions, you can refer to the hello_world example.

Once defined, these functions, now called via satellite.Export, enable the seamless integration of Ollama into Tau’s environment:



func main() {
    server := new(context.TODO(), "/tmp/ollama-wasm")
    server.init()
    satellite.Export("ollama", server)
}

Writing Tests for the Ollama Plugin

Testing the plugin is streamlined and straightforward. Here's how you can write a serverless function test in Go:



//export pull
func pull() {
    var id uint64
    err := Pull("gemma:2b-instruct", &id)
    if err != 0 {
        panic("failed to call pull")
    }
}

Using Tau's test suite and Go builder tools, you can build your plugin, deploy it in a test environment, and execute the serverless functions to verify functionality:



func TestPull(t *testing.T) {

    ctx := context.Background()

<span class="c">// Create a testing suite to test the plugin</span>
<span class="n">ts</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">suite</span><span class="o">.</span><span class="n">New</span><span class="p">(</span><span class="n">ctx</span><span class="p">)</span>
<span class="n">assert</span><span class="o">.</span><span class="n">NilError</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>

<span class="c">// Use a Go builder to build plugins and wasm</span>
<span class="n">gob</span> <span class="o">:=</span> <span class="n">builder</span><span class="o">.</span><span class="n">New</span><span class="p">()</span>

<span class="c">// Build the plugin from the directory</span>
<span class="n">wd</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">os</span><span class="o">.</span><span class="n">Getwd</span><span class="p">()</span>
<span class="n">pluginPath</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">gob</span><span class="o">.</span><span class="n">Plugin</span><span class="p">(</span><span class="n">path</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span><span class="n">wd</span><span class="p">,</span> <span class="s">"."</span><span class="p">),</span> <span class="s">"ollama"</span><span class="p">)</span>
<span class="n">assert</span><span class="o">.</span><span class="n">NilError</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>

<span class="c">// Attach plugin to the testing suite</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">ts</span><span class="o">.</span><span class="n">AttachPluginFromPath</span><span class="p">(</span><span class="n">pluginPath</span><span class="p">)</span>
<span class="n">assert</span><span class="o">.</span><span class="n">NilError</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>

<span class="c">// Build a wasm file from serverless function</span>
<span class="n">wasmPath</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">gob</span><span class="o">.</span><span class="n">Wasm</span><span class="p">(</span><span class="n">ctx</span><span class="p">,</span> <span class="n">path</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span><span class="n">wd</span><span class="p">,</span> <span class="s">"fixtures"</span><span class="p">,</span> <span class="s">"pull.go"</span><span class="p">),</span> <span class="n">path</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span><span class="n">wd</span><span class="p">,</span> <span class="s">"fixtures"</span><span class="p">,</span> <span class="s">"common.go"</span><span class="p">))</span>
<span class="n">assert</span><span class="o">.</span><span class="n">NilError</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>

<span class="c">// Load the wasm module and call the function</span>
<span class="n">module</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">ts</span><span class="o">.</span><span class="n">WasmModule</span><span class="p">(</span><span class="n">wasmPath</span><span class="p">)</span>
<span class="n">assert</span><span class="o">.</span><span class="n">NilError</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>

<span class="c">// Call the "pull" function from our wasm module</span>
<span class="n">_</span><span class="p">,</span> <span class="n">err</span> <span class="o">=</span> <span class="n">module</span><span class="o">.</span><span class="n">Call</span><span class="p">(</span><span class="n">ctx</span><span class="p">,</span> <span class="s">"pull"</span><span class="p">)</span>
<span class="n">assert</span><span class="o">.</span><span class="n">NilError</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>



}

Code

You can find the complete code here https://github.com/ollama-cloud/ollama-as-wasm-plugin/tree/main/tau

What's Next?

You can now build LLM applications with ease. Here are the steps to get started:

Start locally using dream: Set up your local environment to develop and test your application.
Create a project: Begin a new project with Tau to harness its full potential.
Create your production cloud: Deploy your project in a production cloud environment.
Drop the plugin binary in the /tb/plugins folder.
Import your project into production
Show off!

Building a Cloud Development Kit (CDK)

Samy Fodil — Wed, 03 Jul 2024 07:39:46 +0000

What Exactly is a Cloud Development Kit (CDK)?

Imagine you're a developer who needs to set up a bunch of cloud resources. Traditionally, you might deal with endless lines of JSON or YAML configurations. It's precise but can get pretty tedious, right? Well, this is where a Cloud Development Kit, or CDK, comes in handy. Instead of those endless configuration files, you use a programming language you’re already comfortable with—like TypeScript, Python, or Java. This means you can code your cloud infrastructure just like you’d code anything else.

Why is the CDK a Game Changer?

Let’s break it down:

Productivity on Steroids: Forget about switching gears between applications and infrastructure. Now, it's all in one place, with tools you already love (and understand!). Autocomplete, refactoring, and error checking are right there with you.
Reuse and Recycle: Craft a piece of infrastructure once, wrap it up into a component, and reuse it anywhere you need it. This not only saves time but also keeps your setups consistent.
Tailor-Made Solutions: Extend the basic setups with your own tweaks. Need a special kind of storage or a unique authentication method? Just code it in.
Transparent and Controlled: It’s all in your codebase, visible and version-controlled. Every change is clear and trackable—no surprises.

AWS CDK

AWS was one of the first to jump on this bandwagon with their CDK. It takes the power of AWS CloudFormation and makes it friendlier. Instead of wading through those YAML or JSON templates, you write in a comfortable, expressive programming language.

Here’s an example of AWS CDK with TypeScript:

import * as cdk from '@aws-cdk/core';
import * as s3 from '@aws-cdk/aws-s3';

class MyCloudStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Setting up an S3 bucket with version control
    new s3.Bucket(this, 'MyFirstBucket', {
      versioned: true,
      removalPolicy: cdk.RemovalPolicy.DESTROY,
    });
  }
}

const app = new cdk.App();
new MyCloudStack(app, 'MyCloudStack');

See how intuitive and straightforward defining cloud resources can be with CDK. It’s just like writing any other piece of software!

AWS CDK Workflow Explained

The AWS Cloud Development Kit (AWS CDK) utilizes a sophisticated workflow that integrates various AWS services to provide a seamless development and deployment experience. Here’s a breakdown of the typical workflow and how it leverages these services:

Define Infrastructure as Code: You start by defining your cloud resources using familiar programming languages such as TypeScript or Python. This code defines what resources you need, such as databases, storage buckets, or compute instances.
Synthesis: The AWS CDK app takes this high-level code and compiles it into a lower-level CloudFormation template. This process, known as synthesis, involves the CDK CLI (cdk synth) transforming your declarative setup into a set of instructions that AWS CloudFormation can understand.
Deployment: Once the CloudFormation template is synthesized, you deploy it using the CDK CLI (cdk deploy). This command instructs AWS CloudFormation to provision and manage the resources as specified in the template.
Utilization of AWS Services: Throughout this process, AWS CDK interacts with various AWS services like AWS CodePipeline and AWS CodeCommit for continuous integration and delivery, enhancing the CI/CD pipeline's efficiency and robustness.

Building a CDK

Now that we understand what a CDK is, let's look at designing one for a cloud platform. Here, I'll be focusing on tau, an open-source CDN PaaS I founded.

Tau uses YAML for resource definition. This approach is not unique; Kubernetes and many other platforms use YAML as well. However, what sets tau apart is its design philosophy: Git is the single source of truth. This means there are no APIs to call for defining resources like storage buckets. Instead, everything is managed through Git.

This design is advantageous because it essentially provides us with an equivalent to AWS CloudFormation, but integrated directly with Git. This tight integration simplifies infrastructure management and makes it inherently version-controlled and collaborative.

All we need to do is figure out a way to generate and edit YAML files with code.

The Schema Package

Because Tau is a cloud platform you can write tests for, we already built a package called schema to manipulate YAML configuration files. The catch is that it has to be in Go, while most CDKs at least support JavaScript/TypeScript and Python.

This creates a challenge: how can we extend our CDK to support these popular languages while leveraging the existing schema package?

WebAssembly

Go is one of the first languages that supported WebAssembly as a compilation target. Additionally, most programming languages either have a native implementation or bindings for a WebAssembly runtime.

So, if we compile the schema package into a WebAssembly module, we should be able to call it from a variety of languages. This approach leverages the strengths of Go and the flexibility of WebAssembly, making it possible to interact with Tau's infrastructure definitions from languages like JavaScript, TypeScript, and Python.

Extism

Writing code to initiate a WebAssembly runtime, load our module, and call exported functions can be quite a lot of work. Luckily, the team at dylibso has thought about that and built Extism, a framework for building plugins with WebAssembly.

Example of Extism Plugin

Here’s a simple example of a Go plugin using Extism:

package main

import (
    "github.com/extism/go-pdk"
)

//export greet
func greet() int32 {
    input := pdk.Input()
    greeting := `Hello, ` + string(input) + `!`
    pdk.OutputString(greeting)
    return 0
}

func main() {}

Loading and calling greet from JavaScript is straightforward:

import createPlugin from '@extism/extism';

const plugin = await createPlugin('plugin.wasm', {
  useWasi: true,
});

const out = await plugin.call('greet', 'Samy');
console.log(out.text());

await plugin.close();

Using Extism, we can easily compile our Go code into WebAssembly and call it from other languages like JavaScript, making the integration process seamless and efficient. This capability significantly reduces the complexity of working with WebAssembly, enabling us to extend Tau’s CDK functionalities to various programming environments effortlessly.

The Plugin

The plugin is a wrapper around the schema package that uses the Extism PDK. Just like schema, the first step involves opening a project by reading a folder that contains configuration files (the folder can also be empty).

Here's how you can do it in Go:

package main

import (
    "github.com/extism/go-pdk"
    proj "github.com/taubyte/tau/pkg/schema/project"
)

var project proj.Project

//export openProject
func openProject() int32 {
    var err error
    project, err = proj.Open(proj.SystemFS("/mnt"))
    if err != nil {
        pdk.SetError(err)
        return 1
    }

    return 0
}

You might notice the global variable project. This isn't a novice mistake:

We will load the module each time we open a project.
WebAssembly is single-threaded, at least for now.

The JavaScript CDK

On the JavaScript side, we have:

import createPlugin from '@extism/extism';
import * as path from 'path';

export async function core(mountPath: string): Promise<any> {
    const coreWasmPath = path.resolve(__dirname, '../core.wasm');

    return await createPlugin(
        coreWasmPath,
        {
            useWasi: true,
            allowedPaths: { '/mnt': mountPath },
        }
    );
}

This function, core, loads the module and attaches the folder containing the configuration files to it as /mnt.

To open a project, we load the Wasm module, call openProject, and then return a Project object that references an instance of the core module initialized to our project.

export async function open(mountPath: string): Promise<Project> {
    const plugin = await core(mountPath);

    await plugin.call('openProject');

    return new Project(plugin);
}

From there, we can write code like this:

import open from '@taubyte/cdk';

const prj = await open('path/to/folder');
await prj.functions().new({
  name: 'ping',
  method: 'http',
  // other configurations
});

await prj.close();

Next Steps

Now that we have a CDK, the next step is to integrate it with Tau's CI/CD workflow. This integration will enable seamless deployment and management of cloud resources directly from your development pipeline. I’ll cover this integration in a separate article, where we will dive into the specifics of automating your deployments.

Conclusion

In this article, we explored what a Cloud Development Kit (CDK) is and why it’s a game changer for cloud infrastructure management. We looked at how AWS CDK simplifies defining and deploying cloud resources using familiar programming languages.

By leveraging WebAssembly and the Extism framework, we were able to create a versatile CDK that supports multiple languages, making tau’s infrastructure management more accessible and efficient.

My vision, is that this will evolve to also allows developers to embed infrastructure definitions directly within their application code, providing a seamless and cohesive development experience.

Stay tuned for the next article, where we’ll dive into integrating the CDK with tau’s CI/CD workflow.

🔥 10x the odds of landing a Go job 🔥

Samy Fodil — Sun, 12 May 2024 05:18:08 +0000

First it was the market, and now it's AI. Let's face it, landing a developer job nowadays is challenging, even for non-JavaScript positions—and it's even harder for junior developers!

So the question is: How can you stand out?

Most software developers gravitate toward JavaScript and Python because, although they are not the easiest to debug, they are the easiest to learn. Guess what? Large Language Models (LLMs) excel in these languages as well. By choosing less common languages like Go, you're already improving your job prospects. Just to be clear, I'm not saying these languages are bad; every language has its pros and cons.

If you're a junior Go developer, you likely started by building some ORM-based backends in Go. Think about it; it's what everybody does: it's easy and familiar. But this approach does not help you build the real skills that set you apart.

Go is very popular in infrastructure; just look at Terraform, Kubernetes, and many other extremely popular projects—they are all built in Go. Some, like Kubernetes, even have a pure-Go policy.

I hope at this point you see where I'm going: If I have in front of me a list of candidates and one of them demonstrates they can go beyond the ORM use-case, that person will definitely get an interview!

Now, without professional experience, how would you demonstrate or at least signal that you're a Go-wiz? The answer is simple: contributing to open-source projects.

You have to choose the projects you'd like to contribute to very carefully. Larger projects may not have many issues for a junior developer; however, it might take forever to get your PR merged, if it even makes it there.

The great news is there are a lot of newer projects in the infrastructure space. These projects have easy-to-tackle issues and will be more willing to work with you and even merge your PRs very quickly.

Here are a few projects you should consider:

Tau - Open-source Cloud Computing Platform on Autopilot.
Drand - A Distributed Randomness Beacon Daemon - Go implementation.
KCL - KCL Programming Language (CNCF Sandbox Project).
Fury - A blazingly fast multi-language serialization framework powered by JIT and zero-copy.
Camel-K - Apache Camel K is a lightweight integration platform, born on Kubernetes, with serverless superpowers.

Before you dig in, a few things you should keep in mind:

How complex is the setup: Generally not an issue with Go projects unless you see a Makefile at the root of the repository.
How many contributors: If the project has a lot of contributors, you should expect a longer time for your PR to get accepted.
Is there any issue with the 'good first issue' label: You don't have time nor enough knowledge of the repository to create your own issue.

I hope this advice helps you enhance your Go skills and boosts your chances in the job market!

Introducing A Framework for Building Cloud Computing Platforms

Samy Fodil — Tue, 06 Feb 2024 04:46:29 +0000

If you're reading this, chances are you're either curious about the next big thing in cloud computing or you're actively seeking ways to enhance your development workflow. As the creator of Taubyte, I'm here to share a journey into the future of cloud computing—a future we're building to be as intuitive as software development itself. This isn't just another tool; it's a paradigm shift designed with you, the developer, at heart.

Taubyte simplifies the process of building a cloud computing platform, making it a no-brainer. Unlike other products, such as Kubernetes, which were not designed with software developers in mind nor as platforms, Taubyte eliminates the need for DevOps and platform engineering. In essence, Taubyte is designed for software developers, requiring no DevOps nor platform engineering skills. To achieve this, we aimed not just for automation but for autonomy.

If you're a DevOps professional reading this, you might think it sounds too good to be true, or perhaps you're inclined to deny its feasibility, fearing it will make your job obsolete. However, I invite you to stay with us because Taubyte is as real as it is open source. Moreover, I believe it will give DevOps and Platform Engineers an unprecedented advantage.

Let's get our hands dirty!

The best way to learn something is by doing it, and that's exactly what we're going to do. All you need is npm. While there are other methods, we'll stick to npm as it's the simplest and most widely available, even on Windows.

What we'll do is install dreamland, a project hosted on github.com/taubyte/dreamland that allows you to run a Taubyte-based Cloud Computing Platform locally. It's an excellent tool for local development and testing, but we'll use it in a slightly different way.

Run the following command to install dreamland:

$ npm i @taubyte/dream

Now, you should have the dream command-line tool available. Let's start a cloud:

$ dream new multiverse
[INFO] Dreamland ready
[SUCCESS] Universe blackhole started!

You can ignore any ERROR messages you might see about seer.

Once you see SUCCESS, it means your cloud has been fully started.

A first look

First, let's explore what's happening using the command line. Run:

$ dream status universe

This should output something like:

┌───────┬─────────────────────┬────────┬───────┐
│ Nodes │ elder@blackhole     │ p2p    │ 14051 │
│       ├─────────────────────┼────────┼───────┤
│       │ tns@blackhole       │ http   │ 14466 │
...

This lists all the running nodes. Each node could be running a specific protocol (e.g., tns or substrate) and will reflect that. Others will have a role like elder (bootstrap node) or client.

Also, @blackhole indicates that the node is part of the blackhole universe. Each universe represents a cloud and is cryptographically isolated from any other cloud on the peer-to-peer layer.

On the right side, you can see the TCP ports each node is using along with the number of copies.

Now, let's leave the terminal and head to console.taubyte.com where you're going to locate the Dreamland button. Then click on it.

The button will be disabled or hidden if dreamland is not detected.

Now, locate the sidebar: (1) click on Network, then (2) on blackhole.

After a few seconds, you should see something similar to:

On the network graph, which you can manipulate, you can see all the same nodes as before from the CLI.

If you hover over any node, it'll display the TCP ports it's using.

What's with the mesh network?

Hosts, or as we call them, Nodes, form a Taubyte-based Cloud Computing Platform and communicate with each other using a peer-to-peer network. Running locally through dreamland, all the nodes are interconnected, although this might not always be the case in production, especially if you have many nodes.

The peer-to-peer nature of Taubyte-based clouds gives them important characteristics like resilience and redundancy.

Protocols

While in production a node can run multiple protocols by defining a shape (a term reminiscent of the internal name tau had: odo, a reference to the shape-shifting Star Trek character), in dreamland, each node runs only one protocol. This design choice simplifies protocol debugging.

Without further ado, here's what each protocol does:

auth handles token validation and secret management.
tns ensures project registry consistency across the cloud.
seer is responsible for DNS resolution and load balancing.
patrick bridges git events into CI/CD jobs.
monkey executes CI/CD jobs.
hoarder manages replication.
substrate handles request processing and serving.
gateway manages L7 load-balancing.

The other nodes

The other nodes, referred to as Simples (admittedly, not the best name), include elder and client. While they are not crucial for the purpose of this article and could be excluded when starting the universe, it's helpful to understand their intended roles:

elder nodes are used for bootstrapping. Since dreamland interconnects all nodes, an elder is included for testing purposes when using libdream.
client nodes are lightweight and can be used to test protocols or peer-to-peer services deployed on the cloud.

Note: Some fixtures, which will be covered in another article, may require Simples to function correctly.

Creating a project

Now, let's create a project on this cloud. Although it can be done using tau-cli, which you might prefer if you're a terminal enthusiast like me, we're going to use the Web Console instead.

Open a new browser window or tab and navigate to console.taubyte.com.

(1) Type your email, then (2) click on the network selection input, and finally (3) click on Dreamland/blackhole.

Next, click the Login with GitHub button.

You will be asked to grant the app GitHub access. This access is local to your browser and is used to create repositories related to your project, which will be cloned into a browser-based virtual file system.

The next screen will display "Your projects" and should show no projects. You'll want to click on Create project.

Enter a project name, a description, toggle private (unless you want the project repositories to be public), then click on Create Project.

The Web Console will create two repositories: one for configuration, containing mainly YAML files, and another for inline code, containing code in different languages to be compiled to WebAssembly. If everything goes well, you'll be directed to the dashboard.

Creating a Function

Taubyte-based Clouds support serverless functions. Let's create one!

Click on Functions in the side menu, then on the + button.

To expedite the process, we'll use a template. Skip filling any field for now and click on the Template Select button.

This action opens another modal at the top where you can select a template. Although you can choose Go, Rust, or AssemblyScript at the time of writing this article, we'll keep it simple and go for Go! (pun intended). So: (1) select Go, (2) select ping_pong, and finally (3) close the modal.

All fields have been automatically filled except for the domains. Proceed by (1) clicking on the list and (2) selecting GeneratedDomain. This action will create a domain resource for you.

It's worth noting that the Web Console clones your repositories in-browser, meaning there's always a code equivalent to everything you do. Let's review the YAML config by clicking on the Yaml tab.

You will see YAML code resembling the following:

id: ''
description: Returns pong to a ping over HTTP
tags: []
source: .
trigger:
  type: https
  method: GET
  paths:
    - /ping
domains:
  - GeneratedDomain
execution:
  timeout: 10s
  memory: 10MB
  call: ping

Most of this YAML should be self-explanatory, but let me clarify a few points:

source: can either be . for inline code or the name of a library (to be explored in other articles).
execution.call: specifies the function to be executed and must be exported by the WebAssembly module.

Next, let's inspect the code. The Web Console features a modest code editor useful for quick inline code updates. Access it by clicking on the Code tab.

In the editor, you will observe (1) the usual Go files including go.mod, (2) the code for our handler function ping, and (3) a .taubyte folder defining how the function is built.

While understanding the entire code or mastering Go isn't necessary, I'll explain a few basic concepts that will map in a way or another to any serverless function in supported languages:

The package name can be anything except main. The building container, taubyte/go-wasi on Docker Hub, wraps your function in a special main function.
The github.com/taubyte/go-sdk package simplifies interactions with the Taubyte VM, making it straightforward to use VM primitives.
We use tinygo to compile Go into small, efficient WebAssembly modules. To export a function, annotate it with the //export comment.
Taubyte supports various trigger types, so handlers receive an event. We care about efficiency, so event.Event is really a wrapper of a uint32, minimizing unnecessary memory allocation and copying.

Click Done to proceed.

The function should now appear listed.

Everything done so far is confined to the virtual filesystem in your browser. To apply these changes, we need to push them. Find the green button at the bottom right of the screen and click on it.

A modal will guide you through the code changes, starting with configuration changes. Click on the domains folder, then open the GeneratedDomain.yaml file.

Copy the FQDN generated for you as we will need it later:

fqdn: gftxhd6h0.blackhole.localtau

Click on Next to review code changes.

One more click on Next takes you to the final step, where you'll (1) enter a commit message and (2) push the changes to GitHub.

In production, this push would trigger an event captured by patrick. However, because we're using dreamland and GitHub cannot reach your nodes directly, we run a fixture called push-all to emulate the git events. Switch back to your terminal and run:

$ dream inject push-all

Return to the Web Console and, on the side menu, click on Builds. You should see two CI/CD jobs, one for configuration and one for code.

After a few seconds, the build should complete. Click on Refresh if it seems delayed. Then, click on the stack icon to view the ping_pong function.

Executing the Function

Now that our function has been built and its configuration published on tns, we can execute it by hitting the endpoint. Since we're running a local cloud, we need to find out which port the substrate (or gateway, if you're using one) node uses to handle HTTP requests. To do this, run:

$ dream status substrate

In my case, the HTTP port is 14529.

┌─────────────────────┬────────┬───────┐
│ substrate@blackhole │ copies │     1 │
│                     ├────────┼───────┤
│                     │ dns    │ 14304 │
│                     ├────────┼───────┤
│                     │ http   │ 14529 │
│                     ├────────┼───────┤
│                     │ p2p    │ 14282 │
└─────────────────────┴────────┴───────┘

You can test the function using curl as follows, making sure to replace gftxhd6h0.blackhole.localtau with your own generated domain:

$ curl -H "Host: gftxhd6h0.blackhole.localtau" http://127.0.0.1:14529/ping

Output:

PONG

Alternatively, you can add the generated domain to your local hosts file (/etc/hosts in my case) as follows:

$ sudo vi /etc/hosts
127.0.0.1 gftxhd6h0.blackhole.localtau

Save and exit, then run curl again but without the Host header:

$ curl http://gftxhd6h0.blackhole.localtau:14529/ping

Output:

PONG

If you've added the generated domain to your /etc/hosts, you can also use the Web Console to hit the function's endpoint: (1) In the sidebar, click on Functions, then (2) click on the thunder icon.

This action opens a new tab where you should see something like this:

Congratulations! You've successfully created a cloud and executed a serverless function on it!

What Just Happened!

In this section, we'll take a step back to review the processes and protocols involved in each step of deploying and executing a serverless function with Taubyte. This overview will help clarify the interactions between different components and the significance of each protocol in the overall architecture.

Project Creation

The creation of a project is predominantly a client-side operation, facilitated by either the Web Console or the tau-cli. This process involves direct communication with GitHub to establish the necessary repositories. Only towards the end does the auth protocol come into play, tasked with registering these repositories. This registration process includes storing critical secrets such as the deployment key, essential for cloning code during CI/CD processes, and the webhook secret, used to authenticate events coming from GitHub.

The sequence diagram below illustrates these interactions in detail:

Pushing Changes to GitHub

When changes are pushed to GitHub, a sequence of events is triggered, initiating a process that involves multiple protocols to manage and execute CI/CD jobs. The patrick protocol plays a pivotal role in verifying the legitimacy of events and creating jobs, while the monkey protocol is responsible for the execution of these jobs. Subsequently, the tns stores and replicates the compiled configuration, and the hoarder protocol ensures the replication of updated assets.

The sequence diagram below visualizes this workflow:

It's important to note that because we are using Dreamland, a simulated environment, there's no direct way for GitHub to communicate with your nodes. This limitation is addressed by using the fixture called push-all, which emulates the git events that would typically trigger the CI/CD process in a production environment.

Hitting the Endpoint

The process for handling a request at an endpoint varies depending on the network's configuration. In a simple setup without gateways and HTTPS, the request's host name is resolved by the seer protocol before the request itself is processed by a substrate node. However, in networks with gateways or HTTPS requests, additional steps, like retreiving x509 certificate from auth, are involved.

The diagrams below detail these processes for different configurations:

Without Gateway and HTTPS:

With Gateway:

With HTTPS:

Note: Taubyte supports the Automatic Certificate Management Environment (ACME) protocol for auto-generating certificates using Let’s Encrypt.

These diagrams demonstrate the seamless and autonomous nature of the Taubyte platform, which requires no manual configuration and ensures a smooth deployment and execution process for serverless functions.

A Glimpse into the Future

This article is but a gateway into the vast potential that Taubyte unlocks. Through a hands-on demonstration of setting up a cloud environment and deploying serverless functions, I hope to have given you a taste of the simplicity and efficiency Taubyte brings to cloud computing.

Our mission with Taubyte is to democratize cloud development, making it as natural and creative as the development process itself. Whether you're venturing into the cloud for the first time or you're a seasoned developer seeking to streamline your processes, Taubyte is designed to empower your innovation.

As we embark on this journey together, I invite you to explore, to experiment, and to contribute. With Taubyte, the future of cloud computing is not just about technology; it's about the community and creativity it fosters. Welcome to a new era of cloud computing, where the skies are not the limit—they're just the beginning.

Harnessing LLM with WebAssembly

Samy Fodil — Tue, 07 Nov 2023 16:38:37 +0000

In the cloud computing ecosystem, infusing applications with the power of Language Learning Models (LLMs) has become increasingly accessible. This article will guide you through integrating an LLM into your cloud project, utilizing the capabilities of WebAssembly (WASM) within the open-source framework of the 8bit.ws cloud.

LLM Capabilities via 8bit.ws Cloud
The 8bit.ws cloud extends LLM features through its Satellite system, enabling serverless functions to harness AI's predictive power. Here’s how to exploit these capabilities in your cloud functions.

LLAMA Satellite & its SDK
Our Cloud Computing Network provides LLM capabilities through what we call a Satellite. It does so by exporting llama.cpp capabilities to the Taubyte Virtual Machine, which powers Serverless Functions (or DFunctions, as per Taubyte's terminology). The source code for the Satellite can be found here.

Satellites export low-level functions that aren't very intuitive to use directly. Fortunately, it's possible to address that with a user-friendly SDK. As of today, we offer a Go SDK. The source code can be found here.

Building a Predictive Function
Let’s construct a function that uses LLM to process prompts.

Before proceeding, let's ensure you have a project and a DFunction ready to go. If not, please refer to "Create a Function".

If you followed the steps from Taubyte's Documentation, your basic function should look something like this:

package lib

import (
    "github.com/taubyte/go-sdk/event"
)

func ping(e event.Event) uint32 {
    h, err := e.HTTP()
    if err != nil {
        return 1
    }

    h.Write([]byte("PONG"))

    return 0
}

We will transform this ping function to a predict function that takes a prompt from the POST request body and returns a prediction.

Incorporating LLM Predictions
Here's how to modify your function to perform LLM predictions with the LLAMA SDK:

package lib

import (
    "github.com/taubyte/go-sdk/event"
    "github.com/samyfodil/taubyte-llama-satellite/sdk"
    "io"
)

func predict(e event.Event) uint32 {
    h, err := e.HTTP()
    if err != nil {
        return 1
    }
    defer h.Body().Close()

    prompt, err := io.ReadAll(h.Body())
    if err != nil {
        panic(err)
    }

    p, err := sdk.Predict(
        string(prompt),
        sdk.WithTopK(90),
        sdk.WithTopP(0.86),
        sdk.WithBatch(512),
    )
    if err != nil {
        panic(err)
    }

    for {
        token, err := p.Next()
        if err == io.EOF {
            break
        } else if err != nil {
            panic(err)
        }
        h.Write([]byte(token))
        h.Flush()
    }

    return 0
}

Explanation
This function reads the POST body as the prompt for the LLM. It then requests a prediction and enters a loop to write each token received back to the HTTP response. We’ve used the sdk.Predict method to send the prompt to the LLM and p.Next to iterate over the tokens.

Deploying Your Function
Once your function is ready, deploy it to the 8bit.ws cloud. Ensure your function's entry point and method are configured correctly to handle POST requests.

Conclusion
By integrating LLM with WASM in the 8bit.ws cloud, we unlock the potential for applications that are not just reactive, but proactive and intelligent. As open-source initiatives, 8bit.ws and Taubyte provide fertile ground for innovation, inviting developers to build, share, and improve upon a collective vision for the future of cloud computing.