DEV Community: Simon Pfeiffer

Build your own VS Code extension

Simon Pfeiffer — Mon, 10 Jun 2024 12:19:48 +0000

Visual Studio Code is a powerful code editor known for its extensibility. Users can install various extensions to fit their needs, and if something isn't available, they can build their own extensions to enhance productivity. To do this, an extension author needs to be familiar with the VS Code API and the development workflow for creating extensions.

In this blog post, we will show you the framework we used to build an extension using a webview to display anything you like. We will guide you through the development of an extension, where you can modify the project to do specific tasks, and finally, we will show you how to officially publish an extension on the VS Code Marketplace.

The published VS Code extension

Extension listed in VS Code

Preparation

The VS Code extension will be a TypeScript project, and for the webview, we will use Svelte as the frontend framework. We have prepared a VS Code extension development template so that you can clone this repository and get started right away:

GitHub repository: https://github.com/codesphere-cloud/vscode-extension-template

git clone https://github.com/codesphere-cloud/vscode-extension-template
cd vscode-extension-template
npm i

This will download the VS Code template and install all the dependencies you need for developing your extension.

That's it. You are ready to test and build your extension. First, we want to make sure that the tutorial extension works. When developing an extension, we want to test every change right away, and testing should be a smooth workflow.

We can open a debug VS Code window called Extension-Development-Host to test our extension. First, we run a command to compile our extension while working on it.

npm run watch

Tip: open a new Terminal with this Keyboard combination: Ctrl + Shift + .

This compiles our extension every time we are changing our project. now we open either src/SidebarProvider.ts or src/extension.ts file and push F5 to open a new extension-developing-host. an this is the result:

Activated tutorial extension in an extension-development-host

The template extension we provide adds a view to the activity bar on the side with a webview a user can interact with. We kept this tutorial template as simple as possible, so you can follow along with this tutorial and get to know the project better while adding new features to it.

This tutorial focuses on creating a webview in the side panel of VS Code, but you can do much more than that. In the last section of this blog post, we will list a bunch of links to useful resources for further reference.

Now, we will build a GitHub Copilot-style chat interface together, but with a locally installed Llama.cpp.

Project idea

The idea is to replicate the GitHub Co-Pilot chat interface on VS Code but with a locally self-hosted Llama.cpp instance. The use case for this would be if you are working on a project which data should not be exposed to other services. Such an approach of using AI would be data compliant.

We need to set up our local Llama.cpp instance and use the server example of this repository to host our own OpenAI compatible chat completion, so we have a nice to use API. Now, we guide you through the process of setting up your locally installed Llama.cpp instance.

Firstly, you need to clone the GitHub repository of llama.cpp and build the project with make . Make sure that you are in the root directory of the extension:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp/
make

Depending on your local resources this command might take while. After completion we can continue to set up the OpenAI compatible server:

cd examples/server
make server

Like before this might take a while. After building the server we need to download our model we want to use. The important thing is to use models in the .gguf format. You can browse avalable models on HuggingFace: https://huggingface.co/models?library=gguf&sort=trending

We chose the qwen2-7b-instruct-q8_0.gguf model: https://huggingface.co/Qwen/Qwen2-7B-Instruct-GGUF/tree/main

It is 8 GB and to download it takes a while. You can download it to your models/ directory in your root:

cd models/
wget https://huggingface.co/Qwen/Qwen2-7B-Instruct-GGUF/resolve/main/qwen2-7b-instruct-q8_0.gguf?download=true

Simply replace this link with the link to the model you want to use. Now, you need to wait until this model is downloaded.

When downloading is finished you can use this model right away. Execute this command in the root directory of llama.cpp:

chmod a+x ./server
./server -m models/qwen2-7b-instruct-q8_0.gguf -c 2048

And again, if you chose an other model replace the path to the model to the right file.

You can now use the Llama.cpp interface on localhost:8080

Llama.cpp server on localhost

You can for sure optimize the performance of the model running now on your localhost, but lets stick with this for this tutorial because we want to build a VS Code extension.

Since we are using the API we wont need that UI. Here is a documentation about the API: https://github.com/ggerganov/llama.cpp/tree/master/examples/server#API-Endpoints

Features to add

Inside our sidepanel extension we want to add these features:

start your AI model out of VS Code
place an input field for your prompts
use the completion endpoint of our model to generate a response
display the response to our webview

We want to keep this extension simple and if you want to you can take the challenge and improve it as you like! 🤗

Creating a Button to Start and Stop Llama.cpp

Let's create buttons so that we don't have to type the command in the terminal every time we want to use our Llama instance. Additionally, it would be convenient to have a button to stop the model from running, as we don't want to manually find out the PID in order to kill the process.

The idea is to add icons to the top-right corner. Perhaps you're familiar with some extensions that have these small icons in the top-right corner (e.g., GitHub Co-Pilot).

Icons in the right top corner

To add these buttons, we need to modify our package.json file first. The package.json file in our project is an important file filled with various metadata about your extension. This file serves as one of the main entry points to your extension. Inside the package.json, there is a field called menus. We will add two additional icons to our existing icon by appending these objects to that array.

"menus": {
      "view/title": [
        {
          "command": "tutorial.tutorial",
          "when": "view == tutorial-sidebar",
          "group": "navigation"
        },
        {
          "command": "tutorial.start",
          "when": "view == tutorial-sidebar && !tutorial.modelIsRunning",
          "group": "navigation"
        },
        {
          "command": "tutorial.stop",
          "when": "view == tutorial-sidebar && tutorial.modelIsRunning",
          "group": "navigation"
        }
      ]
    },

In the when attribute, you can specify conditions for when the icons should be displayed. We can control variables through the VS Code API. We want the start icon to be displayed when the model is not running, and when it is running, we want to display the stop button.

We aim to use small icons for these commands. There is a variety of internal icons available for use: Icon Listing

In our package.json file, there is a commands field where we can add such icons. You can simply copy this snippet and replace it with the existing one:

"commands": [
      {
        "command": "tutorial.tutorial",
        "title": "tutorial",
        "icon": "$(log-out)",
        "category": "Tutorial command"
      },
      {
        "command": "tutorial.start",
        "title": "Start",
        "icon": "$(play)",
        "category": "Tutorial command"
      },
      {
        "command": "tutorial.stop",
        "title": "Stop",
        "icon": "$(stop)",
        "category": "Tutorial command"
      }
    ]

Now we save with Ctrl + S and navigate to our Extension Development Host, then press Ctrl + R to refresh the window. You should now see the icons displayed in the top-right corner.

Icons in the top corner

Perhaps you noticed that the stop button is not displayed. This is because of the condition we provided in the package.json.

Now let's code the commands that are executed when clicking on these icons. We need to open the src/extension.ts file to register our commands with the extension. We register the commands in the activation function of this file. In our package.json, there is a field named activationEvents where we can specify when the activation function is executed.

"activationEvents": [
    "onStartupFinished"
  ],

The activation event is set to onStartupFinished. This means the activation function is executed when VS Code is fully loaded. You can copy this code snippet and paste it into extension.ts:

context.subscriptions.push(
  vscode.commands.registerCommand('tutorial.start', async () => {
    vscode.commands.executeCommand('setContext', 'tutorial.modelIsRunning', true);
  })
);

context.subscriptions.push(
  vscode.commands.registerCommand('tutorial.stop', async () => {
    vscode.commands.executeCommand('setContext', 'tutorial.modelIsRunning', false);
  })
);

Now, when you reload the Extension Development Host with Ctrl + R, you can test these icons, and you will notice that after clicking, the icon changes. This is the result of using the condition. We can change the context for the modelIsRunning variable during the runtime of your extension.

Now, we actually want to add functionality to our buttons. We can do that by simply extending the registerCommand() code block with the logic we want to have.

We want to start our LLama.cpp instance when clicking start and stop it when clicking stop. We need to execute these bash commands. You can again simply copy and paste this code snippet into the code you already have:

a) start Llama.cpp on localhost:8080:

context.subscriptions.push(
  vscode.commands.registerCommand('tutorial.start', async () => {

    const extension = vscode.extensions.getExtension('Tutorial.tutorial')?.extensionPath;

    // Your Llama.cpp folder has to be inside the extension folder
    const serverPath = path.join(extension, 'llama.cpp', 'server');
    const modelPath = path.join(extension, 'llama.cpp', 'models', 'qwen2-7b-instruct-q8_0.gguf');
    const bashcommand = `${serverPath} -m ${modelPath} -c 2048`;

    const test = 'echo hi';

    exec (bashcommand, (error, stdout, stderr) => { 
        if (error) {
            console.error(`exec error: ${error}`);
            return;
        }

        if (stderr) {
            console.error(`stderr: ${stderr}`);
            return;
        }

        console.log(`stdout: ${stdout}`);
    });
    vscode.commands.executeCommand('setContext', 'tutorial.modelIsRunning', true);
  })
);

Note: We will save the PID (process ID) of the process to our extension's globalStorage to ensure that we can kill this process with our extension's stop button. The globalStorage can be used for state management in our extension.

b) stop Llama.cpp server:

context.subscriptions.push(
    vscode.commands.registerCommand('tutorial.stop', async () => {
        const pid = context.globalState.get("codesphere.runningPID");
        console.log(pid)

        const bashcommand = `kill ${pid as number + 1}`;

        const childProcess = exec(bashcommand, (error, stdout, stderr) => {
            if (error) {
                console.error(`exec error: ${error}`);
                return;
            }

            if (stderr) {
                console.error(`stderr: ${stderr}`);
                return;
            }

            console.log(`stdout: ${stdout}`);
        });

        context.globalState.update("codesphere.runningPID", "");
        vscode.commands.executeCommand('setContext', 'tutorial.modelIsRunning', false);
    })
);

We loaded the PID from our globalStorage and used it to terminate the correct process. And there we have it! Now we can start and stop our local Llama.cpp model as needed.

Tip: We actually have a webview developer console inside the extension development host to debug our extension. You can open it as follows:

Ctrl + Shift + P
select Developer: Open Webview Developer Tools

Developer Console inside VS Code

Place an input field to our webview

The user needs to place the prompt somewhere. Let's create the UI for that within our Svelte file. For that, we simply need HTML, CSS, and JavaScript for the functionality.

In our Svelte file, we have a <script> tag for JavaScript code and a <style> tag for CSS. In the rest of the file, you can place the HTML code. Let's keep it simple, and if you want, you can practice your UI/UX design by enhancing this code as you like.

Add the following two code snippets to src/webviews/components:

a) CSS

.prompt-container {
        display: flex;
        align-items: center;
        padding: 10px;
        width: 100%;
        max-width: 600px;
        margin: 0 auto;
        position: fixed;
        bottom: 0;
        left: 0;
    }

    .prompt-input {
        flex: 1;
        padding: 10px;
        border: none;
        outline: none;
        font-size: 16px;
    }

    .prompt-button {
        display: flex;
        justify-content: center;
        align-items: center;
        padding: 10px 20px;
        font-size: 16px;
        border: none;
        cursor: pointer;
        border-radius: 4px;
        margin-left: 10px;
        width: 50px;
        height: 50px;
        align-self: end;  
    }

b) HTML

<div class="prompt-container">
        <textarea type="text" placeholder="Ask your model something about your code" wrap="hard" class="prompt-input"></textarea>
        <button class="prompt-button">
            <svg xmlns="http://www.w3.org/2000/svg" width="100px" height="100px" viewBox="0 0 16 16">
                <path fill="currentColor" fill-rule="evenodd" d="m4.25 3l1.166-.624l8 5.333v1.248l-8 5.334l-1.166-.624zm1.5 1.401v7.864l5.898-3.932z" clip-rule="evenodd"/>
            </svg>
        </button>
    </div>

And this is the result:

Added input field

As you can see, we didn't add a color to our button, but it's blue. That's because in our project, we have some standard VS Code styles imported. You can find the styles in the media directory of our project.

Split the webview to two different sections

It's time to split up our webview since the HelloWorld template has nothing to do with our chat interface. We can have multiple webviews in our sidebar simultaneously, like an accordion, where you can open and close the different webviews as you like.

"views": {
      "tutorial-sidebar-view": [
        {
          "type": "webview",
          "id": "tutorial-sidebar",
          "name": "Tutoial",
          "icon": "media/tutorial.svg",
          "contextualTitle": "Tutorial"
        },
        {
          "type": "webview",
          "id": "tutorial-chat",
          "name": "Chat",
          "icon": "media/tutorial.svg",
          "contextualTitle": "Chat"
        }
      ]
    },

With that simple trick we created a second webview window inside our sidebar:

Second window in the sidebar

The content in our new chat window is loading infinitely because we haven't registered content for that window in our extension. Let's change that by adding code in src/extension.ts:

const chatProvider = new ChatProvider(context.extensionUri, context);
context.subscriptions.push(
  vscode.window.registerWebviewViewProvider(
  "tutorial-chat",
  chatProvider
  )
);

Now, when you refresh the extension development host, you will see that the content is loaded into the second webview. The reason this works without adding extra files is that we provided the necessary files in the template. Here is a list of the files you need to add to register this new webview:

ChatProvider.ts in /src
Chat.svelte in webviews/components
chat.ts in webview/pages

If you are about to create another webview and you are using our SidebarProvider.ts as a template for that, you need to change the compiled JavaScript and CSS files to the correct ones.

const styleResetUri = webview.asWebviewUri(
      vscode.Uri.joinPath(this._extensionUri, "media", "reset.css")
    );
    const scriptUri = webview.asWebviewUri(
      // change here
      vscode.Uri.joinPath(this._extensionUri, "out", "compiled/chat.js")
    );
    const styleMainUri = webview.asWebviewUri(
      // change here
      vscode.Uri.joinPath(this._extensionUri, "out", "compiled/chat.css")
    );
    const styleVSCodeUri = webview.asWebviewUri(
      vscode.Uri.joinPath(this._extensionUri, "media", "vscode.css")
    );

Add functionality to our chat-interface

Now we have our UI, but we need the logic for handling our prompts and displaying the response. In order to do that, we need to send our prompt to our model via the API and send the response to our webview so that we can display it. It follows the same pattern when implementing logic for our extension:

Send data from the webview to our extension via the postMessage method.
Handle the message within our extension's code (e.g., ChatProvider.ts) inside the switch-case block.
Send back data to our webview with the postMessage method.

First, we define a function to forward our prompt to the extension and add this prompt to our chat interface. You can replace the someMessage() function with the following code:

function someMessage() {
    // Clear the textarea
    document.getElementById('promptInput').value = '';

    // add textblock to chat interface
    const chat = document.createElement('div');
    let randomId = generateRandomId();
    chat.innerHTML = `<div style=" background-color: #00BCFF; 
                                    color: black; 
                                    padding: 10px; 
                                    margin: 10px; 
                                    border-radius: 10px; 
                                    display: inline-block;">${prompt}</div>`;
    document.getElementById('chatContainer').appendChild(chat);

    // Create response div
    const responseDiv = document.createElement('div');
    // styling
    responseDiv.style = "background-color: #6F40D3; color: black; padding: 10px; margin: 10px; border-radius: 10px; display: inline-block;";
    responseDiv.id = randomId;
    document.getElementById('chatContainer').appendChild(responseDiv);

    vscode.postMessage({
        type: 'prompt',
        value: {
            prompt: prompt,
            id: randomId
        }
    });
}

The postMessage() method will send a message to our extension, and the type property of the message body will serve as the identifier for which case in our switch-case code block will be executed within the ChatProvider.ts.

webviewView.webview.onDidReceiveMessage(async (data) => {
      switch (data.type) {
        case "prompt": {
          if (!data.value) {
            return;
          }

          let prompt = data.value.prompt;
          let id = data.value.id;

          const Test = async (prompt: string, id: any) => {
            try {
              let response = await fetch("http://127.0.0.1:8080/completion", {
                method: 'POST',
                headers: {
                  'Content-Type': 'application/json',
                },
                body: JSON.stringify({
                  prompt,
                  n_predict: 30,
                  stream: true,
                }),
              });

              if (!response.ok) {
                throw new Error(`HTTP error! status: ${response.status}`);
              }

              if (!response.body) {
                throw new Error('Response body is null');
              }

              const reader = response.body.getReader();
              const decoder = new TextDecoder();
              let result = '';

              while (true) {
                const { done, value } = await reader.read();
                if (done) {
                  break;
                }
                result += decoder.decode(value, { stream: true });

                const lines = result.split('\n');
                for (const line of lines) {
                  if (line.startsWith('data:')) {
                    try {
                      const json = JSON.parse(line.substring(5).trim());
                      console.log(json.content);
                      let token = json.content;
                      this._view?.webview.postMessage({
                        type: "response",
                        value: token,
                        id: id
                      });
                    } catch (e) {
                      console.error('Error parsing JSON:', e);
                    }
                  }
                }
                result = lines[lines.length - 1];
              }

              this._view?.webview.postMessage({
                type: "response",
                value: "done",
                id: id
              });
            } catch (error) {
              console.error('Error:', error);
            }
          };

          await Test(prompt, id);

          break;
        }
      }
    });

This will handle our request to our running LLM. We call it in stream mode, which means that every token is sent separately back. Every time we get a token, we send this token back to our extension via the postMessage method so that we can add it to our chat interface within the webview's code.

onMount(() => {
    window.addEventListener('message', event => {
        const message = event.data; // The JSON data our extension sent
        console.log(`message received: ${JSON.stringify(message)}`);
        switch (message.type) {
            case 'response':
                if (message.value === 'done') {
                    // Mark the end of the response
                    break;
                }
                let responseDiv = document.getElementById(message.id)
                responseDiv.innerHTML += message.value;

                break;
        }
    });
});

Now we can have a chat with our locally hosted LLM.

VS Code extension LLM chat

Publish an extension

When we decide that our extension is ready to be published, we can do so using the vsce CLI tool for managing VS Code extensions.

Fist we need to install the vsce CLI:

npm install -g @vscode/vsce

Packaging your project to a .vsix file is the easiest way to share your extension. You need to run this command in the root directory of your project:

vsce pack

However, since it's not always trustworthy to share files directly, we might want to publish our extension in the official VS Code Extension Marketplace. Since Microsoft's official documentation for publishing extensions is clear, we will just reference their documentation here: Publishing Extension.

Once you've completed everything and your extension is accepted, it will be listed and available for other users to install.

Further links

This is a list of links which were useful to us when building our own VS Code extension (will be updated when finding good reference in the ):

Youtube tutorial (Ben Awad): https://www.youtube.com/watch?v=a5DX5pQ9p5M&list=WL&index=1
Genral information: https://code.visualstudio.com/api
Commands: https://code.visualstudio.com/api/extension-guides/command
Webviews: https://code.visualstudio.com/api/extension-guides/webview
Build-in commands: https://code.visualstudio.com/api/references/commands
Activation events: https://code.visualstudio.com/api/references/activation-events
about package.json: https://code.visualstudio.com/api/references/extension-manifest
publishing extensions: https://code.visualstudio.com/api/working-with-extensions/publishing-extension

Conclusion

This tutorial serves as a comprehensive guide for creating a Visual Studio Code extension from scratch. We utilized the template provided to build an extension that interacts with a running Llama.cpp instance on localhost:8080. While this tutorial covers a lot, there are many more possibilities to explore when creating VS Code extensions, but covering all of them would be too much for one blog post.

If you're interested in building your own VS Code extension, you can join our Discord Server to connect with fellow software developers, Codesphere users, and the Codesphere team: Discord Server.

Furthermore, if you want to improve this tutorial extension, there are plenty of things you can do! It's a great project to practice your coding skills and your VS Code extension creation skills. Here are some ideas for improvement:

Persistent Chats : Implement a feature to save chat histories or conversations between sessions.
Creating a HuggingFace LLM Browser as a Separate Webview to Download Models : Develop a separate webview interface where users can browse and download models from HuggingFace.
Fine-Tune Prompt Settings or Create a UI for User Settings : Allow users to fine-tune prompt settings or provide a user-friendly UI for configuring settings related to the extension.
Improve UI/UX : Enhance the user interface and experience of the extension to make it more intuitive and visually appealing.
User-Friendly Installation of Llama.cpp : Simplify the installation process of Llama.cpp for users, possibly by automating or providing clear instructions within the extension.
Improve Model Performance Depending on User's Hardware : Implement optimizations to enhance the performance of the model, taking into account the hardware specifications of the user's machine.
Implement Error Handling : Develop mechanisms to handle errors gracefully, providing meaningful error messages and guiding users on how to resolve issues when they occur.
Cross-Platform Compatibility : Ensure that the extension functions smoothly on the Mac, Linux, and Windows operating systems.

Self-host AI interface builder OpenUI on Codesphere

Simon Pfeiffer — Mon, 08 Apr 2024 13:14:04 +0000

AI tools helping with any and all parts of software development have been popping up left and right. The promise replacing coding tasks that are typically both mundane and require a learning curve is very appealing.

With OpenUI you can create frontends in most common programming languages i.e. HTML, React, Svelte, Vue or web components with simple natural language prompts. The created draft is then refined by the AI by adding comments to elements just like you'd be collaborating with a design team in Figma. And the best? It works both with managed AI endpoints from OpenAI and self hosted models available via Ollama.

Step 1: Create workspace

If you haven't created an account navigate to codesphere.com and sign up. Next you want to create a workspace, if you plan on using a self hosted model via Ollama it needs to be equipped with one of our GPU plans. Otherwise a smaller plan should be sufficient.

Step 2: Install Ollama

If you want to use OpenAI instead of a self hosted model you can skip this step.

We will need to follow a slightly altered version of the official installation guide as we need to install Ollama in our local home/user/app directory. Open a terminal an type:

sudo curl -L https://ollama.com/download/ollama-linux-amd64 -o /home/user/app/ollama

Then provide execution rights to the downloaded file via:

sudo chmod +x /home/user/app/ollama

Step 3: Clone OpenUI Repository

Next get the source code for the UI interface builder.

git clone https://github.com/wandb/openui.git

This will create a new folder in your app directory with the frontend and backend code for the UI builder.

Step 4: Install required Python version and configure virtual environment env variables

Codesphere pre-installs Python but for this project we will need a newer version. Firstly we need to configure our pyenv - this will make sure our installed Python version is persistent across workspace restarts.

Navigate to setup/env vars and set PYENV_ROOT to /home/user/app/.pyenv
Grab your workspace dev domain via echo $WORKSPACE_DEV_DOMAIN and copy to clipboard
Set OPENUI_HOST to the value you just copied
Set OPENAI_API_KEY to xxx (for local Ollama models) or to your actual Open AI API key
Set OPENUI_ENVIRONMENT to local (please note for production use you'll want to set this to production and set up the GitHub SSO app for session handling)
Open a new terminal (existing terminals don't get updated env vars)
Type pipenv install --python 3.10.0 and confirm with Y - this will take a few minutes
Activate with pipenv shell

Verify that your Python version is 3.10 via python -V

Step 5: Install dependencies for OpenUI

First let's build the frontend artefacts. In your terminal navigate into the frontend directory with:

cd openui/frontend

Then run the installation and build.

npm install
npm run build -- --mode hosted

Now navigate to the backend directory (assuming your still in the frontend directory) with:

cd ../backend

Now install the Python dependencies via:

pip install .

Step 6: Adjust port and host

Currently OpenUI does not support changing the port via a command line flag or similar so we will need to do a small change to the source code.

Open the file /openui/backend/openui/ __main__.py and change line 55 to:

            host="0.0.0.0",

and line 57 to:

            port=3000,

Now save the changes with command+s.

Step 7: Download model & run Ollama server

Navigate to the root directory via:

cd /home/user/app

And launch the Ollama server via:

./ollama serve&./ollama run llava

This will download the model (takes a few seconds) and start a prompt shell, to verify it is actually working type a prompt and see if it will produce output. If it works exit the prompt context via control + d (mac) or strg + c on windows.

Step 8: Run the UI builder and enjoy!

Make sure your still inside your pipenv and then in the same! terminal now type:

cd /home/user/app/openui/backend
python -m openui

How to Get Started with Umami on Codesphere, Your Google Analytics Alternative

Simon Pfeiffer — Fri, 23 Feb 2024 14:43:41 +0000

In today's digital landscape, data privacy is paramount. With giants like Facebook and Google tracking users across the web using cookies, the need for privacy-centric solutions has never been greater. Umami stands out by prioritizing user privacy and avoiding the use of cookies altogether. By not using cookies, Umami ensures that it doesn't collect personal data, thus safeguarding the privacy of your audience. This commitment not only protects user information but also ensures that you maintain control over your data. By choosing Umami on Codesphere, you're not just opting for powerful analytics, but also actively safeguarding the privacy of your users.

Umami

Umami is a modern, open-source analytics solution offering advanced features and privacy-centric design. It provides a comprehensive understanding of user interactions with websites or applications, rivaling Google Analytics in functionality.

Users may prefer Umami for its self-hosted option, ensuring full control over data and compliance with privacy regulations, while still offering robust analytics capabilities. Its simplicity, flexibility, and focus on user privacy make Umami a compelling alternative to Google Analytics for those seeking more control over their data.

In this tutorial, we'll guide you through setting up your own Umami instance on Codesphere and conducting an A/B test.

What is an A/B test?

An A/B test, also known as a split test, is a method used to compare two or more versions of a webpage or app to determine which one performs better. In an A/B test, users are randomly divided into groups and shown different versions of the same webpage or app, with each version containing a specific variation or change. The performance of each version is then measured based on predefined metrics, such as click-through rates, conversion rates, or user engagement.

By analyzing the results of the test, you can determine which version is more effective in achieving your goals and make data-driven decisions to optimize your product or website.

Prerequisites

For following along this tutorial like blog post you need to have these things before diving in with us.

To deploy Umami

Codesphere Account
Database (MySQl, Postgres)

Note: If you haven't a Database connection yet you can get one in the marketplace tab in your Codesphere teams overview.

To conduct an A/B test

A domain
Two versions of a website to analyze
Deployed Umami instance on Codesphere

Deploy Umami on Codesphere

To set up Umami on Codesphere, follow these steps:

Open a new workspace on Codesphere and paste this GitHub link into the text field:

GitHub link: https://github.com/codesphere-community/umami

Create a new workspace on Codesphere

For the next step you need a MySQL or Postgres database which we can connect to our Umami instance. If you don't have a database yet, acquire one from the marketplace tab in the teams overview.

Create a new database in the marketplace

To initialize a new database may take several minutes. When your database is ready to be used you can click on Show info on the right side to get all relevant information:

Ready to use database on Codesphere

We need to create a new environmental variable in our Codesphere workspace, so that Umami can use the database. To do that switch to the set up tab on the left side of your workspace and navigate to Env vars:

Environmental variables UI

By clicking + Add new variable you can create a new environment variable.

Key: DATABASE_URL
Value: Your database URL

If you created your database on Codesphere, you find the database URL in the Services tab in your Codesphere team overview:

When clicking Show info the database URL is listed under Connection string. Just copy the string into the value field when creating a new environmental variable.

Creating a new environmental variable on Codesphere

Now you can start the 'Prepare' stage of the CI-Pipeline, and upon completion, start the 'Run' stage. You can navigate to the CI-Pipeline UI by clicking CI-Pipeline at the bottom of your workspace:

CI-Pipeline

Access your Umami instance by clicking the Open deployment button in the top-right corner of your workspace. When initializing Umami for the first time, you need to sign in with the following credential:

user: admin
password: umami

After signing in, we recommend changing the admin credentials immediately to ensure security. You can do this in the settings on Umami.

Umami account settings

Feel free to discover what you can change in the settings on Umami and set up your Umami instance according to your preferences.

After setting up Umami, you're ready to proceed with setting up an A/B test on Codesphere and utilizing Umami to analyze the results.

A/B testing with Umami on Codesphere

To set up an A/B test on Codesphere, you need a domain so that multiple workspaces can connect to it. If you haven't connected a custom domain yet, we have a tutorial on how to do this:

Link: https://codesphere.com/articles/custom-domains?ref=codesphere.ghost.io

To start setting up this A/B test, you need to create a workspace for each version of your website you want to test.

For this example I used the static landing page with Express.js and Tailwind.CSS Template: https://github.com/codesphere-cloud/landingpage-temp

After creating these workspaces, you need to connect them all to the same custom domain. This allows Codesphere to automatically handle traffic distribution, ensuring that traffic is evenly distributed among the different versions.

You can do this in the teams overview on Codesphere by navigating to the domains tab. All you need to do is select the workspaces that should be associated with this domain:

Connect a custom domain to several workspaces

Don't forget to save the changes by clicking the save button on the right side.

Now we want to add our website to Umami. To do that, we navigate to the settings once again and click on the blue button in the top right corner. You just need to fill out the modal with the name of your website and your custom domain, which you set up on Codesphere:

Add your Website to Umami

Now we can obtain the tracking code, which needs to be inserted into the <head> tag of each version of our website. You can find it by editing your website settings and navigating to the tracking code section, where you can copy the tracking code:

Tracking code

When pasted the tracking code into your html file in the <head> you need to save the file with Ctrl + S

The tracking code inside a <script> tag

With that being set up we can see the traffic data in our umami dashboard when navigating to Dashboard and then click on View details :

Umami Dashboard

Now, we want to conduct an A/B test on one of our buttons within our website. For that, we need to write a tracking event code in JavaScript, which sends data to Umami every time this specific button is clicked. This JavaScript code needs to be pasted into each of our version's HTML files:

<script>
    function trackButtonClick(buttonId) {
        umami.track('Button_Click', { button_id: buttonId });
    }
</script>

then we need to give the button on each of our versions a on_click attribute like this:

a) Version 1:

<button onclick="trackButtonClick('Version_A_Button')" type="submit" class="[...]">
  Notify me
</button>

b) Version 2:

<button onclick="trackButtonClick('Version_B_Button')" type="submit" class="[...]">
  Notify me
</button>

With that setup, Umami can distinguish between both buttons and count how often each of them is clicked. We can view the results in our Umami dashboard by navigating to Event Data and then clicking on our event, Button_Click.

Event data from the button

With that setup, we can track which version receives more clicks. You can view the results on Umami in your website's overview.

This is a simple example of A/B testing, and there are more use cases for A/B testing with Codesphere and Umami.

Conclusion

Umami is a powerful Google Analytics tool that's worth exploring further. It offers much more than the example we've shown in this blog post. Additional features of Umami include funnel analytics, drop-off rates, traffic sources, real-time data, and much more. Moreover, you have full control over your data.

If you're interested in trying Umami and need assistance, you can join our Discord community. There, you can connect with the Codesphere Team and fellow Codesphere users.

Discord: https://discord.gg/codesphere

AI-Enhanced Knowledge Hub: Improving Access to our Blog and Docs

Simon Pfeiffer — Fri, 26 Jan 2024 11:08:14 +0000

Over the years, we have amassed a significant amount of content about Codesphere and what is possible to do with it.

One issue with how our content has been structured so far has been the distinction between our blog and documentation as separate platforms.

In this article I will talk in retrospective about how we consolidated our documentation and our existing blog and combined it with an LLM-powered search using Qdrant (as described in my last article), to have a consolidated, natural language-enabled search across all our information.

Setting the scene

At Codesphere, we not only had rather traditional documentation, we also publish multiple blog articles per week which include tutorials that reach from simple deployment guides to more complex topics like building out an email marketing engine.

This left us with the problem that informational content about Codesphere, including our most sophisticated tutorials, was hosted outside of our documentation. Therefore it was not searchable from our docs page which would be where users go when they are looking for information on what to do with and how to use Codesphere.

That is why we set out to find a way to consolidate our docs and blog.

Our existing setup

Our existing setup mainly consisted of three different pillars to host our information:

Gitbook: We stored all out docs in Gitbook, a standard docs application used by many tech startups around the world.
Ghost.io: As we_'_ve mentioned many times before, we use Ghost.io as the content management system for our blog.
Custom Blog Frontend: Our blog is a fully custom static site generator which gives us full flexibility on how to set it up which is what we profited from for this project.

While Gitbook comes with some quality of life improvement like an automatic search and a clean UI, it wasn_'_t really possible to connect it with our blog to create a consolidated search which is why we set out to create our own.

How we made it happen

Before we get into the different steps, let me quickly mention what should already be a given: Everything we did was implemented in Codesphere or through services available through the Codesphere marketplace.

Data migration

It became clear to us pretty early on that we'd need to say goodbye to Gitbook to realize what we had in mind.

That is why we implemented all our existing docs to Ghost.io and retrofitted them with fitting tags that we could use for filtering in our frontend.

This way, we were now able to pull our docs and blog posts separately from Ghost and save them in the Node environment where our consolidated hub would run.

Implementing Search Capabilities

We used the same technique I explained in my last blog article to create an AI-powered search using Qdrant. Using Xenova/all-MiniLM-L6-v2 as the transformer for our embeddings. We are hosting the Qdrant instance in a custom Docker image, set up through the Codesphere Marketplace.

This now allows for us to search both our blog and docs simultaneously, using natural language or more traditional queries alike, enabling high quality semantic search that goes beyond just simple keywords.

Adjusting the frontend

Of course, all those changes would need to be reflected in our frontend. I already had some design improvements for the blog article page brewing in the background and this was the perfect opportunity to let them come to light.

One big addition we made was implementing a similar navigation to Gitbook.

We were able to also implement this in a static way, keeping our blog fast and snappy.

The last thing to add was the search for our frontend. For that, we simply added a prominent search bar to the top or our page. We might add this to the navbar to make it even more prominent in the future.

Conclusion

The transition came with its own challenges and tedious steps but at the end of the day, it was a lot simpler than we (or at least I) initially expected.

We now have a new and state of the art way for our users to search all of the content we have ever published to get them to their goal quicker.

Now that we have set up this database, this opens up new possibilities for us as we could use it to power an LLM-powered Codesphere assistant or provide an even more tailored content experience.

The possibilities are endless and we can't wait to keep on exploring what's possible.

If you would like to set up something similar and don't know where to start, feel free to join our Discord community and ask for further assistance.

Cloud Deployments & Live Code Sync from Visual Studio Code

Simon Pfeiffer — Fri, 22 Dec 2023 14:22:44 +0000

Codesphere has its own cloud based code editor. It is optimized for speed and works as a simple collaborative text editor for coding. While we are constantly expanding its functionality it is of course not as powerful as the leading local IDEs out there. However in this blog post we are going to show you a way to connect your local Visual Studio Code (or the browser version thereof) to any Codesphere workspace in 5 minutes.

Picture this: using your local Visual Studio Code to seamlessly sync with Codesphere Workspaces bidirectionally, giving you the full power of Visual Studio Code and the deployment & compute in Codesphere. You can leverage the full compute power of your workspace even for local development. The days where local development requires expensive hardware will soon be in the past, what we are showing today works on any device, as long as you have internet and a browser. It even works from an iPad or your phone.

You can fully utilise the collaborative possibilities, co-create with other developers or share sessions with project managers and other stakeholders. 'Works on my machine' is no longer a valid excuse. Ready to dive in?

How to set it up (takes 5 min or less)

Open any existing Codesphere workspace
Open a terminal and enter the following commands

curl -Lk 'https://code.visualstudio.com/sha/download?build=stable&os=cli-alpine-x64' --output vscode_cli.tar.gz

tar -xf vscode_cli.tar.gz

./code tunnel

Follow the instructions in the terminal, you can login via GitHub or a Microsoft account
We select GitHub and navigate to https://github.com/login/device, enter the code provided in the terminal and submit
Back in the terminal, provide a name for this tunnel and open the link provided in any browser https://vscode.dev/tunnel/<machine_name>/<folder_name>
Alternatively you can open your local VS Code, install the Remote - Tunnels extension from Microsoft and follow the instructions to connect your tunnel

And that's it. If you now open a terminal in your Visual Studio instance you can run all commands like you would from Codesphere. Files changed either on Codesphere or in VS Code are bidirectionally synced in almost real-time.

Why is that amazing?

Access to higher performance e.g. GPU's from a limited machine
Share it with PMs, Stakeholders, and Tech Leads immediately (it's live and deployed)
Access the whole landscape from your laptop (e.g. test databases, other services)
Use your local extensions to modify a preview deployment
Use this in meetings as a replacement for screen sharing, both can edit live, run, and test the code
'Works on my machine' is a thing of the past
No code needs to be stored on your laptop -> more secure
No more installing of the development environment, it is already set up and is the same as previews, staging, and production
Just connect to any deployment from GitHub and fix it if it did not work 😄

This is also different to other CDE platforms like Codesandbox and Gitpod, because Codesphere covers the whole software lifecycle from preview to production, with CI Pipelines, Replicas, Autoscaling, Microservice Support, Managed Services, GPU's, etc.

Try it yourself

Happy coding!

GrapesJS - Self Hosted Low-Code Webpage Builder on Codesphere

Simon Pfeiffer — Wed, 20 Dec 2023 17:15:41 +0000

Working with Low Code tools can save a lot of development time, allowing you to allocate resources more efficiently elsewhere. GrapesJS is one such Low-Code tool that enables you to easily build your webpage by dragging and dropping elements.

Hosting GrapesJS on Codesphere is straightforward and can be done within a few minutes. In this blog article, we will collaboratively create a GrapesJS instance. Additionally, we will demonstrate how to deploy your generated page effortlessly on Codesphere.

Deploy GrapesJS on Codesphere

We've created a GitHub repository that serves as the foundation for deploying GrapesJS on Codesphere.

To get started, simply create a new workspace within one of our Codesphere teams. A free plan workspace is sufficient. When setting up the workspace, paste this GitHub link into the designated field:

https://github.com/codesphere-cloud/deploy_grapesjs

The GitHub repository includes a CI pipeline that contains everything necessary to deploy GrapesJS.

To begin, execute the 'Prepare' stage of the CI pipeline to install all dependencies. This process may take up to two minutes. Once the 'Prepare' stage is complete, you can directly execute the 'Run' stage, and within a second, our GrapesJS will be deployed.

In this project, GrapesJS is deployed using an Express server. Additionally, we have set up the option within that Express server to deploy the HTML code created in GrapesJS directly from this Codesphere workspace.

Here is our documentation about creating a workspace with a GitHub repository:

Build Your Webpage with GrapesJS

Once GrapesJS is deployed, you'll see an overview on the main route. This overview has been provisionally set up by us and is not an integral part of GrapesJS.

To use the GrapesJS Webpage builder, press the respective 'GrapesJS - Low Code Webpage builder' button on this homepage. From here, you enter the GrapesJS tool.

GrapesJS UI

Initially, a demo webpage from GrapesJS is displayed, illustrating what can be achieved with the 'webpage preset.' To clear everything, use the 'clear canvas' button in the top-right corner of the user interface.

On the right side, there's a section where you can choose from various 'blocks' to use on your webpage. As an example, let's drag and drop the 2 Columns block into the white canvas, and you'll see an element with two columns added at the respective position.

Clicking on the element brings up a menu on the right side, allowing you to make various settings. For instance, you can set the padding of the box here.

Example in the menu for setting the padding

Now, let's fill the boxes with text by inserting the text block into one of the two columns. Next, we'll integrate a Youtube video into our webpage using the 'video block'. Here you need to paste this Video-ID in the respective input field in the settings: KJwYBJMSbPI

Example page in GrapesJS

Now, we've created a admittedly very simple webpage. We can retrieve the HTML code by using the import button. From the pop-up window, we can copy our HTML code.

Generated HTML

The HTML code can be employed in various ways. In this sample project, we've configured the Express server to deploy the HTML code when it's inserted as an .html file into the my_webpages folder. It becomes accessible through the main route in the overview.

Deploy Your Webpage on Codesphere

Now, let's deploy the webpage we just created. Navigate to the 'grapesjs' folder in your IDE, then go to the 'my_webpages' folder. Right-click, choose 'New File,' and give the file any name with the extension .html (e.g., Example.html).

Next, copy the HTML code and paste it into the empty .html file:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>My-Webpage</title>
</head>

then we copy the generated HTML code out of GrapesJS and paste it below the head tag:

<body>
  <div id="iynq" class="gjs-row">
    <div id="i21h" class="gjs-cell">
      <div id="inhh">Hello World!
      </div>
    </div>
    <div id="i6qj" class="gjs-cell">
      <iframe allowfullscreen="allowfullscreen" id="il92" src="https://www.youtube-nocookie.com/embed/KJwYBJMSbPI?"></iframe>
    </div>
  </div>
</body>
<style>* {
  box-sizing: border-box;
  }
  body {
    margin: 0;
  }
  .gjs-row{
    display:flex;
    justify-content:flex-start;
    align-items:stretch;
    flex-wrap:nowrap;
    padding:10px;
  }
  .gjs-cell{
    min-height:75px;
    flex-grow:1;
    flex-basis:100%;
  }
  #iynq{
    display:flex;
    width:auto;
    padding:100px 100px 100px 10px;
  }
  #inhh{
    padding:25px 10px 25px 10px;
    width:auto;
    font-family:Comic Sans MS, cursive, sans-serif;
    font-size:109px;
  }
  #il92{
    height:350px;
    width:615px;
  }
  @media (max-width: 768px){
    .gjs-row{
      flex-wrap:wrap;
    }
  }
</style>

Now, save the file by pressing Ctrl + S. Next, revisit your workspace URL by, for example, clicking the 'open deployment' button in the Codesphere IDE's top right corner.

Your created HTML file will now appear in the overview, and you can preview it by clicking the respective button.

Finished webpage

Customization and Beyond

In this blog article, we utilized GrapesJS's Webpage preset. However, what makes GrapesJS interesting is its high level of customization. Everything, including the user interface and the individual blocks you can use, is adjustable in the source code. Moreover, there are numerous plugins that you can integrate into GrapesJS, allowing you to tailor it to your specific needs. So actually GrapesJS is not only a low code webpage builder, it is also a framework to build your own low code webpage builder.

Here is the link to the official documentation: https://grapesjs.com/docs/

In this documentation, you can also find information on how to integrate plugins.

Discord Community Server

If you interested in connecting with fellow Codesphere users and developers you can join our Discord Community Server: https://discord.gg/codesphere

n8n - a Cost-Saving Zapier Alternative Hosted on Codesphere

Simon Pfeiffer — Tue, 21 Nov 2023 17:57:02 +0000

In the world of automating tasks, Zapier has been a go-to for many. But here's the thing – if you're looking for more control and a wallet-friendly option, consider n8n the open-source workflow automation tool.

The cool thing is that you can host it yourself, and if you do it on Codesphere, you'll save money compared to use Zapier. Plus, you get full control over your data and how your workflows play out.

In this article, we're going to show you just how simple it is to set up your own n8n instance on Codesphere. We will guide you through the process of getting started effortlessly. Let's make the most of n8n's potential with the ease of self hosting on Codesphere – it's a win-win.

Set up the n8n workspace

Let's get your n8n workspace up and running on Codesphere. You just need to follow these steps:

Choose Your Plan:

Pick either "Always On" or "Off When Unused" for your workspace. If you choose "Off When Unused," remember it shuts down after an hour of inactivity. Don't worry, though—your workspace will reboot the next time you use it, although it might take a bit.

Add this GitHub Repository:

link: https://github.com/codesphere-cloud/n8n-Template

Run "Prepare" and "Run" stages in the CI-Pipeline:
Open Deployment and Set Up the user account for this n8n instance

Create a workspace for hosting n8n on Codesphere

Your n8n workspace is set! Now, dive in and start creating your workflows. The n8n interface is user-friendly, making it easy to build workflows tailored to your tasks.

Next up, we'll guide you through setting up the workflow integrations: Discord, Todoist and Rowbase.

Loading the predefined workflow

Now that your n8n workspace is up and running on Codesphere, let's dive into effortlessly loading a predefined workflow. Follow these simple steps:

Download the Workflow JSON:

In the Codesphere IDE, right-click on the provided JSON file for the predefined workflow. Select "Download" to save it directly to your device.

Import the Workflow:

Inside your n8n workspace, navigate to the workflows section. Look for the "Import" option and upload the downloaded JSON file. This action will seamlessly import the predefined workflow into your n8n instance.

Save Changes

Import the predefined workflow into n8n

Next up, we'll guide you through creating a workflow. This one uses RSS feeds to share the latest news on Discord, organizes article data in a table, and sets up a task in a to-do list. This workflow is a hands-on example to demonstrate n8n's flexibility and power.

Prerequisites

Once you've completed setting up the n8n instance on Codesphere, ensure you have these accounts ready before diving into the showcase workflow:

Prerequisites for the Showcase:

Baserow Account: Create an account on Baserow and create a new database.
Todoist Account: Make sure you have a Todoist account, as it's part of our workflow.
Discord Account: If you don't have one yet, sign up for a Discord account and create a new server

Let's get started with setting up the workflow integrations and see what n8n has to offer!

Setting Up Workflow Integrations: Discord

Navigate to your Discord server settings and select "Integrations" from the menu. Create a new webhook and copy its URL. Paste this URL into the n8n webhook URL field. In the n8n content field, you can define the message the webhook should generate based on the RSS-feed data.

Create a Discord webhook

Setting Up Workflow Integrations: Todoist

In the n8n UI, create a new credential for Todoist. Copy the API token from Todoist's settings in the "Integrations" tap and select "Developers" to obtain the API token. Paste the API token into the password field of the n8n credential.

Setting up Todoist integration

Setting Up Workflow Integrations: Rowbase

For this integration, you need to set up a Rowbase credential. Simply click on 'Create New Credential,' and enter the email address associated with your Rowbase account in the username field, along with your Rowbase account password in the respective fields.

Once this step is completed, proceed to configure the additional required fields in the Rowbase node to populate the data in the desired table. To achieve this, you'll need to extract the Database ID from the API Documentation of your Rowbase Database. Copy and paste the Database ID into n8n.

To load the database into the node, exit the node menu and re-enter it. Now you can choose the table where you want to input the data. Once again, exit the node menu and re-enter it to select the specific columns for your article data.

Setting up Rowbase integration

Once completing these steps, you can save your workflow and activate it. This ensures seamless communication between n8n, Discord, Todoist, and Rowbase, allowing you to automate tasks effortlessly.

Adding a Custom RSS-Feed Trigger

To integrate any RSS feed of your choice into the workflow, follow these steps to create a new RSS-feed trigger:

Create a New RSS-Feed Trigger:

Within the n8n workspace, add a new RSS-feed trigger node to your workflow.

Copy RSS-Feed URL:

Obtain the URL of the desired RSS feed that you want to integrate.

Configure the RSS-Feed Trigger:

Paste the copied RSS-feed URL into the URL field of the RSS-feed trigger node. This ensures that the trigger is connected to the specific RSS feed you want to monitor.

Connect Trigger to Workflow Nodes:

Link the RSS-feed trigger to each subsequent node in your workflow. This connection ensures that the workflow responds to updates from the specified RSS feed.

Create a new RSS-Feed trigger for any RSS-Feed you desire

Link: Top 100 RSS-Feeds about IT

Conclusion

With these simple steps, you've now extended the flexibility of your workflow to incorporate updates from any RSS feed of your choice. The dynamic nature of n8n allows you to adapt and customize your automation seamlessly. Your workflow is now ready to efficiently process and act upon the latest information from the "subscribed" RSS feed.

With n8n's incredible flexibility, the options for automation are limitless. This powerful tool offers a variety of features, leaving a wide array of automation possibilities ready for you to explore.

Are you interested to learn more about Codesphere and n8n? Connect with fellow Codesphere users and reach out to our support team by joining our Discord server!

Join the Codesphere Community Server on Discord

Getting started with Stable Diffusion Text2Img AI

Simon Pfeiffer — Mon, 13 Nov 2023 15:11:54 +0000

Stable diffusion has become the staple of open source image generation AI. Similar to Llama anyone can use and work with the stable diffusion code. Tons of other open source projects build on top of it.

Generally you can use stable diffusion & related models to either generate images from prompts or edit images with prompts (text2img or img2img).

Running stable diffusion

The most comfortable way to work with stable diffusion is via the open source webUI https://github.com/AUTOMATIC1111/stable-diffusion-webui. Unlike other alternatives that require deeper understanding of the underlying code and working with command line prompts - this webUI comes with a 1-click installer and a web view with all the configuration options you might need. You can run it locally (if you have a strong graphics card & CPU) or run it via a cloud provider that offers GPU environments. If you’re looking for free cloud GPU's check out our comparison article here https://codesphere.com/articles/5-best-free-cloud-gpu-providers-for-hobbyists

If you are using Codesphere to deploy your stable diffusion webUI you can use our template.

After running the installation (which takes a few minutes) and your run pipeline you should see a WebUI similar to this one.

How to write good text to image prompts

This is the typical way of working with generative image AI. You provide a prompt describing visually what you expect to see in the image and the AI takes the input, tries to find semantically similar images from its training data and merges them into an image.

As you can see there is a wide array of settings available in the webUI and after playing around with it for a bit you will see that results differ dramatically.

Generating usable images from this requires mainly two things:

A good prompt with relevant context
Settings appropriate for your use case (there is a trade-off between generation speed & quality - more below)

Let's look at some example prompts first:

" Romantic painting of a ship sailing in a stormy sea, with dramatic lighting and powerful waves."

This prompt is a good start because it visually describes key features, i.e. what style of image is expected (romantic painting), the main object (ship) and the surrounding with enough detail. Finding the right amount of details can be a bit tricky and might take some practice - in general if you describe clearly and unambiguously what you'd like to see as concise as possible you'll get decent results.

Unfortunately the picture output doesn't have much details yet:

"best quality, masterpiece, (photorealistic:1.4), ship in stormy seas, (powerful waves:1.2), dramatic lighting, hyper quality, intricate detail, ultra realistic, maximum detail, foreground focus, instagram, 8k, volumetric light, cinematic, octane render, uplight, no blur, 8k"

Can we still improve the output? Yes! In order to understand what changed between the first prompt and the second prompt it is helpful to think about how stable diffusion and similar models are trained:

They get millions of labeled images of differing quality, assign patterns & similarity through arranging vector representations of labels and images closer in proximity of each other if similar. Now we can trick the model to only look on a subset of the pictures it was trained on, namely those we associate with beauty. By adding comma separated attributes associated with high quality input (i.e. masterpiece or cinematic) we can improve the output significantly. You can also add weights (i.e. photorealistic:1.4) if you want to prioritize between different attributes.

& negative prompt:

"nsfw, ng_deepnegative_v1_75t,badhandv4, (worst quality:2), (low quality:2), (normal quality:2), lowres,watermark, monochrome"

Another great way to improve an already excellent output image is by specifying attributes and output visual you want to avoid. Your frontend provides a second box for this. By adding quality descriptions like low and normal to the negative prompt we can enforce the ultra high quality even stronger. By adding thing like monochrome & watermark we can enforce more variation in our output.

Text 2 image settings

As mentioned there is a ton of different settings so we will focus on the important ones here:

Sampling method

The default is DPM++ 2M Karras which is a great mix between speed & quality. If you have time and want a bit more detail try switching to DPM++ SDE Karras

Other choices include some that are a lot faster like UniPC at the expense of detail

Hires. fix

This handles the upscaling of your output images. Per default images are 512x512 pixels which is the size of the training images used. You can upscale your images to higher resolution at the cost of speed. I recommend a value between 1.5 and 1.7 anything higher will get critically slow and might even crash the GPU with a out of memory error.

Refiner

This allows you to add a second model that takes the output of the first model and refines it.

Alright that's a wrap for today! - Have fun creating stunning visuals with this fascinating new technology

Getting started with Jupyter Notebooks in AI

Simon Pfeiffer — Thu, 26 Oct 2023 13:13:45 +0000

Jupyter notebooks are important for machine learning as they allow testing and experimenting with data and models. This blog post will teach you how to create your first Jupyter notebook in Codesphere. You will also get to try out some cool AI examples. One example is using a stable diffusion AI image generator in a notebook. Our GPU notebooks offer a similar but more customizable experience as Google Colab spaces.

Jupyter notebooks are great because they have a user interface where you can add cells to run code. You can use different programming languages like Python, R, and Julia. There are many other options available thanks to the open source community. It's about time that we added support for it to Codesphere and explored some use cases, so let's dive right in.

Set up your first Jupyter Notebook in Codesphere

First you'll want to create a new empty workspace (or clone a repo with your existing notebooks of course). Since we will be using Python inside the notebook (which requires Python 3.9 or higher) we will need to update Python first.

[Workaround] If you want your python version to persist across workspace restarts currently you are required to set the following environment variable (via setup/env vars)

Key:

PYENV_ROOT

Value:

/home/user/app/.pyenv

This will ensure the python version selection gets stored in the persistent app directory. Afterwards open a new terminal (Do not use existing terminals as they are missing the environment variable you just set) - in this new terminal type

pipenv install jupyter --python 3.9.11

When prompted to confirm whether you want to install the new Python version confirm with Y. Wait till all the dependencies were added successfully - this might take a minute.

Afterwards either activate your pipenv via pipenv shell followed by a jupyter notebook or run

pipenv run juptyer notebook

Now you can open the deployment with the icon in the top right corner. Per default Notebooks are secured by a password / token mechanism - this ensure's that only you and your team have access to the notebook server.

That means also you will need to authenticate yourself as the owner of this notebook server with a token - your specific token will be printed in the terminal output of the last command you ran. Copy the token and insert it into the webUI of your notebook server.

After providing your token you should be greeted by the Notebook Server UI that contains all the files in your directory. If you started with an empty workspace you should only find the pipenv files here. Right click anywhere in the UI and select new notebook. This will open a new tab in your browser with an untitled and empty jupyter notebook. You still need to select the kernel, for this simple example we'll go with the default iPython kernel.

Notebook basics (skip ahead to use cases if you are familiar with notebooks already)

Notebooks are organized in cells and there are 3 types of cells:

Code cells
Markdown cells
Raw text cells

Code cells contain your python code or terminal commands like pip install pandas and can be executed one by one. You can run a single cell multiple times and if the code you ran contains terminal output the notebook frontend will display the output below the cell. Likewise if your code doesn't run through because of unhandled errors it will display these below the respective cell.

Markdown cells on the other hand contain markdown formatted text. This is great for creating interactive documents that allow others (or yourself in the future) to follow along and understand why you built the code blocks the way you did.

Try adding some Python code to the first cell and click the play icon in the menu.

print("Hello this code was excecuted in a jupyter cell")

You should see whatever text you put inside the "" of the print statement below the cell.

If you want to use a markdown cell instead, select the cell and switch the from code to markdown. If you enter valid markdown notion and run the cell you will get formatted text in return.

Use case - Running your own open-source DALL-E alternative

For this example we will run a Text to image AI. It requires a notebook on one of our GPU plans - free BETA available through special signup link: https://ai.codesphere.com/

We will be using this performance optimized version of stable diffusion https://huggingface.co/segmind/SSD-1B because that is small enough to be run on our free tier GPU. Deploying and running inference of the Segmin SSD 1B model is actually really easy in Codesphere.

In your notebook you need to add & execute the following code cells:

pip install git+https://github.com/huggingface/diffusers

pip install transformers accelerate safetensors

from diffusers import StableDiffusionXLPipeline
import torch
pipe = StableDiffusionXLPipeline.from_pretrained("segmind/SSD-1B", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
pipe.to("cuda")
# if using torch < 2.0
# pipe.enable_xformers_memory_efficient_attention()
prompt = "A glamorous ski town in the Swiss alps in the 80s. A retro fotograph style picture with street lights, fancy cars and snowy mountains in the background and fancy dressed people enjoying a hot drink outside." # Your prompt here
neg_prompt = "ugly, blurry, poor quality" # Negative prompt here
image = pipe(prompt=prompt, negative_prompt=neg_prompt).images[0]

image

Prepare to be amazed! The output is a beautiful and stunning scene it feels as if we where there in 1980s Zermatt 💜 You can hold shift+right click on the image to copy or save it from the notebook to your laptop.

And just for the fun of it, you don't even have to come up with the detailed prompts yourself - ask chatGPT or your trusted self hosted Llama to to help with the prompts:

"I need your help to write prompts for a stable diffusion text to image generator. The goal is to generate a series of images that are detail rich, have a cosy atmosphere and a retro look.

Can you describe this "A glamorous ski town in the Swiss alps in the 80s. A retro fotograph style picture with street lights, fancy cars and snowy mountains in the background and fancy dressed people enjoying a hot drink outside." from multiple angles and perspectives so that I can plug each prompt into the image generator?"

Off when unused deployment mode now available

Simon Pfeiffer — Thu, 24 Aug 2023 08:16:28 +0000

This week we released another very exciting milestone. When creating a new workspace in Codesphere, you now have the option to chose between two different deployment modes. The default for paid plans is always on - until now all our paid plans are deployed like this, with dedicated 24/7 resources from our own datacenter.

The new deployment mode is called off when unused or sometimes referred to as standby or on demand workspaces. Workspaces deployed with this setting receive dedicated resources only when used and go into standby after ~60 minutes without usage. Upon receiving a request at the connected domain the server resources will boot up and restart your application (as long as there is a valid CI pipeline defined). Through under-provisioning, intelligent scheduling, and different deployment algorithms our servers have much faster cold starts (~1s) than those of competitors. Since these deployments only consume resources as they are used we can offer them at 10% of the cost of an always on deployment.

This allows teams to spam deployments more freely as cloud spend is only a fraction of what it would otherwise be. Imagine having a larger dev team, like at Codesphere for example, we have ~600 branches and ~250 pull requests open in parallel on a given day. Each of them get's deployed automatically as preview deployments - this has made us roughly 6x faster but the cost can get considerable fast. We estimate that all things considered, our on demand deployments can make preview deployments 70% cheaper than with conventional setups.

You can read more technical details in the corresponding doc page. In this article we want to explore some of the exciting use cases and advantages this opens up.

Use cases

Obviously you don't want a multi second delay for high traffic production setups but there are plenty of cases where this is totally acceptable.

The obvious example are preview deployments (read more about these here: https://docs.codesphere.com/getting-started/preview-deployments). These are used during the software development process to automatically deploy every committed code change and making a working version available to all team members. Developers can test things that require infrastructure not available locally easier, code reviews can be done much faster and you can even share pre-release versions of your software in client demos. Codesphere allows automating preview deployments from GitHub, Bitbucket & Gitlab (coming soon!) via our CI integrations.

Another use case is spinning up sandbox environments where you might still need ample computing power. For example we have been experimenting with self-hosted Llama2 LLMs. These are very resource intense applications but you might not be willing to spend big time just for trying things out. For example a smaller 7b param Llama2 runs fine on our Pro plan as we showed in another recent article (find it here: https://codesphere.com/articles/self-hosted-chatgpt-in-codesphere). With the new off when unused deployment mode you can try this yourself for as little as $8/m.

Slightly less obvious but equally neat is the benefit this update provides to all of our free tier users. The free plan workspaces were always deployed with an off when unused logic but now they benefit from the same wakeup on domain access logic as all paid workspaces deployed in this mode. Imagine you are hosting a portfolio site where you showcase your amazing dev projects - typically these sites will not be high traffic (unless you're rockstar in your field which justifies an always on plan I'd say 😎). You can now host sites like this free forever - whenever someone wants to take a look at it, it will take a few seconds to boot up but will be available momentarily.

Happy Coding from the entire Codesphere team.

From first click to prompt output in 1m38s - Running Llama2 in Codesphere

Simon Pfeiffer — Fri, 18 Aug 2023 09:18:49 +0000

What started out of pure curiosity: how difficult can it be to run your own LLM inside a Codesphere workspace, will it even run and if so will it be performant enough, running inference on our CPUs? Turned into one of the most exciting projects I've worked on in recent weeks.

Unlike OpenAi and their GPT models, Meta has open sourced their entire suite of Large Language Models called Llama1 & Llama2 alongside the pre-trained chat versions. In independent performance tests these land somewhere between GPT3.5 and GPT4 and they are actually a bit faster than GPT4 in their responses.

These models come in different sizes, based on how many parameters they were trained on. Typically inference for LLM's is run on GPU instead CPU processors because these computation are very memory intense and GPU's have a clear edge there. GPU servers are still very expensive and not as widely accessible as CPU servers - Codesphere is planning to offer shared (cheaper) & dedicated GPU plans in the near future but as of today we only offer them after receiving a pre-order requesting early access.

Therefore today we are going to test if we can still run the smaller model (trained on 7 Billion parameters) on a CPU based server inside of Codesphere. Since we know it will challenging to get a smooth response, we are going with our pro plan, providing 8 state of the art vCPUs, 16GB RAM and a 100GB of storage.

Getting Llama2 running on Codesphere is actually very easy thanks to the amazing open source community, providing C++ based wrappers (llama.cpp) and huggingface offering pre-compiled and compressed model versions for download.

It is actually so easy that I decided to do a timed run. From the first click to the first chat response took 1 minute 38 seconds. It really blows my mind.

Let's take a look how it's done inside of Codesphere. If you still need to create a Codesphere account now is as good a time as any. If your machine is strong enough this tutorial will also work locally (at least for Linux & MacOS with small adjustments)

Step 1: Create a workspace from the Llama.cpp repository

Sign in to your Codesphere account, click the create new workspace button at the top right and paste this into the repo search bar at the top:

https://github.com/ggerganov/llama.cpp

Next you'll want to provide a workspace name, select the pro plan and hit the start coding button. This plan is 80$/m for a production always on plan and 8$/m for a standby when unused deployment mode. We know 80$/m seems like a lot but consider that renting a GPU is typically more than 1000$/m.

Step 2: Compile the code

Open up a terminal and type:

make

This command will compile the c++ code to be readable for Linux. The Llama.cpp repository contains a Makerfile that tells the compiler what to do.

Step 3: Download the model

First type cd models in the terminal to navigate to the folder where Llama.cpp expects to find the model binaries. There are a wide variety of versions available via hugging face, as mentioned we are picking the 7b params size and opt for the pre-trained chat version of that.

Even for this specification there are ~10 different flavours available, pick the one that suits your use case best - we found this repo to contain good explanations alongside models: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML

In the models directory run this command and replace the model name with the flavour that suits you best:

wget https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q4_K_M.bin

Step 4: Run your first query

Now we can ask our very own chatbot the first question. Navigate back to the main directory with cd .. and then run this command from the terminal:

make -j && ./main -m ./models/llama-2-7b-chat.ggmlv3.q4_K_M.bin -p "Why are GPUs faster for running inference of LLMs" -n 512

It's going to take a few seconds to load the 4GB model into memory but then you should start to see your Chatbot typing an answer to your query into the terminal.

Once completed it will also print the timings, the initial load can take up to 30s but subsequent runs take less than 1s to start providing a response - also the speed is not quite as fast as interacting with chatGPT in the browser but it still returns around 4 words per second which is pretty good.

The images show the timing of the initial run vs. subsequent runs.

[Optional] Step 5: Getting the example chatbot web interface running on Codesphere

The llama.cpp repository comes with simple web interface example. This provides an experience closer to what you might be used to from ChatGPT.

Navigate to CI pipeline and click the Define CI Pipeline button. For the prepare stage enter make as command.

And for the run stage enter this command, making sure the model name point to the binary of the version you downloaded:

./server -m ./models/llama-2-7b-chat.ggmlv3.q4_K_M.bin -c 2048 --port 3000 --host 0.0.0.0

We need to set the port to 3000 and the host to 0.0.0.0 in order to expose the frontend in Codesphere.

Now you can run your prepare stage (which won't do anything if you previously ran make already via the terminal) but might be needed after workspace restarts.

Next run your run stage and click the open deployment icon in the top right corner. Now you and anyone you share the URL with can have chats with your self-hosted ChatGPT clone 😎

Let us know what you think about this! Also feel free to reach out to us if you are interested in getting early access to our GPU plans.

Happy Coding!

Back to the roots: advantages to static web development

Simon Pfeiffer — Tue, 08 Aug 2023 13:34:08 +0000

As web developers page speed and performance considerations are at the top of mind. They affect all important KPIs, starting with user experience and ultimately affecting how well your content performs on search platforms.

Today we are going to be talking about advantages of static websites, consisting of simple HTML, CSS and a Javascript server that serves this static content. That doesn't sound very sexy at first but bear with us for a second.

The fastest growing web development framework in recent years is arguably Next.js. The popularity is caused by a combination of easy to use (heads off to the people at Vercel!) and the fact that Next.js managed to solve some key concerns of serverside rendering frameworks like React. Next.js achieves this by pairing react with an intricate logic for serving static props - and on top it runs well on Serverless infrastructure, making infrastructure setup a no brainer (at least for simple use cases).

Now under the hood Next.js is still a react based framework and as such it’s the server and setup are not exactly lightweight. Don't get me wrong, it is very impressive how much faster Next.js can be compared to a standard React in many cases - but we want to argue today that for smaller / mostly static use cases, Next.js might be overkill and can easily be outperformed by a more simple setup: Static HTML, CSS & Express.js server. We call this back to the roots, because all mentioned web frameworks rely on these and build on top of them. At Codesphere we have recently been turning to this simple setup for a lot of use cases - originally not because of speed considerations but because stripping away all unneeded complexity it made us much faster in the marketing team.

When concerned with speed few things can beat a simple express.js server serving static html content. The main issue here is that most use cases are not simply serving static content. If your use case is purely static i.e. a landing page you should probably look no further. We have recently switched all our landing pages to this approach and can highly recommend it! It even makes things like a/b testing soo much easier 😎

If your use case is not as simple as serving a static landing page it might still be worth considering if it can't be turned into a static website with a few tricks.

Let’s take a look at an obvious example - a blog frontend is not static per se as you will keep adding or updating content regularly. Upon closer inspection you might concede though, that updates are not that frequent, maybe a few times a week, depending how active you are. In such a case it’s actually rather easy to turn to fully static pages, you will just need a way to automate replacing the static content regularly - more on that example in a bit.

Even for other cases it might be worth asking what portion of your content actually changes frequently and as a consequence if it might make sense to serve the static parts as such, simple html files, served via express.js and only adding dynamic pieces on top. You will be rewarded with much faster pages in return.

Case Study - how we transformed our blog to a static frontend

So far our codesphere blog is part of our monolithic website, which uses quite a few frameworks and databases to ultimately pull and serve the content from our ghost.io backend.

It includes middleware for caching, webpack for compiling the files and typescript for all the server operations. As part of our move towards more simple frameworks, that the marketing team can handle and adjust without Dev team support we are also redoing our website, and therefore building a standalone version of our blog hosted inside of codesphere.

When turning a non static use case into a purely static website you basically have two potential approaches to handling updates to the underlying content. Either you periodically run code that pulls updates and replaces the static files with updated versions or you can attach a trigger to the event that caused the update in the first place. Which option you prefer might depend on how frequent you expect updates, how computationally expensive (in terms of resources) the static file generation is and more.

For our blog we expect updates a few times a week at most, currently we do not have comments, if we did we would probably add these dynamically on top. Also our CMS ghost.io provides built in triggers (webhooks) whenever content changes.

Now the webhook offers somewhat limited functionality - therefore we did need to implement a small server for handling the requests and forwarding them to GitHub with proper authentication. In GitHub we have repository which get's the updates as a commit and the configured GitHub action then takes the updates and triggers a deployment of the updated static html, css and javascript. The functionality we exploit here is Codesphere's preview deployment via GitHub actions - just that we (mis^)use it for the actual production deployment of our headless static blog frontend. We are already working on building a public Codesphere API, this will make it even more flexible.

If running (and paying) two separate workspaces is a concern for you, you could also configure the webhook listening & committing the changes to the repo into one application, we decided against that because we don't want to occupy our static server with anything but serving static content (for maximum performance) - but you could also counter this by restricting updates to run nightly only, when traffic is low anyways.

Alright this leaves us to finish adding some additional functionality to our headless blog frontend but we managed to fit a non static case into a lightweight static infrastructure. The whole thing took around one day to setup, it would be less if you were not building an entire headless frontend from scratch and instead go with an existing html template.

Performance tests

First of all I'd like to point out that the static regeneration only runs a couple of seconds for our full set of 160+ blog posts, if the number increases greatly it would be somewhat easy to restrict regenerated files to those that changed. Also you might be concerned that the workflow with multiple workspaces and GitHub action in the middle takes long, it doesn't! From triggering a change to the update being live on the deployed frontend takes 1-2 minutes only.

Now to the actual static website, performs like a champ! We are not quite ready to share the URL yet as we are still tweaking the design but stay tuned, revisiting our blog in the future might just blow your mind 😎

Happy coding!