DEV Community: Peli de Halleux

Make Your Docs Friendly To LLMs

Peli de Halleux — Tue, 29 Oct 2024 20:50:19 +0000

This article shows how to modify your Astra Starlight documentation site so that it renders each markdown page in pain text at a known route (e.g. adding .md to each page).

HTML to text is messy, lossy

Why would you go through the trouble of providing a clean, original, copy of your markdown? Because it's much easier to download raw text than try to reconstruct it from a HTML page.

Here's an example to illustrate what tools how to deal with, with the GenAIScript tutorial page

You really want LLMs web crawler to ingest your documentation cleanly, so that RAG works better, and it'll make GenAI prompts downstream work even better.

llms.txt

This idea comes from llms.txt. llms.txt proposes to provide a new well known file (./llms.txt) to tell LLMs how to crawl the site. It also proposes to render documentation pages as markdown/plain text at a known route (<doc path>.md) so that LLM crawlers can consume the clean, original markdown contents of your docs.

Here are some examples:

Astro Starlight

Astro Starlight is a popular documentation system.

The implementation of this feature is suprisingly short and the Astro #starlight Discord was amazing at actually solving this request.

add this file in pages/[...entry].md.ts

// test.ts
/**
 * Add a .md route that return the raw markdown content of the page.
 * This is useful for markdown pages; heavy mdx pages will need more work.
 */
import type { APIRoute } from "astro"
import type { InferGetStaticPropsType, GetStaticPaths } from "astro"
import { getCollection } from "astro:content"
export const getStaticPaths = (async () => {
    const entries = await getCollection("docs")
    return entries.map((entry) => ({
        params: { entry: entry.slug },
        props: { entry },
    }))
}) satisfies GetStaticPaths

type Props = InferGetStaticPropsType<typeof getStaticPaths>
export const GET: APIRoute<Props> = (context) => {
    return new Response(context.props.entry.body, {
        headers: {
            "content-type": "text/markdown",
        },
    })
}

That's it, build again and enjoy easy rag-ing of your docs.

What about MDX?

Oopsy, all the cool stuff in your MDX component won't render nicely to text so this is still an unsolved iss.e

Search And (LLM) Transform with GenAI

Peli de Halleux — Wed, 18 Sep 2024 21:38:07 +0000

This article shows an evolution of the "search and replace" feature from text editor,
where the "replace" step has been replaced by a LLM transformation.
The example is using GenAISCript.

It can be useful to batch apply text transformations that are not easily done with
regular expressions.

For example, when we added the ability to use a string command in
the exec command, we needed to convert all calls using argument arrays to this new syntax:

host.exec("cmd", ["arg0", "arg1", "arg2"])

host.exec(`cmd arg0 arg1 arg2`)`

While it's possible to match this function call with a regular expression

host\.exec\s*\([^,]+,\s*\[[^\]]+\]\s*\)

it's not easy to formulate the replacement string... unless you can describe it in natural language:

Convert the call to a single string command shell in TypeScript

Here are some example of the transformations where the LLM correctly handled variables.

concatenate the arguments of a function call into a single string

- const { stdout } = await host.exec("git", ["diff"])
+ const { stdout } = await host.exec(`git diff`)

concatenate the arguments and use the ${} syntax to interpolate variables

- const { stdout: commits } = await host.exec("git", [
-     "log",
-     "--author",
-     author,
-     "--until",
-     until,
-     "--format=oneline",
- ])
+ const { stdout: commits } = 
+   await host.exec(`git log --author ${author} --until ${until} --format=oneline`)

Search

The search step is done with the workspace.grep
that allows to efficiently search for a pattern in files (this is the same search engine
that powers the Visual Studio Code search).

const { pattern, glob } = env.vars
const patternRx = new RegExp(pattern, "g")
const { files } = await workspace.grep(patternRx, glob)

Compute Transforms

The second step is to apply the regular expression to the file content
and pre-compute the LLM transformation of each match using an inline prompt.

const { transform } = env.vars
...
const patches = {} // map of match -> transformed
for (const file of files) {
    const { content } = await workspace.readText(file.filename)
    for (const match of content.matchAll(patternRx)) {
        const res = await runPrompt(
            (ctx) => {
                ctx.$`
            ## Task

            Your task is to transform the MATCH with the following TRANSFORM.
            Return the transformed text.
            - do NOT add enclosing quotes.

            ## Context
            `
                ctx.def("MATCHED", match[0])
                ctx.def("TRANSFORM", transform)
            },
            { label: match[0], system: [], cache: "search-and-transform" }
        )
        ...

Since the LLM sometimes decides to wrap the answer in quotes, we need to remove them.

    ...
    const transformed = res.fences?.[0].content ?? res.text
    patches[match[0]] = transformed

Transform

Finally, with the transforms pre-computed, we apply a final regex replace to
patch the old file content with the transformed strings.

    const newContent = content.replace(
        patternRx,
        (match) => patches[match] ?? match
    )
    await workspace.writeText(file.filename, newContent)
}

Parameters

The script takes three parameters: a file glob, a pattern to search for, and a LLM transformation to apply.
We declare these parameters in the script metadata and extract them from the env.vars object.

script({ ...,
    parameters: {
        glob: {
            type: "string",
            description: "The glob pattern to filter files",
            default: "*",
        },
        pattern: {
            type: "string",
            description: "The text pattern (regular expression) to search for",
        },
        transform: {
            type: "string",
            description: "The LLM transformation to apply to the match",
        },
    },
})
const { pattern, glob, transform } = env.vars

Running

To run this script, you can use the --vars option to pass the pattern and the transform.

genaiscript run st --vars 'pattern=host\.exec\s*\([^,]+,\s*\[[^\]]+\]\s*\)' 'transform=Convert the call to a single string command shell in TypeScript'

GenAI Image Alt Text

Peli de Halleux — Fri, 30 Aug 2024 21:11:28 +0000

It is a best practice to provide an alt attribute for images.
This attribute is used to describe the image to users who are unable to see it.
It is also used by search engines to understand the content of the image.

<img src="..." alt="describe the image here" />

However, this task can be tedious and developer are often tempted to skip it, or provide a generic alt text like "image".

<img src="..." alt="image" />

GenAI to the rescue

To solve this issue, we created a GenAIScript script that uses the OpenAI Vision model (gpt-4-turbo-v or gpt-4o) to analyze the documentation images and generate a description alt text.

We assume that the script is run on a single image file
and we use defImages (provided by GenAIScript) to add it to the prompt context.

const file = env.files[0] // selected in UI or cli
defImages(file) // encode image in openai json payload

Then we give a task to the LLM to generate a good alt text.

$`You are an expert in assistive technology. 
Generate a description alt text for the image.`

finally, we use defFileOutput to define a file output route.

defFileOutput(file.filename + ".txt", `Alt text for image ${file.filename}`)

Usage in Astro

The GenAIScript documentation uses Astro, which allows to author pages in MDX.
The code below shows how the generate alt text, stored in a separate text file, is injected in the final HTML.

import { Image } from "astro:assets"
import src from "../../../assets/debugger.png"
import alt from "../../../assets/debugger.png.txt?raw"

<Image src={src} alt={alt} />

The debugger.png image shows the screenshot of a debugging session and the generate alt text file contents.

Full source

script({ model: "openai:gpt-4-turbo-v", maxTokens: 4000, })
const file = env.files[0]
defImages(file)
$`You are an expert in assistive technology. 
Generate a description alt text for the image.
- Do not include Alt text in the description.`
defFileOutput(file.filename + ".txt", `Alt text for image ${file.filename}`)

GenAI Git Commits

Peli de Halleux — Tue, 27 Aug 2024 14:26:53 +0000

Generating a git commit message is quickly become a classic GenAI application for developers.

To help with this, we will craft a GenAIScript script.

The script acts as a regular node.js automation script and uses runPrompt
to issue calls to the LLM and ask the user to confirm the generated text.

🔍 Explaining the Script

We start by using git check if there are any staged changes in the Git repository:

let { stdout } = await host.exec("git", ["diff", "--cached"])

If no changes are staged, we ask the user if they want to stage all changes. If the user confirms, we stage all changes. Otherwise, we bail out.

    const stage = await host.confirm("Stage all changes?", {
        default: true,
    })
    if (stage) {
        await host.exec("git", ["add", "."])
        stdout = (await host.exec("git", ["diff", "--cached"])).stdout
    }
    if (!stdout) cancel("no staged changes")

We generate an initial commit message using the staged changes:

message = (
    await runPrompt(
        (_) => {
            _.def("GIT_DIFF", stdout, { maxTokens: 20000 })
            _.$`GIT_DIFF is a diff of all staged changes, coming from the command:
\`\`\`
git diff --cached
\`\`\`
Please generate a concise, one-line commit message for these changes.
- do NOT add quotes`
        },
        { temperature: 0.8 }
    )
).text

The prompt configuration above indicates that the message should be concise,
related to the "git diff --cached" output, and should not include quotes.

User chooses how to proceed with the generated message:

    choice = await host.select(
        message, [
            { name: "commit", value: "commit", description: "accept message and commit" },
            ...
        ])

Options are given to edit or regenerate the message. If the user chooses to edit the message, we ask them to input a new message:

    if (choice === "edit") {
        message = await host.input("Edit commit message", {
            required: true,
        })
        choice = "commit"
    }

If the user chooses to commit the message, we commit the changes:

    if (choice === "commit" && message) {
        console.log((await host.exec("git", ["commit", "-m", message])).stdout)
    }

🚀 Running the Script

You can run this script using the CLI.

genaiscript run gcm

If you are using npx,

npx --yes genaiscript run gcm

This command will run the script, and guide you through the process of generating and committing a Git message using AI, making your commits more informative and consistent.

You can wrap this command in a gcm.sh file or in your package script section in package.json:

{
    "devDependencies": {
        "genaiscript": "..."
    },
    "scripts": {
        "gcm": "genaiscript run gcm"
    }
}

Then you can run the script using:

npm run gcm

Acknowledgements

This script was inspired from Karpathy's commit message generator.

Keeping the Your README Fresh and Engaging using AI

Peli de Halleux — Sat, 24 Aug 2024 17:25:23 +0000

In the world of open source, a well-maintained README file acts as the front door to your project. It's often the first thing potential users and contributors see, and as such, it should be both informative and inviting. Today, we're diving into the GenAIScript that helps keep the README of the GenAI project as fresh as a daisy! 🌼 Check out the actual script file for the details.

This blog post was mostly co-authored with a script.

The Intention Behind the Script

The script we're analyzing is a maintenance tool designed to import relevant information from documentation and samples into the README to enhance its appeal to users. It ensures that the README is not just a static file but a vibrant, updated document that accurately reflects the features and capabilities of GenAI.

Line-by-Line Explanation

Let's walk through the script code as if we are crafting it from the ground up:

script({
    description:
        "Maintenance script for the README that imports information from the documentation and samples to make it more attractive to users.",
    tools: ["fs"],
})

Here, we're defining the script's metadata, including a description of its purpose and the tools it will utilize. The fs tool indicates file system operations will be involved.

def("README", { filename: "README.md" })
def("FEATURES", { filename: "docs/src/content/docs/index.mdx" })

These lines declare two important files: the README itself and a FEATURES file that contains information to be imported into the README.

$`You are an expert open source maintainer.
...
`

In this template literal, we're outlining the tasks for the script, including guidelines for updating the README with features, samples, and documentation links while preserving certain sections unchanged.

defFileOutput("README.md")

Finally, we specify that the output of this script will be an updated README.md file.

How to Run the Script

To execute this maintenance script, you'll need the GenAIScript CLI. If you haven't installed it yet, head over to the official documentation for installation instructions. Once you have the CLI ready, run the following command in your terminal:

genaiscript run readme-updater.genai.mts

This command will kick off the script and apply the enhancements to your README file, ensuring it's up-to-date and user-friendly.

Conclusion

A meticulous README is a hallmark of a well-maintained open source project. With this GenAIScript, the GenAI project sets an excellent example of automating the upkeep of project documentation. Embrace the power of automation to keep your project's welcome mat clean and welcoming. Happy coding! 👨‍💻👩‍💻

Use AI to detect of Outdated Descriptions

Peli de Halleux — Thu, 22 Aug 2024 12:26:58 +0000

Developer documentation typically includes a description in each file. This descriptions can become outdated, leading to confusion and incorrect information. To prevent this, you can automate the detection of outdated descriptions in your documentation using a bit of AI and GenAIScript.

Markdown and frontmatter

Many documentation systems use the markdown format to write documentation and a 'frontmatter' header to store metadata. Here’s an example of a markdown file with frontmatter:

---
title: "My Document"
description: "This is a sample document."
---

# My Document

Lorem ipsum dolor sit amet, consectetur adipiscing elit.

The goal is to create a script that detects when the description field in the frontmatter is outdated.

The script

GenAIScript is meant to run on files and provides a special variable env.files that contains the list of files to be analyzed. You can use this variable to include the files in the context using the def function. We limit each file to 2000 tokens to avoid exploding the content on large files.

// Define the file to be analyzed
def("DOCS", env.files, { endsWith: ".md", maxTokens: 2000 })

The next step is to give a task to the script. In this case to check that the content and description field in the frontmatter match.

// Analyze the content to detect outdated descriptions
$`Check if the 'description' field in the front matter in DOCS is outdated.`

Finally, we leverage the built-in diagnostics generation feature to create an error for each outdated description.

// enable diagnostics generation
$`Generate an error for each outdated description.`

Running in Visual Studio Code

Once you save this script in your workspace, you will be able to execute it on a file or a folder through the context menu
by selecting Run GenAIScript....

Automation

You can automatically run this tool on your documentation files to identify outdated descriptions using the cli.

npx --yes genaiscript run detect-outdated-descriptions **/*.md

This script can be integrated into your CI/CD pipeline to automate the detection process.

Unlocking the Power of Prompts - A Gentle Introduction to GenAIScript 🚀

Peli de Halleux — Tue, 20 Aug 2024 16:20:58 +0000

Ever wondered how to leverage the power of AI and Large Language Models (LLMs) in your projects? Look no further!
Hello GenAIScript,
a tool designed to simplify the creation of prompts and interactions with LLMs. Let's dive in! 🌊

What is GenAIScript?

GenAIScript uses a stylized version of JavaScript to generate prompts, which are then sent to an LLM.
Scripts are stored as files (genaisrc/*.genai.mjs), executed to produce the prompt text and structured results (files, diagnostics) are extracted automatically. It runs as a VSCode extension or as a CLI.

Getting Started

Here's a simple example to get you started. Create a file named poem.genai.mjs in the genaisrc folder and add the following code:

$`Write a one sentence poem.`

When executed, this script will generate the following prompt:

👤 User

Write a one sentence poem.

🤖 Assistant

Roses bloom, hearts swoon, under the silver moon.

Adding Context

GenAIScript can also use context variables, allowing you to interact with files or other data sources. Let's see an example where we define a context variable using env.files:

def("FILES", env.files)
$`You are an expert technical writer and proofreader.
Review the documents in FILES and report the 2 most important issues.`

Execute this script to see the generated user message and the assistant's response. The context variable FILES will contain the list of files in the environment.

👤 User

FILES: 
file="src/samples/markdown.md"
What is Markdown?
Markdown is a lightweight markup language that...

You are an expert technical writer and proofreader.
Review the documents in FILES and report the 2 most important issues.

🤖 Assistant

I reviewed the document in "src/samples/markdown.md" 
and found the following two important issues:

1. **Missing Consistency in Heading Styles**: ...

Metadata and Script Configuration

You can add metadata to your script using the script function. This helps in organizing and configuring the script, including specifying the model and other parameters. GenAIScript supports various LLM providers, such as OpenAI, Azure OpenAI,
GitHub Models, Ollama and more.

script({
    title: "Technical proofreading",
    description: "Reviews the text as a tech writer.",
    model: "openai:gpt-3.5-turbo",
    temperature: 0.1,
})
def("FILES", env.files)
$`You are an expert technical writer and proofreader.
Review the documents in FILES and report the 2 most important issues.`

Next Steps

Getting started guide to configure and start using GenAIScript.
Explore more advanced scripts by following the Prompt As Code guide.

There you have it! A gentle introduction to GenAIScript to get you started on your prompt engineering journey. Happy scripting! 💻✨

DeviceScript and LCD screens

Peli de Halleux — Mon, 09 Oct 2023 16:18:15 +0000

This article shows how to use DeviceScript to program a microcontroller (ESP32-C3), a temperature/humidity sensor (BME680) and a LCD character screen to build a tiny weather dashboard. This project does not require soldering or embedded skills.

In a previous article, we looked at using an OLED screen to display temperature readings using DeviceScript.

DeviceScript

DeviceScript brings JavaScript/TypeScript to tiny IoT devices such as the ESP32. DeviceScript provides a user friendly editing/debugging experience in Visual Studio Code.

You can follow along this tutorial without any hardware and use the simulators provided by DeviceScript. Otherwise, the parts needed are

Seeed Studio Xiao + Grove shield

The Xiao ESP32-C3 is a low-cost compact microcontroller board running the Espressif ESP32-C3.

The Grove shield for Xiao is a convenient base for the Xiao that provides an easy access to OLED screens, Grove connectors and battery charging.

DeviceScript provides a shield drivers that configures the pins and the OLED display for this hardware combo.

import { XiaoGroveShield} from "@devicescript/drivers"

const shield = new XiaoGroveShield()

LCD screen as character screen

The Grove LCD screen supports 16 columns and rows of characters; it provides an I2C interface that dramatically reduces the complexity of connecting this kind of devices (just plug the grove connector and go). The driver is written in TypeScript/DeviceScript.

The sceen is modelled as a character screen. Using services allows DeviceScript to provide simulation and device twin capabilities to your coding experience.

import {
    XiaoGroveShield,
    startGroveRGBLCD16x2,
} from "@devicescript/drivers"

const shield = new XiaoGroveShield()
const screen = await startGroveRGBLCD16x2()
await screen.message.write('starting...')

BME680

The BME680 is a high-precision temperature/humidity/gas sensor that can be conveniently connected through one of the grove connector.

The sensor is exposed as 3 services, temperature, humidity, air quality, that can be used to poll the sensor data and display it on the screen.

import {
    startBME680,
} from "@devicescript/drivers"

...
const { temperature, humidity } = await startBME680({
    address: 0x76
})
const temp = await temperature.reading.read()
const humi = await humidity.reading.read()
await screen.message.write(`t ${temp}\nh ${humi}`)

Simulation

You try this code without hardware. DeviceScript will spin up simulators for the Xiao, the sensors and screens.

Value dashboard

Since display sensor values is a common task, DeviceScript provides a helper ValueDashboard to render a list of sensor data.

The value dashboard

import { ValueDashboard } from "@devicescript/runtime"

...
const dashboard = new ValueDashboard(screen, {
    temperature: { digits: 1, unit: "C" },
    humi: { digits: 0, unit: "%" },
})

setInterval(async () => {
    dashboard.values.temperature = temperature.reading.read()
    dashboard.values.humi = humidity.reading.read()
    await dashboard.show()
}, 1000)

All together

The final sample is a tiny weather display that shows the temperature/humidity reported by the BME680 sensor.

import {
    startGroveRGBLCD16x2,
    startBME680,
    XiaoGroveShield,
} from "@devicescript/drivers"
import { ValueDashboard } from "@devicescript/runtime"

const shield = new XiaoGroveShield()
const { temperature, humidity } = await startBME680({
    address: 0x76,
})
const screen = await startGroveRGBLCD16x2()

const dashboard = new ValueDashboard(screen, {
    temp: { digits: 1, unit: "C" },
    humi: { digits: 0, unit: "%" },
})

setInterval(async () => {
    dashboard.values.temp = await temperature.reading.read()
    dashboard.values.humi = await humidity.reading.read()
    await dashboard.show()
}, 1000)

DeviceScript and OLED screens

Peli de Halleux — Tue, 26 Sep 2023 16:12:14 +0000

This article shows how to use DeviceScript to program a microcontroller (ESP32-C3), a temperature/humidity sensor (BME680) and a OLED screen to build a tiny weather dashboard. This project does not require soldering or embedded skills.

DeviceScript

DeviceScript brings JavaScript/TypeScript to tiny IoT devices such as the ESP32. DeviceScript provides a user friendly editing/debugging experience in Visual Studio Code.

You can follow along this tutorial without any hardware and use the simulators provided by DeviceScript. Otherwise, the parts needed are

Seeed Studio Xiao + Expansion board

The Xiao ESP32-C3 is a low-cost compact microcontroller board running the Espressif ESP32-C3.

The Xiao Expansion board base is a convinient base for the Xiao that provides an easy access to OLED screens, Grove connectors and battery charging.

DeviceScript provides a shield drivers that configures the pins and the OLED display for this hardware combo.

import { XiaoExpansionBoard } from "@devicescript/drivers"

const board = new XiaoExpansionBoard()
const display = await board.startDisplay()

OLED screen as character screen

The OLED screen on the expansion board is monochrome and has a 128x64 pixels resolution. The display is crisp and perfect to display sensor information.

To simplify working with the screen, we model it as a character screen. Using services allows DeviceScript to provide simulation and device twin capabilities to your coding experience.

import {
    ...,
    startCharacterScreen,
} from "@devicescript/drivers"

...
const screen = await startCharacterScreen(display)
// update text on character screen
await screen.message.write(":)")

BME680

The BME680 is a high-precision temperature/humidity/gas sensor that can be conveniently connected through one of the grove connector.

The sensor is exposed as 3 services, temperature, humidity, air quality, that can be used to poll the sensor data and display it on the screen.

import {
    startBME680,
} from "@devicescript/drivers"

...
const { temperature, humidity } = await startBME680({
    address: 0x76
})
const temp = await temperature.reading.read()
const humi = await humidity.reading.read()
await screen.message.write(`t ${temp}\nh ${humi}`)

Value dashboard

Since display sensor values is a common task, DeviceScript provides a helper ValueDashboard to render a list of sensor data.

The value dashboard

import { ValueDashboard } from "@devicescript/runtime"

...
const dashboard = new ValueDashboard(screen, {
    temperature: { digits: 1, unit: "C" },
    humi: { digits: 0, unit: "%" },
})

setInterval(async () => {
    dashboard.values.temperature = temperature.reading.read()
    dashboard.values.humi = humidity.reading.read()
    await dashboard.show()
}, 1000)

All together

The final sample is a tiny weather display that shows the temperature/humidity reported by the BME680 sensor.

import {
    XiaoExpansionBoard,
    startCharacterScreen,
    startBME680,
} from "@devicescript/drivers"
import { ValueDashboard } from "@devicescript/runtime"

const board = new XiaoExpansionBoard()
const { temperature, humidity } = await startBME680({
    address: 0x76
})
const display = await board.startDisplay()
const screen = await startCharacterScreen(display)

const dashboard = new ValueDashboard(screen, {
    temperature: { digits: 1, unit: "C" },
    humi: { digits: 0, unit: "%" },
})

setInterval(async () => {
    dashboard.values.temperature = await temperature.reading.read()
    dashboard.values.humi = await humidity.reading.read()
    await dashboard.show()
}, 1000)

DeviceScript - Temperature + MQTT

Peli de Halleux — Mon, 18 Sep 2023 15:56:12 +0000

This article uses an ESP32-C3 board Adafruit QT Py C3
and a SHTC3 sensor to publish a temperature reading to
the Adafruit.io MQTT APIs every minute using DeviceScript.

DeviceScript

DeviceScript brings JavaScript/TypeScript to tiny IoT devices such as the ESP32. DeviceScript provides a user friendly editing/debugging experience in Visual Studio Code.

Reading temperature

We start by configuring the script for the QT Py and adding a scheduled interval to read the temperature
from the SHTC3 sensor every 60 seconds.

// hardware configuration and drivers
import "@dsboard/adafruit_qt_py_c3"
import { startSHTC3 } from "@devicescript/drivers"
import { schedule } from "@devicescript/runtime"

// mounting a temperature server for the SHTC3 sensor
const { temperature } = await startSHTC3()

schedule(
    async () => {
        // read data from temperature sensor
        const value = await temperature.reading.read()
    },
    { timeout: 1000, interval: 60000 }
)

Configuration and Secrets

To connect to Adafruit.io, you will to get an account with https://io.adafruit.com
and store your username and password in the settings
as the IO_USERNAME and IO_KEY keys (make sure your key is in env.local).
Also create a feed and update the feed key in the example below.

import { readSetting } from "@devicescript/settings"

// TODO: update feed key
const feed = "temperature"
const username = await readSetting("IO_USERNAME")
// this secret is stored in the .env.local 
// and uploaded to the device settings
const password = await readSetting("IO_KEY")

Starting the MQTT client

Following the Adafruit documentation,
we start a MQTT connection and craft a topic that will route the data to our account.

import { startMQTTClient } from "@devicescript/net"

...

const mqtt = await startMQTTClient({
    host: `io.adafruit.com`,
    proto: "tls",
    port: 8883,
    username,
    password,
})
const topic = `${username}/feeds/${feed}/json`

Publish data

With the MQTT client and the topic, we can add a call to mqtt.publish in the scheduled worker
to upload the data to Adafruit (note that { value } expands to JSON { "value": value } automatically)

schedule(
    async () => {        
        ...
        // publish data to Adafruit
        await mqtt.publish(topic, { value })
    },
    { timeout: 1000, interval: 60000 }
)

All together

Putting all the pieces together we get the following program.

import "@dsboard/adafruit_qt_py_c3"
import { startSHTC3 } from "@devicescript/drivers"
import { startMQTTClient } from "@devicescript/net"
import { readSetting } from "@devicescript/settings"
import { schedule } from "@devicescript/runtime"

const { temperature } = await startSHTC3()

const feed = "temperature"
const username = await readSetting("IO_USERNAME")
device settings
const password = await readSetting("IO_KEY")

const mqtt = await startMQTTClient({
    host: `io.adafruit.com`,
    proto: "tls",
    port: 8883,
    username,
    password,
})
const topic = `${username}/feeds/${feed}/json`

schedule(
    async () => {
        const value = await temperature.reading.read()
        await mqtt.publish(topic, { value })
    },
    { timeout: 1000, interval: 60000 }
)

Extra points: Filtering data

You could use observables to smooth the sensor
data. For example, apply an exponentially moving average
on the feed of temperature readings.

import { ewma, auditTime } from "@devicescript/observables"
...
temperature.reading
    .pipe(ewma(0.5), auditTime(60000))
    .subscribe(async value => await mqtt.publish(topic, { value }))

DeviceScript - TypeScript for microcontrollers

Peli de Halleux — Wed, 17 May 2023 03:26:27 +0000

If you are familiar with TypeScript, but could never really catch on with embedded programming, this article is for you!

TypeScript for microcontrollers

DeviceScript brings a professional TypeScript developer experience to low-resource microcontroller-based devices, like the ESP32 or Raspberry Pi Pico.

Think of DeviceScript as a way to write TypeScript applications, not for the web or a backend or a phone but for a device that has only 64kb of RAM, runs on battery for months and can interface with tons of sensors and actuators.

How does it work

The DeviceScript language is a subset of TypeScript designed to run efficiently on tiny microcontrollers with very limited storage and memory. Because it is TypeScript, you still benefit from the great developer experience in Visual Studio; on top of additional features provided by the DeviceScript extension.

The DeviceScript compiler then turns your code into a bytecode that is executed in a virtual machine (written in C) on the microcontroller. The bytecode is designed to be compact and memory efficient.

From your point of view, it's just like you coding in TypeScript , yay!

Blinky

Every embedded journey starts with a blinking LED and DeviceScript is no different. We will use a Raspberry Pi Pico in this example and we will flip a GPIO pin between high and low.

According to the pico pinout diagram, the LED in the picture above is connected from pin 2, labelled GP1 to ground.

The full guide is available at https://microsoft.github.io/devicescript/getting-started/samples/blinky. Shorter version inlined.

New project

After installing the DeviceScript extension (there is also a cli), you can use the DeviceScript: Create New Project... option to start a new project.

The project looks very much like a npm-project with a package.json, a few helper files for IDEs and a main.ts file.

A first (simulated) run

Open main.ts and click the run menu item on the file to start a debugging session.

The extension will look for a connected device and, if not found, will default to a simulator (which happens to be the wasmed C runtime).

Configuring hardware

Since we know the target microcontroller, we can click on the wand icon and import the pin mapping for the pico as well as starting a lightbulb server on pin P1.

import { pins } from "@dsboard/pico"
import { startLightBulb } from "@devicescript/servers"

// start a lightbulb server on pin GP1
// and store client in `lightBulb` variable
const lightBulb = startLightBulb({
    pin: pins.P1,
})

In DeviceScript, hardware is virtualized as services where sensors are servers that clients in application can interact with him... just like servers and clients in the web.

Read and write registers

To blink the LED, we call toggle on the ligh bulb client every second. toggle is async so we also add the await keyboard.

import { pins } from "@dsboard/pico"
import { startLightBulb } from "@devicescript/servers"

const lightBulb = startLightBulb({
    pin: pins.P1,
})

// start interval timer every 1000ms
setInterval(async () => {
    // toggle brightness
    await lightBulb.toggle()
}, 1000)

Run this program. Since the program uses a light bulb service, the extension will suggest you to start a light bulb simulator (server) so that the client can be bound and send commands.

Connecting the pico

So far, we've developed the code using simulators. It's time to hit the metal. Connect your Raspberry Pi Pico ( ESP32 is also supported), and flash the DeviceScript runtime firmware. Once this is done, it is ready to run the bytecode.

Click on the connect button again to connect your device and run your file... and blink the LED!

Beyond Blinky

Obviously Blinky is just the start. DeviceScript opens the door to embedded programming for developer who are already familiar with TypeScript. Stay tuned for more articles or dive into the docs.

DeviceScript - Blinky

Peli de Halleux — Tue, 16 May 2023 05:15:54 +0000

What if you could write embedded code in TypeScript?

Enter DeviceScript, TypeScript for microcontrollers...