DEV Community: Maksim Balabash

How to Create a Browser Extension that Runs an LLM Locally

Maksim Balabash — Wed, 04 Jun 2025 10:42:35 +0000

The Internet always felt like a little crazy person, but in a good way. It still feels this way, but after recent years of expansion, it seems like that person is going through some mental health issues currently (or a transformation, like puberty).

When reading online content today, it's unfortunately common to realize that the text has a hidden agenda attempting to draw you into someone's camp. And AI has only made the situation worse.

However, AI creates numerous opportunities, leading to the emergence of new tools and services daily.

WebLLM is one such tool that makes it incredibly easy to use LLM locally in your browser. We will create a Chrome extension that tries to uncover ideological markers, promoted values, emotional manipulations, and logical fallacies behind the texts we read (here's a link to the repository).

We want an extension that:

extracts content from the page, a selected fragment, or textual information from an image
analyzes the content and identifies:
- ideological influences
- promoted values
- emotional manipulations
- logical fallacies
presents a concise report summarizing its findings

What can extensions do?

modify and enhance pages
read and analyze content of a page
automate browser tasks
interact with browser tabs and windows
integrate 3d party APIs or services
provide UI within the browser
run tasks in the background
enhance security and privacy
etc.

What limitations do extensions have?

can't directly access computer's files outside of special user interactions
extensions must request permissions to access certain data (pages, cookies, history), ensuring privacy and transparency
they run within the browser context and cannot modify system settings directly

At this point, we are good to go. Gathering data from a page, processing it in the background, and displaying insights in the popup is possible.

Anatomy of an extension

Manifest
- defines permissions, scripts, settings, resources, and security policies
Background script
- runs initially upon extension installation or browser startup
- continuously listens for events, manages extension-wide tasks, and maintains global state
- can be paused or stopped by the browser after inactivity
Content script
- injected automatically into specified pages whenever such page loads or reloads
- has direct access to the page's DOM, allowing modifications or reading of page data
- destroyed when the page is closed or refreshed
Popup script
- controls the popup UI
- runs when users click your extension's toolbar icon inside its own separate, temporary context
- immediately terminates when the popup is closed

WebLLM

WebLLM is a project built upon MLC LLM (Machine Learning Compilation for Large Language Models).

MLC LLM compiles and runs code on a unified, high-performance, cross-platform LLM inference engine, leveraging hardware acceleration (like GPUs or optimized CPU instructions) to ensure faster inference times.

This is how they describe themselves: "The mission of this project is to enable everyone to develop, optimize, and deploy AI models natively on everyone's platforms." Kudos to them!

WebLLM includes all the essential features you might need, such as in-browser inference, full OpenAI API compatibility, extensive model support, custom model integration, streaming, and more.

Three interesting directions that this tool opens up:

we can enhance privacy by analyzing sensitive data on-device, such as personal notes, emails, or private documents, without sending any information externally
our apps can deliver advanced AI services even in offline situations or areas with low connectivity
as models improve, we can personalize experiences and offer more interactive features to users without incurring additional costs for cloud-based AI inference.

Now that we have covered all the basics, let's move on to the implementation!

Choosing the Tech Stack

Manifest v3 (to comply with modern browser standards)
WebLLM (for executing ML models directly in-browser)
Tesseract.js (for OCR capabilities to extract text from images)
@mozilla/readability (to extract cleaned text from a page's DOM)
TypeScript (to make our life easier)
Vite (to build and bundle project files)

Model controller

Although we can run models locally, it would be nice to have OpenAI and Anthropic models as well. This would allow us to test additional models and make our extension compatible with older devices (which may have poor performance when running LLM on their hardware).

We can accomplish this by using the strategy pattern. We will put specific details of each implementation into separate classes, all of which will implement the same abstract interface that defines the model API, making their objects interchangeable.

This is the interface that every model provider must implement:

export interface LLMStrategy {
  id: string
  contextWindowSize: number
  getCompletions(request: CompletionRequest): Promise<CompletionResponse>
  stopGeneration(): void
}

This is our entry point when working with the model:

// src/scripts/model.ts

export class ModelController {
  private strategy: LLMStrategy | null = null

  constructor(strategy?: LLMStrategy) {
    if (strategy) {
      this.strategy = strategy
    }
  }

  setStrategy(strategy: LLMStrategy) {
    this.strategy = strategy
  }

  getContextWindowSize() {
    if (!this.strategy) {
      throw new Error('To obtain the context window size, the model strategy must be set')
    }

    return this.strategy.contextWindowSize
  }

  getModelId() {
    if (!this.strategy) {
      throw new Error('To obtain the model id, the model strategy must be set')
    }

    return this.strategy.id
  }

  async getCompletions(request: CompletionRequest): Promise<CompletionResponse> {
    if (!this.strategy) {
      throw new Error('To get completions, the model strategy must be set')
    }

    try {
      return this.strategy.getCompletions({ temperature: 0.99, maxTokens: 4000, ...request })
    } catch (error) {
      throw new Error('Failed to get completions', { cause: error })
    }
  }

  stopGeneration() {
    if (!this.strategy) {
      throw new Error('To stop completions generation, the model strategy must be set')
    }

    return this.strategy.stopGeneration()
  }
}

And here's how we use it:

// src/scripts/model.ts
const model = new ModelController()

if (
  !config.modelToUse.length ||
  config.modelToUse === 'WEBLLM_LLAMA' ||
  config.modelToUse === 'WEBLLM_QWEN' ||
  config.modelToUse === 'WEBLLM_DEEPSEEK_LLAMA' ||
  config.modelToUse === 'WEBLLM_MISTRAL'
) {
  const localModelStrategy = new WebLLMStrategy()
  await localModelStrategy.initModel(config.modelToUse)
  model.setStrategy(localModelStrategy)
} else if (config.modelToUse === 'OPENAI') {
  model.setStrategy(new OpenAIStrategy(config.openAiApiKey))
} else if (config.modelToUse === 'ANTHROPIC') {
  model.setStrategy(new AnthropicStrategy(config.anthropicApiKey))
}

...

const response = await model.getCompletions({ ... })

The optimal way to utilize models through WebLLM inside an extension is by using the CreateExtensionServiceWorkerMLCEngine, which supports caching, webGPU, and more.

First, we need to register the worker in the background script of our extension:

// src/scripts/background.ts
... 

let webLLMWorker: ExtensionServiceWorkerMLCEngineHandler | undefined

chrome.runtime.onConnect.addListener(port => {
  if (webLLMWorker === undefined) {
    webLLMWorker = new ExtensionServiceWorkerMLCEngineHandler(port)
  } else {
    webLLMWorker.setPort(port)
  }

  port.onMessage.addListener(message => {
    webLLMWorker?.onmessage.bind(webLLMWorker)(message)
  })
})

...

Here's a lightweight example of how this can be used:

const webllm: MLCEngineInterface = await CreateExtensionServiceWorkerMLCEngine('Llama-3.2-3B-Instruct-q4f16_1-MLC')

const completion = await webllm.chat.completions.create({
  stream: true,
  temperature: 1,
  response_format: { type: 'json_object' },
  messages: [...]
})

That's all we need. Yep, it's as simple as that!

OpenAIStrategy and AnthropicStrategy essentially create different objects and call the corresponding APIs.

You can find the complete code for implementing each model strategy here (it's pretty straightforward).

Settings

Here are two code fragments that update the extension's settings in browser storage and render the actual configuration:

// src/scripts/settings.ts

export class ConfigManager {
  private static readonly POSSIBLE_MODELS: ModelType[] = [
    'WEBLLM_LLAMA',
    'WEBLLM_MISTRAL',
    'WEBLLM_QWEN',
    'WEBLLM_DEEPSEEK_LLAMA',
    'OPENAI',
    'ANTHROPIC'
  ]

  static validateModelType(model: string): model is ModelType {
    return this.POSSIBLE_MODELS.includes(model as ModelType)
  }

  static async saveModelChoice(model: ModelType): Promise<void> {
    await chrome.storage.sync.set({ modelToUse: model })
  }

  static async saveApiKey(provider: 'openai' | 'anthropic', key: string): Promise<void> {
    const storageKey = provider === 'openai' ? 'openAiApiKey' : 'anthropicApiKey'
    await chrome.storage.sync.set({ [storageKey]: key })
  }
}

export class SettingsUIManager {
  private elements: {
    modelInputs: NodeListOf<HTMLInputElement>
    openAiInput: HTMLInputElement
    anthropicInput: HTMLInputElement
  }

  constructor() {
    const modelInputs = document.querySelectorAll<HTMLInputElement>('input[name="modelToUse"]')
    const openAiInput = document.getElementById('openAiApiKeyInput') as HTMLInputElement
    const anthropicInput = document.getElementById('anthropicApiKeyInput') as HTMLInputElement

    if (!modelInputs.length || !openAiInput || !anthropicInput) {
      throw new Error('Required UI elements not found')
    }

    this.elements = { modelInputs, openAiInput, anthropicInput }
  }

  initializeUI(config: ExtensionConfig): void {
    this.elements.modelInputs.forEach(input => {
      input.checked = input.value === config.modelToUse
    })
    this.elements.openAiInput.value = config.openAiApiKey
    this.elements.anthropicInput.value = config.anthropicApiKey

    this.updateInputStates(config.modelToUse)
  }

  setupEventListeners(): void {
    this.elements.modelInputs.forEach(input => {
      input.addEventListener('change', () => this.handleModelChange(input))
    })

    this.elements.openAiInput.addEventListener('change', e =>
      this.handleApiKeyChange('openai', (e.target as HTMLInputElement).value)
    )

    this.elements.anthropicInput.addEventListener('change', e =>
      this.handleApiKeyChange('anthropic', (e.target as HTMLInputElement).value)
    )
  }

  private updateInputStates(modelToUse: ModelType): void {
    const isWebLLM = modelToUse.startsWith('WEBLLM_')

    this.elements.openAiInput.disabled = isWebLLM || modelToUse === 'ANTHROPIC'
    this.elements.anthropicInput.disabled = isWebLLM || modelToUse === 'OPENAI'
  }

  private async handleModelChange(input: HTMLInputElement): Promise<void> {
    if (input.checked && ConfigManager.validateModelType(input.value)) {
      await ConfigManager.saveModelChoice(input.value as ModelType)
      this.updateInputStates(input.value as ModelType)
    }
  }

  private async handleApiKeyChange(provider: 'openai' | 'anthropic', value: string): Promise<void> {
    await ConfigManager.saveApiKey(provider, value)
  }
}

We can now easily switch between different models for convenience and to evaluate the output quality of each one.

Content extraction

To retrieve any information from the page, we need to request it from the content or background script, which will then pass the data back to the popup context, where we can process it. That's how things work in extensions.

We request it by emitting specific events that scripts will treat and handle as commands. Our extension has three trigger buttons (two in the popup and one in the context menu) that activate those events. As a result, the text analysis function is called with user data:

// src/scripts/user-input.ts

...

export class UserInputController {
  ...

  handleImageInput = async (event: Event, onDataReady: analyzeContentFn) => {
    const input = event.target as HTMLInputElement
    const files = input.files
    if (!files || !this.ocrWorker) {
      return
    }

    const response = await this.ocrWorker.recognize(files[0])
    const content = removeExtraWhitespaces(response.data.text || '')

    await onDataReady(content, 'image')
  }

  setupEventListenerForTextSelection = (onDataReady: analyzeContentFn) => {
    const processSelection = async (selection: Record<string, unknown>) => {
      if (selection && selection.text && typeof selection.text === 'string') {
        await onDataReady(removeExtraWhitespaces(selection.text), 'selection')
      }
    }

    chrome.runtime.onMessage.addListener(message => {
      if (message.type === 'pendingSelectionReady') {
        chrome.runtime.sendMessage({ type: 'getPendingSelection' }, processSelection)
      }
    })
  }

  handlePageContentInput = async (onDataReady: analyzeContentFn) => {
    const documentClone = await this.getPageContent()
    const article = new Readability(documentClone).parse()
    const mainContent = removeExtraWhitespaces(
      `${article?.title || ''}\n${article?.textContent || ''}`
    )

    await onDataReady(mainContent, 'page')
  }

  private async getPageContent(timeout: number = 5000): Promise<Document> {
    if (!this.contentScriptConnection) {
      throw new Error('Connection to content script not established')
    }

    return new Promise((resolve, reject) => {
      const timeoutId = setTimeout(
        () => reject(new Error('Request timeout: Unable to retrieve page content')),
        timeout
      )

      const handleResponse = (response: { type: string; contents: unknown }) => {
        clearTimeout(timeoutId)

        const doc = this.domParser.parseFromString(response.contents as string, 'text/html')
        this.contentScriptConnection!.removeMessageHandler('pageContent', handleResponse)

        resolve(doc)
      }

      this.contentScriptConnection!.addMessageHandler('pageContent', handleResponse)
      this.contentScriptConnection!.sendMessage({ type: 'getPageContent' })
    })
  }
}

...

Prompt

We reached the point where we can obtain the text the user wants us to process, and we can run local and cloud-based LLMs. Let's create a prompt to guide the model in extracting the insights we need about the text.

// src/scripts/prompt.ts

export function craftInstructions(text: string): Prompt {
  if (!text.trim()) {
    throw new Error('Text parameter is required and cannot be empty')
  }

  const intensityKeywords = ['NONE', 'WEAK', 'MODERATE', 'STRONG', 'EXTREME'].join(' | ')

  const prompt = {
    system: `
        You are a media analysis assistant specializing in identifying manipulation tactics and bias in text.
        Your task is to analyze the provided text and generate a structured, evidence-based report in JSON format.
        Follow the detailed framework below to ensure comprehensive analysis:
        1. Requirements:
          - Evaluate both explicit and subtle manipulation tactics.
          - Base your analysis on clear, evidence-backed reasoning.
          - Include direct quotes for all identified elements.
          - Provide a concise explanation for each identified tactic, justifying why it qualifies as manipulation or bias.
          - Rate intensity levels as: ${intensityKeywords}.
          - Your response must be formatted as a properly structured JSON object.
        2. Analysis Framework:
          A) Claims and Logic:
            - Identify unsupported or unverifiable claims.
            - Detect logical fallacies.
            - Evaluate the coherence and consistency of arguments.
          B) Emotional Manipulations:
            - Recognize sensationalist language and hyperbole.
            - Identify emotional manipulation strategies.
            - Spot oversimplifications of complex issues.
            - Detect clickbait elements or misleading headlines.
            - Highlight loaded terms, charged language, or exaggerated claims.
            - Note out-of-context data, meaningless comparisons, or correlation-causation fallacies.
            - Flag anonymous or vague sources and appeals to false authority.
            - Identify instances of missing context or lack of diverse perspectives.
            - Assess fear-mongering, urgency creation, bandwagon effects, or social proof manipulation.
            - Evaluate false scarcity tactics, gaslighting techniques, guilt-tripping, and in-group vs. out-group framing.
          C) Political Ideology:
            - Detect ideological markers or affiliations.
            - Assess the strength and influence of ideological bias.
          D) Promoted Values:
            - Highlight explicitly or implicitly promoted values or agendas.
        3. Output Format:
          {
            ideologicalStrength: ${intensityKeywords},
            emotionalManipulationStrength: ${intensityKeywords},
            ideologicalMarkers: { quote: string, explanation: string, associatedIdeology: string }[],
            promotedValues: { quote: string, explanation: string, associatedIdeology: string }[],
            emotionalManipulations: { quote: string, explanation: string, targetEmotion: string }[],
            logicalFallacies: { quote: string, explanation: string, fallacyType: string }[],
          }
    `,
    user: `
      TEXT TO ANALYZE:
      """
      ${text}
      """
    `
  }

  prompt.system = removeExtraWhitespaces(prompt.system)
  prompt.user = removeExtraWhitespaces(prompt.user)

  return prompt
}

The full code for generating the model instructions can be found here.

Putting it all together

The main component of almost every extension is its popup, and here is how it looks in our case:

// src/scripts/popup.ts

...

const modelController = new ModelController()
const userInputController = new UserInputController()
const renderer = RenderController.getInstance()

async function initializePopup() {
  try {
    const tab = await getActiveTab()
    chrome.runtime.connect({ name: 'popup' })
    const config = await getExtensionConfigFromStorage()

    // initialize user input controller
    await Promise.all([
      ensureContentScriptInjected(tab.id!),
      userInputController.initTools(tab.id!)
    ])

    // bind input handlers
    renderer.setupEventListeners({
      onPageInput: () => userInputController.handlePageContentInput(analyzeContent),
      onImgInput: event => userInputController.handleImageInput(event, analyzeContent),
      registerSelectionInputHandler: () =>
        userInputController.setupEventListenerForTextSelection(analyzeContent),
      onCancel: () => modelController.stopGeneration()
    })

    // initialize model controller
    await renderer.transition(AppStates.modelInit)
    await modelController.initialize(config)
    await renderer.transition(AppStates.modelReady)

    await renderer.transition(AppStates.idle)
  } catch (error) {
    console.error('Error initializing the app:', error)

    const { name, message } = error as Error
    await renderer.transition(AppStates.error, { name, message })
  }
}

const analyzeContent: analyzeContentFn = async content => {
  try {
    if (!content.length) {
      throw new Error('Content and sourceType are required to perform the analysis')
    }

    const startTime = Date.now()
    await renderer.transition(AppStates.analysis, { initial: true })

    const metrics = await extractMetricsFromContent(content, modelController)
    if (metrics.length === 0 && renderer.currentState === AppStates.idle) {
      // user has canceled the operation
      return
    }

    await renderer.transition(AppStates.report)
    const report = await generateReport(metrics)

    await renderer.transition(AppStates.result, { data: report })
    console.info('Result:', report, 'done in:', (Date.now() - startTime) / 1000, 'seconds')
  } catch (error) {
    console.error('Error analyzing the content:', error)

    const { name, message } = error as Error
    await renderer.transition(AppStates.error, { name, message })
  }
}

window.onload = async () => {
  try {
    await initializePopup()
  } catch (error) {
    console.error('Error initializing app:', error)

    const { name, message } = error as Error
    await renderer.transition(AppStates.error, { name, message })
  }
}

I intentionally left some fragments out to avoid unnecessary details. Here is a link to the repository where you can find the full version of the code.

Building is done using Vite, effectively running the TypeScript compiler, minimizing JavaScript, and copying asset files. The packaging puts all the essential files into a .zip archive.

Here are some helpful resources to share your extension with others: Chrome Extensions Best Practices, Creating a great listing page, Publish in the Chrome Web Store.

Afterword

I must admit that, after conducting diverse tests (using different models and texts), LLMs themselves exhibit numerous biases. Various models have shown differing attitudes towards a range of subjects. This is something we will all have to deal with in the future, but that is a topic for another time.

I also noticed some performance issues with most of the models, especially with larger ones (>=7B) on the M1 MacBook Pro. I'm not sure if this is due to WebLLM's potential current code inefficiencies or if this is the current way it functions on such hardware. I did not have time to get into it (maybe next time).

Another observation is that I'm terrible at UI design, and this project gave me a lot of trouble trying to create something that would work (although I still don't like how it looks). If you feel my pain and have a positive experience with any UI/UX design automation tools like uizard, usegalileo, etc, I would love to hear more about it from you.

The main takeaway from this project is that tools like WebLLM and transformers.js open up a ton of opportunities, WebNN is on the way and I'm pretty sure I missed some of the others.

Have fun hacking! and thanks for your attention 👋

How to Create a VSCode Extension That Uses a Machine Learning Model Under the Hood

Maksim Balabash — Mon, 26 May 2025 10:42:00 +0000

After the first release of ChatGPT 3.5, many people started with prompt engineering. But typical prompts are just strings of text. Often, they are pretty lengthy strings, as you occasionally end up adding more instructions and details to the context:

When working with multiple prompts, the dull and same-looking appearance of the strings might make it difficult to focus or lead to forgetting about some outdated text parts, especially after iterating on them for over an hour.

Wouldn't it be nice to add some highlighting like we do for regular code? This is exactly what we are going to do with our extension! (here's a link to it, and here is the code).

We want an extension that:

brings theme-agnostic highlighting to prompts
does the highlighting as close to real time as possible (combining heuristics and ml model)
highlights only those strings that we explicitly requested and does not alter any other strings

What can extensions do?

customize IDE appearance (themes)
extend UI with specialized panels (Docker, Git, etc.)
render custom webpages (Markdown, diagrams, etc.)
add language support (syntax, formatting, intellisense, etc.)
enable runtime-specific debugging tools (Python, Node.js, etc.)

There are many examples and APIs available for each direction.

There are two main restrictions to consider: DOM access and custom style sheets of the VS Code UI are not allowed.

And sure, in case you're wondering, you can run DOOM inside it.

Tokens

Tokens are the smallest units that VSCode recognizes and handles individually:

keywords (if, else, for)
variables (myVariable, userName)
functions (calculateTotal(), getData())
strings ("Hello, world!", 'text')
comments (// my comment, /* comment */)
etc.

They help VSCode understand the code better and offer helpful features like syntax highlighting, autocomplete, and refactoring.

Semantic tokens carry additional information about code's meaning and structure, allowing VSCode to highlight code in a meaningful way:

normally, VSCode only knows basic grammar rules
but with semantic tokens, we can make VSCode behave like something is a variable or a function depending on the context

Here's how it works:

VSCode asks language-specific extensions to analyze the code and provide semantic tokens.
Extensions parse the code, figure out roles for tokens (functions, variables, classes, interfaces, types, parameters), and return these tokens with their semantic meaning.
Once VSCode gets these tokens, it highlights the code. As an additional bonus, it does the job according to the user's current theme, making it theme-agnostic.

But if the prompts are just words in a string, how do we highlight them?

We will identify all strings that begin with our unique marker, which will function as a switch
Then, we will classify each word into 4 classes: actions, objects, subjects, and descriptors
And create semantic tokens for each word in these classes with settings that will make VSCode highlight them differently.

Classifying words using some ML magic

We have a text classification task, and currently, transformer models perform this job the best.

Transformer models are a type of machine learning model that understand the meaning and context of words, sentences, or even images all at once, instead of reading them one by one. They use a mechanism called attention to figure out which parts are most important – like how humans focus on key words when reading. This makes them great for tasks like writing, translating, summarizing, and more.

Since VSCode runs on Electron, the simplest way to use such models in our extension is through the @xenova/transformers library:

gives access to these powerful models without ever leaving JavaScript or TypeScript
models can be run locally in the user's browser, no extra server needed
no need for complex installations, it’s ready-to-use out-of-the-box

Using an LLM would be excessive and negatively impact UX, as even smaller versions require significant storage, and their capabilities are overly complex for our simple task. Therefore, we will create a small model fine-tuned for our task based on DistilBERT.

DistilBERT is a smaller and faster version of a larger model called BERT. BERT is a language model developed by Google and is considered an ancestor of modern LLMs. It performs pretty well in tasks such as text classification, sentiment analysis, filling in missing words, etc.

To create our model, we need to:

Collect the dataset of words for each class
Fine-tune the original model
Ensure it does its job
Convert it to the format recognized by @xenova/transformers
Quantize (compress) the model
Upload it to Hugging Face

This process may take another "How to" article, which we won't cover here. To summarize, here's what I did:

I used GPT-4o to generate words for each class. I managed to produce a couple of thousand for each, even though it required a few iterations
The same GPT-4o helped me with the code for fine-tuning the model
I played with it briefly to ensure it accurately predicted classes. This required several iterations of training the model with different parameters and testing it
Converting it to the required format only takes a single command to execute
Quantizing the model is also a straightforward task that takes a single function call
And finally we simply upload files to make the model available from Hugging Face.

The complete code, training set, and list of used packages related to model creation are here. After completing all the steps, this model essentially enables us to do something like this:

import { pipeline } from "@xenova/transformers";

const classifier = await pipeline("text-classification", "mbalabash/distilbert_subjects_actions_objects_descriptors");

const prediction = await classifier("beautiful");
console.log(prediction)

// OUTPUT:
// [{label: 'DESCRIPTOR', score: 0.9923667311668396}]

The model size is only 66 MB, which is quite suitable for our extension.

Speeding up work

Even though the model should do the job, we can also add a couple of things to improve our extension's UX.

It is possible to add a less compute-intensive method to classify words using regex. This will be our default approach. If this doesn't work, we will use model-based classification.

I once again asked GPT-4 to provide me with suffixes for each word class that would likely match the respective words (and would not conflict with each other).

Here is what I got:

// dictionary.js

{
  subjects: /(or|ian|eer|ster|ist|ite|ee)$/i,
  actions: /(ing|ize|ify|ate|ish|ect)$/i,
  objects: /(tion|sion|ment|ity|ism|ness|hood|age|ance|ence|ery|dom|ship)$/i,
  descriptors: /(ful|ous|ible|able|less|ive|ward|wise|ways|fold|most)$/i
}

Then the usage would be something like this:

// classification.js

function classifyWordUsingHeuristics(word) {
  if (suffixes.subjects.test(word.toLowerCase())) {
    ...

We can also implement an in-memory cache to avoid repeating the classification of words we've already processed.

// classification.js

const PREDICTION_CACHE = new Map()

// in both classification functions, we will check whether we have processed this world before

async function classifyWordUsingModel(word, classifier) {
  if (PREDICTION_CACHE.has(word)) {
    return PREDICTION_CACHE.get(word)
  }
  ...

Putting it all together

To kick off the work on our extension, we only need to use about 2 commands and answer a few yes/no questions in the CLI. After that, it is all set. There is a straightforward official guide for this stage.

We need to fill in specific properties in package.json to inform the world about what (and how) our extension does (see the full list). Think of this as:

our extension’s resume (name, description, version, main, etc.)
and its access control list (what it contributes to VSCode)

There are two essential properties to understand, as they also influence how our extension will function:

activation events: events upon which our extension becomes active
contribution points: declarations that specify which functionalities within VSCode our extension will extend

VSCode extension lifecycle:

VSCode starts and scans all installed extensions’ package.json files
It checks each extension’s activationEvents to see what conditions should trigger activation
Extensions stay inactive until one of their activation conditions is met
Once triggered, VSCode:
- Loads the extension’s main file
- Runs the activate(context) function
Extension does its thing
When the extension is unloaded, deactivate() is called (if it exists).

To minimize the impact on performance, our extension should be activated when the onStartupFinished event is fired. We will not extend any VSCode functionalities so that the contributions property remains empty.

Here is how the main file will appear in our situation:

// extension.js

async function activate(context) {
  // We define that we want to provide semantic tokens for JavaScript, TypeScript, Python, and Go files
  const selector = [
    { language: 'javascript', scheme: 'file' },
    { language: 'typescript', scheme: 'file' },
    { language: 'python', scheme: 'file' },
    { language: 'golang', scheme: 'file' }
  ]
  // We define the types of semantic tokens that we will provide
  const legend = new vscode.SemanticTokensLegend(
    ['operator', 'keyword', 'namespace', 'variable', 'number'],
    ['declaration']
  )

  const classifier = await initializeModel()

  const provider = {
    _eventEmitter: new vscode.EventEmitter(),

    get onDidChangeSemanticTokens() {
      return this._eventEmitter.event
    },

    async provideDocumentSemanticTokens(document) {
      const tokensBuilder = new vscode.SemanticTokensBuilder(legend)

      const text = document.getText()
      const blocks = text.matchAll(blockRegex) // Finds all strings that have the marker (#!promptskeeper)

      for (let block of blocks) {
        const classifiedWords = await extractClassifiedWords(block[0], classifier)
        generateSemanticTokens(classifiedWords, block, document, tokensBuilder)
      }

      return tokensBuilder.build()
    }
  }

  context.subscriptions.push(
    vscode.languages.registerDocumentSemanticTokensProvider(selector, provider, legend)
  )

  context.subscriptions.push(
    vscode.workspace.onDidChangeTextDocument(event => {
      // Check if tokens should be recalculated after changing the file
      if (
        selector.some(
          s =>
            vscode.languages.match(s, event.document) && event.document.getText().includes(MARKER)
        )
      ) {
        // Signal that tokens need to be recalculated
        // This will cause VS Code to request new tokens
        provider._eventEmitter.fire()
      }
    })
  )
}

function deactivate() {}

Before classifying and highlighting words in prompts, we must initialize the model:

// classification.js

const MODEL_ID = 'mbalabash/distilbert_subjects_actions_objects_descriptors'
const MODEL_CACHE_DIR = join(
  contextEnv.HOME || contextEnv.USERPROFILE || '.',
  '.cache',
  'promptskeeper-vscode-highlighting-extension',
  'models'
)

if (!existsSync(MODEL_CACHE_DIR)) {
  mkdirSync(MODEL_CACHE_DIR, { recursive: true })
}

async function getModel(pipeline, progress) {
  return await pipeline('text-classification', MODEL_ID, {
    cache_dir: MODEL_CACHE_DIR,
    local_files_only: false, // download the model if not found in cache
    progress_callback: chunk => {
      if (chunk.status) {
        progress.report({ message: chunk.status })
      }
      if (chunk.progress) {
        progress.report({
          message: `Downloading: ${Math.round(chunk.progress)}%`
        })
      }
    }
  })
}

async function initializeTransformers() {
  process.env.HF_HOME = MODEL_CACHE_DIR
  process.env.TRANSFORMERS_CACHE = MODEL_CACHE_DIR

  try {
    const { pipeline, env } = await import('@xenova/transformers')
    env.cacheDir = MODEL_CACHE_DIR
    env.allowLocalModels = true

    return await vscode.window.withProgress( // show beautiful message in VSCode
      {
        location: vscode.ProgressLocation.Notification,
        title: 'Initializing model...',
        cancellable: false
      },
      async progress => {
        progress.report({ message: 'Downloading...' })
        return await getModel(pipeline, progress)
      }
    )
  } catch (error) {
    console.error('Model initialization error:', error)
    vscode.window.showErrorMessage('Failed to load model: ' + error.message)
    return null
  }
}

Now that our model is in place, we can jump to the function that extracts the words we want to highlight:

// classification.js

async function extractClassifiedWords(prompt, classifier) {
  if (!classifier) {
    vscode.window.showErrorMessage('Word categorization failed: model not initialized')
    return
  }

  const { text, categories } = preparePromptForClassification(prompt)

  try {
    const words = splitTextIntoWords(text)
    const ambiguousWords = []

    for (const word of words) {
      if (isExceptionWord(word)) {
        continue
      }

      // Try to classify the word using heuristics first
      const { confidence, category } = classifyWordUsingHeuristics(word)
      if (confidence === 0) {
        ambiguousWords.push(word)
      } else {
        categories[category].push(word)
      }
    }

    // Handle ambiguous words using our model
    const BATCH_SIZE = 6
    for (let i = 0; i < ambiguousWords.length; i += BATCH_SIZE) {
      const batch = ambiguousWords.slice(i, i + BATCH_SIZE)
      const predictions = await Promise.all(batch.map(word => classifyWordUsingModel(word, classifier)))

      predictions.forEach((prediction, index) => {
        const word = batch[index]
        if (typeof prediction.category === 'string' && prediction.category.length > 0) {
          categories[prediction.category].push(word)
        }
      })
    }
  } catch (error) {
    vscode.window.showErrorMessage(`Word categorization failed: ${error.message}`)
    console.error('Word categorization error:', error)
  }

  return categories
}

This gives us words grouped into classes:

{
  subject: ["word1", "word2"],
  action: [...],
  object: [...],
  descriptor: [...],
  marker: ["#!promptskeeper"]
}

With these words, we can find their exact positions in the document and generate semantic tokens:

// highlighter.js

const HIGHLIGHTING_GROUPS = {
  marker: {
    tokenType: 'operator',
    tokenModifiers: ['declaration']
  },
  subject: {
    tokenType: 'variable',
    tokenModifiers: ['declaration']
  },
  action: {
    tokenType: 'keyword',
    tokenModifiers: ['declaration']
  },
  object: {
    tokenType: 'namespace',
    tokenModifiers: ['declaration']
  },
  descriptor: {
    tokenType: 'number',
    tokenModifiers: ['declaration']
  }
}

function generateSemanticTokens(classifiedWords, block, document, tokensBuilder) {
  const prompt = block[0]
  const categories = Object.entries(classifiedWords)

  for (const [category, words] of categories) {
    const matches = prompt.matchAll(getRegexForCategoryWords(category, words))

    for (const match of matches) {
      // Get the exact word from the match
      const keyword =
        typeof match[0] === 'string' && match[0].length > (match[1] || '').length
          ? match[0]
          : match[1]

      // Get index of word in matched string
      const appendix = (match[0] || '').indexOf(keyword)

      // Get the index of word in the prompt (source string)
      const matchStart =
        appendix !== -1 ? block.index + match.index + appendix : block.index + match.index

      // Get the exact positions in the document
      const startPos = document.positionAt(matchStart)
      const endPos = document.positionAt(matchStart + keyword.length)

      // Generate semantic token
      tokensBuilder.push(
        new vscode.Range(
          new vscode.Position(startPos.c, startPos.e),
          new vscode.Position(endPos.c, endPos.e)
        ),
        // Makes words from the corresponding classes look like their tokenType and tokenModifiers
        HIGHLIGHTING_GROUPS[category].tokenType,
        HIGHLIGHTING_GROUPS[category].tokenModifiers
      )
    }
  }
}

This is it! We have covered the core logic of our extension. Here is the repository with the full code there.

Making sure everything works

There are two ways to test our extension:

Run it with the debugger (that works perfectly for development purposes)
- How: press the F5 key
Build the .vsix package and install it directly (to ensure that everything is ready before publishing it)
- How: run vsce package command in the CLI, then open the command palette in VSCode and find command Extensions: Install from VSIX, then choose the extension's .vsix file

In addition, we can add regular unit tests or even e2e tests (see the official guide).

It's important to note that because the extension runs in a separate instance of VSCode, some exceptions may be swallowed in that instance. This could lead to situations where the extension doesn't function properly, and you have no idea why. I've encountered this a couple of times due to syntax errors that went unnoticed since I did not use TypeScript for this project.

Sharing it with others

The good part of using plain JavaScript is that I do not need to do any bundling at all. It just works.

But it's not always worth it, especially because there is a well-detailed official guide for using TypeScript and bundling the extensions (has examples for esbuild and webpack).

Having said this, I need to prune my dependencies before packaging our extension (otherwise, all of my devDependencies will also be included in the package):

// package.json
...
"scripts": {
  "test": "jest",
  "package": "npm prune --production && vsce package",
  "publish": "npm prune --production && vsce publish"
},
...

Publishing a VSCode extension is a straightforward process and is very similar to publishing an npm package.

You will need the vsce tool and a personal access token. The whole process, from start to finish, is thoroughly documented in the official Publishing Extensions Guide.

Afterword

As you have seen, the result is far from ideal. It operates word by word and cannot recognize phrasal verbs, compound expressions, etc. But I had a great time learning about VSCode extensions and fine-tuning BERT.

Sometimes, it can be tedious or even painful (like when I was collecting the training set), but software engineering gives those moments of magic when you can turn an idea in your head into something that exists and works (even if it's not perfect).

By the way, you can do much more with @xenova/transformers.js in the browser. There are plenty of opportunities to experiment with in natural language processing, computer vision, audio, and more.

I wish you more joy in your journey as a software engineer!

Thanks for your attention! 👋

How to respond to growing supply chain security risks?

Maksim Balabash — Sun, 03 Apr 2022 10:07:33 +0000

Each month since October 2021 we have a new security incident in the npm supply chain.

October 2021 (ua-parser-js)
November 2021 (coa, rc)
December 2021 (qr.js, discord-lofy + discord-selfbot-v14)
January 2022 (colors + faker.js)
February 2022 (25 malicious packages)
March 2022 (azure-sdk-v4 + azureazure)
March bonus: Protestware
March bonus: The Factory

My point is: that we all have to review and adapt how we work with dependencies taking into account supply chain security risks.

At this point, it's unsafe to ignore supply chain risks and there is nowhere to hide from them in the future.

I started following this problem from the discussion at npm about making install scripts opt-in. But install scripts are not the only threat, there are more ways for malicious actors:

Malicious code injection: During the development process an attacker could replace URLs in a lock file or add malicious code (it is dangerous because it may be hard to catch in PR review).
Malicious package release: An attacker may publish malicious packages and hence trick other users into installing or depending on such packages.
Social Engineering: An attacker may manipulate a maintainer to hand over sensitive information.
Account Takeover: An attacker may compromise the credentials of a maintainer to inject malicious code under the maintainer’s name.
Ownership transfer: An attacker can show enthusiasm to maintain popular abandoned packages and transfer the ownership of a package.
Remote execution: An attacker may target a package by compromising the third-party services used by that package.

There are a lot of papers about detecting and mitigating those risks.

One specific paper (What are Weak Links in the npm Supply Chain) inspired me to create sdc-check.

As a big believer in the power of people collaboration, I am sure that the open-source community is our key to mitigating those supply chain risks and threats.

And it is happening right now. Github is opening the GitHub Advisory Database to community submissions. Awesome community NodeSecure builds cool things like scanner and js-x-ray. There are also lockfile-lint, LavaMoat, Jfrog-npm-tools (and I am sure there is more).

As an open-source maintainer, you could take action:

make sure you are following security best practices

As an open-source consumer, you have to think about:

what "extra" you could get with the next dependencies upgrade

Also, try to add additional safety step using a tool like sdc-check / lockfile-lint / scanner / etc.

thank you for attention 🙌

Share your webperf best practices

Maksim Balabash — Mon, 30 Nov 2020 07:46:20 +0000

Hi devs 👋

Last few weeks I have been working on a project that is not doing pretty well with web performance.

Although in the public field we constantly hear that web performance is important, I haven't seen many teams that have a well-built web performance optimization culture.

It would be great if we could share our examples and best practices.

I came up with a survey that I hope might be interesting. If you would like to describe in a detailed manner your processes for monitoring and optimizing web performance, please share them with us (it could be really useful for others).

Poll:

1. How many people are in your project team?

2. Which field is your project related to? (e-commerce, health, education, etc)

3. Describe practices and tools which you use on your project to improve web performance?

4. Which of the negative consequences of poor web performance are the most critical for your project? (you can list own options)

a) users are less engaged and are less likely to buy/use your product;
b) it impacts negatively the perception of your brand;
c) it leads to exclusion of some users.

5. Do you know how web performance correlates with conversion rates and/or other product metrics for your project?

6. Do you have any web performance targets that you are aiming for?

7. Is there any kind of web performance budget for your project?

8. Have you ever faced a regressive web performance degradation issue? (after merging some changes from pr or after dependencies upgrade, etc)

9. Do you use any kind of web performance monitoring tool on your project?

10. Do you measure web performance on your real users' devices?

11. Do you track your own app metrics using the Custom Metrics API?

That's all 🎉

Thanks for your attention 🙏

I would be really glad to speak with you if you send me your interesting cases and/or problems from your experience.