DEV Community: ptvty

Next-Level Web Applications with On-Device Generative AI: A Look at Google Chrome's Built-In Gemini Nano LLM

ptvty — Sat, 13 Jul 2024 18:34:18 +0000

Web development is on the brink of a significant transformation with Google Chrome Canary's latest experimental feature, a new tool called the window.ai API, allowing websites to harness the power of on-device generative AI. With Google’s Gemini Nano AI model built into the browser, websites can offer smarter, more personalized experiences directly on the user's device. Let's dive into what this means and how you can use it to supercharge your web applications.

Meet Gemini Nano

Gemini Nano is a compact yet powerful AI model from Google. It is the same model used in some Google Pixel phones for offline AI features. Its small size and impressive capabilities make it perfect for on-device applications, ensuring users benefit from advanced AI without needing to be online.

What is the `window.ai` API?

Google Chrome Canary consistently introduces exciting new features for developers, and the window.ai API is no exception. This API enables your website's JavaScript code to interact directly with Gemini Nano, a model that operates on the user's device. This means all AI tasks are performed locally using the computer’s GPU, ensuring no data is sent over the internet. This approach significantly enhances privacy and allows for offline functionality.

How to Use the `window.ai` API

Getting started with the window.ai API is straightforward. Here’s how you can integrate it into your website:

Install Google Chrome Canary: Ensure you have the latest version of Google Chrome Canary installed.
Enable the Experimental Feature: Follow the instructions to enable the window.ai experimental feature.
Check for API Availability: In your code, verify that window.ai is defined to ensure the Local AI API is available in the user’s browser.
Start Sending Prompts: Once you know the API is available, you can begin sending prompts to Gemini Nano and receive responses using ai.createTextSession() and session.prompt() APIs.
Utilize the AI's Response: Use the AI's response in various ways, such as displaying it to the user, updating your UI, or for more complex processing.

if (window.ai) {
    logOceanPoem();
} else {
    console.log("The window.ai API is not available on this browser.");
}

async function logOceanPoem(prompt) {
    try {
        const session = await ai.createTextSession({
            temperature: 0,
            topK: 1
        });
        const poem = await session.prompt("Write a poem about the ocean.");
        console.log(poem);
    } catch (error) {
        console.error("Error generating text:", error);
    }
}

// => ' In the realm of vast and boundless blue,
//     Where secrets hide and mysteries brew,
//     There lies a realm of ...'

Some Exciting Use Cases

The possibilities with on-device generative AI are endless. Here are a few exciting ways you can use it:

Privacy-Focused Apps

Use local AI for sensitive applications like health, finance, or where strict data protection policies are required, ensuring complete privacy and data security.

Offline-First PWAs

Enhance chatbots and virtual assistants with more natural and responsive interactions, even when offline.

Enhanced User Inputs

Create smart user interfaces by integrating AI into the existing form components. For example, a select box can suggest the most relevant options if a user types an invalid option, rather than just displaying "Not found."

// The user tries to type in "android developer" in a searchable select box to fill his occupation, not knowing the granularity of the available options

await session.prompt(`which are the 3 closest phrases to "android" from the following phrases:
1. DevOps Engineer
2. SEO specialist
3. Dentist
4. Cashier
5. Mobile Developer
6. Web Developer
7. Carpenter
8. Desktop Developer`);

// => ' The three closest phrases to "Android" from the given phrases are: 1. Mobile Developer 2. Web Developer'

Assisted Writing

Enhance text areas with AI writing assistance, allowing text summarization and rephrasing without sending user data to external servers.

await session.prompt(`Rephrase this sentence: 
Facebook suggested your profile, I looked at your profile and I found your story inspiring.`);

// => ' Facebook displayed your profile to me, and upon reviewing it, I found your story to be incredibly inspiring.'

Accessibility and Improved Website Navigation

Create user interfaces that people with disabilities can use more comfortably. For example, a prompt box can intelligently recommend the most relevant actions, pages, sub-menus, and settings based on users' intent. LLMs can easily understands different words that users might use to express the same idea.

const choices = ['Like post', 'Save post', 'Share post', 'Block user', 'Follow user'];

await session.prompt(`Which choice is the most relevent to "I do not want to see any posts from this user again!":
${choices.map((choice, idx) => `${idx + 1}. ${choice}\n`)}
`)

// => ' The most relevant choice to the command "I do not want to see any posts from this user again!" is **"Block user"**.'

await session.prompt(`Which choice is the most relevent to "I want to see future posts from this user!":
${choices.map((choice, idx) => `${idx + 1}. ${choice}\n`)}
`)

// => ' The most relevant choice is **5: Follow user.**'

Improved Text Search

// When a "Find in page" fails to find an exact match, try to search for similar words!

await session.prompt(`Generate 10 words similar to "battery runtime", in the context of "smart phones"`);

// => '1. Battery life 2. Battery endurance 3. Battery longevity 4. Battery capacity 5. Battery duration 6. Battery power 7. Battery energy 8. Battery performance 9. Battery efficiency 10. Battery usage'

Chat With Your Data

Although the Gemini Nano is not the best model for factual reasoning, it can handle basic questions. For example, it can allow users to ask questions about the data in a table.

await session.prompt(`Who is the tallest person in the following list:
1. Alice, Weight: 52 kg, Height: 169 cm.
2. Bob, Weight: 71 kg, Height: 174 cm.
3. Eve, Weight: 66 kg, Height: 172 cm.
`);

// => 'The tallest person in the list is Bob, who weighs 71 kg and is 174 cm tall.'

Tips for Prompt Engineering

To get the best results from the AI, follow these prompt engineering tips:

Play with Parameters: For factual questions, set the temperature to 0, and set TopK to 1 for more predictable and accurate answers.
Use Line Breaks: Keep related data on the same line to maintain context.
Use Parentheses: Add elaborations and clarifications in parentheses to vague words and phrases.
Main Question First: Put the main question at the beginning of the prompt, followed by related data.
Response Parsing: The model usually uses Markdown to format its response. Use this to your benefit. For example, if you're looking for a few keywords but the response is long, just take the bold words (inside double asterisks).

Why On-Device AI is Awesome

Privacy and Security: All data stays on the user's device, enhancing privacy and reducing the risk of data breaches.
Offline Functionality: Apps work seamlessly without an internet connection, providing uninterrupted service.
Better Performance: Using the device's GPU can lead to faster response times and a more responsive application.
Cost Savings: Less reliance on cloud-based services can lower your operational costs.

Things to Keep in Mind

While the window.ai API is fantastic, there are a few things to consider:

Device Compatibility: Currently, this feature is exclusive to Google Chrome Canary. Ensure compatibility with other browsers and devices that may not have a GPU available.
Resource Use: Running AI models can be heavy on resources, especially on lower-end hardware.
Model Limits: On-device models might not be as powerful as their cloud-based counterparts, so balance functionality accordingly.

Conclusion

The window.ai API in Google Chrome Canary introduces a groundbreaking tool that harnesses the capabilities of on-device generative AI for web development. By utilizing local computation, it can enhance privacy, performance, and user experience. While currently an experimental feature, the potential for wider adoption remains uncertain. Nonetheless, exploring its capabilities now can pave the way for developing smarter, more responsive, and secure web applications in the future.

Effortless GraphQL in Next.js: Elevate Your DX with Codegen and more

ptvty — Tue, 25 Jun 2024 22:02:25 +0000

Introduction

GraphQL endpoints are gaining popularity for their flexibility, efficient data fetching, and strongly typed schema. Putting these powers in hands of API consumers will elevate the Developer Experience (DX) and leads to building robust and maintainable applications. Combining Next.js, GraphQL, and TypeScript offers a powerful stack that can significantly improve your productivity and code quality.

In this article, I will walk you through setting up a Next.js project that uses Hasura for an instant GraphQL endpoint. We will demonstrate how to achieve code auto-completion and type hinting in VSCode using a few tools and extensions and one of GraphQL's superpowers - introspection. By the end of this guide, you'll have a seamless development setup that boosts your efficiency and code accuracy.

Outline

In this article we will walk through the following steps:

Creating a sample GraphQL endpoint, we will use Hasura Cloud.
Installing prerequisites and creating a fresh Next.js app.
Minimal wiring of a GraphQL endpoint in our Next.js app.
Installing extensions and tools for GraphQL IntelliSense in VS Code.
Setting up tools for typed variables and responses in useQuery hook.
Enhancing npm run dev to concurrently run all the required tools.

Setting Up Hasura

Visit Hasura Cloud and create an account. Click "New Project" in the projects tab.

Select desired options and proceed. Your project will be created instantly, click "Launch Console". Switch to the "DATA" tab and click "Connect Neon Database". Neon is essentially a Postgres database on a serverless platform, you can read more on the Neon's website.

Wait a while and click "default" database in the sidebar. Click the first template, "👋 Welcome to Hasura!" in the "Template Gallery" page, then click "Install Template".

We are almost done, just head to the "API" tab and copy both GrphQL endpoint URL, and the string in front of x-hasura-admin-secret request header. To manage our Hasura endpoint URL and admin secret, we'll use environment variables.

Setting Up Prerequisites and creating a fresh Next.js Project

Ensure you have the following:

Basic understanding of Next.js and GraphQL.
Node.js and npm/yarn installed.
Visual Studio Code installed.
A running Hasura instance (I'll use Hasura Cloud's free tier).

Now, let's create a new Next.js project. We will install the necessary dependencies in the next steps. Initialize a New Project:

npx create-next-app@latest my-graphql-app # Proceed with default choices
code my-graphql-app # Open the project in Visual Studio Code

Configuring for Minimal GraphQL Endpoint Access

Switch to VS Code, create a blank file, .env.local, this is Next.js's default file for storing dev environment variables. Add your Hasura Cloud's endpoint URL and the x-hasura-admin-secret value:

NEXT_PUBLIC_GQL_URL="https://charmed-finch.hasura.app/v1/graphql/"
NEXT_PUBLIC_HASURA_ADMIN_SECRET="FHU82EdNTGfMSExUkip4TUtLtLM1T..."

Note that environment variables prefixed with NEXT_PUBLIC_ are exposed to the client-side. I'm using Hasura admin secret for the sake of simplicity, and you should use a proper authentication method for your real project, see Hasura's Authentication and Authorization for more info.

We will urql a rather minimal GraphQL client:

npm i urql

Open app/page.tsx and replace the default content with this minimal query:

"use client"
import { Client, Provider, cacheExchange, fetchExchange, gql, useQuery } from "urql";

const Customer = () => {
  const CUSTOMERS_QUERY = gql`
    query CustomersQuery {
      customer { first_name }
    }
  `;
  const [{ data }] = useQuery({
    query: CUSTOMERS_QUERY,
  });
  return JSON.stringify(data);
}

const client = new Client({
  url: process.env.NEXT_PUBLIC_GQL_URL ?? '',
  exchanges: [cacheExchange, fetchExchange],
  fetchOptions: {
    headers: {
      "x-hasura-admin-secret": process.env.NEXT_PUBLIC_HASURA_ADMIN_SECRET ?? ''
    }
  },
});

export default function Home() {
  return <Provider value={client}>
    <Customer />
  </Provider>;
}

Check your app in a browser, you should see the raw data.

Setting Up Visual Studio Code for GraphQL IntelliSense

Install GraphQL: Language Feature Support VS Code extension
The installed extension expects a GraphQL schema file, so we will use graphqurl to introspect the GraphQL endpoint and download the full schema into a schema.graphql file. We will also use dotenv-cli package to pass variables from the .env.local file to the graphqurl command.
Install required packages:

npm i -D graphqurl dotenv-cli

And add to your package.json scripts:

"dev:introspect": "dotenv -e .env.local -- npm run dev:_introspect",
"dev:_introspect": "gq %NEXT_PUBLIC_GQL_URL% -H \"X-Hasura-Admin-Secret: %NEXT_PUBLIC_HASURA_ADMIN_SECRET%\" --introspect > schema.graphql",

Run npm run dev:introspect and wait for the schema.graphql file to be generated in the project's root directory.
Create a blank file graphql.config.ts with the following content to hint the extension on where the schema file is and in which files the IntelliSense should work:

export default {
    schema: './schema.graphql',
    documents: ['**/*.{graphql,js,ts,jsx,tsx}'],
};

Restart VS Code and test IntelliSense by editing the query:

Setting Up Codegen for Typed Query Variables and Responses

Install dependencies:

npm install -D @graphql-codegen/cli @parcel/watcher

Add codegen.ts file to the project's root directory:

import { CodegenConfig } from '@graphql-codegen/cli';

const config: CodegenConfig = {
  schema: {
    [process.env.NEXT_PUBLIC_GQL_URL?? '']: {
      headers: {
        "x-hasura-admin-secret": process.env.NEXT_PUBLIC_HASURA_ADMIN_SECRET ?? '',
      },
    },
  },
  documents: ['**/*.{ts,tsx}'],
  generates: {
    './__generated__/': {
      preset: 'client',
    }
  },
  ignoreNoDocuments: true,
};

export default config;

Add to your package.json scripts:

"dev:codegen": "graphql-codegen --require dotenv/config --config codegen.ts dotenv_config_path=.env.local --watch",

Use generated graphql function instead of urql's gql literal:

import { graphql } from '../__generated__/gql'

const CUSTOMERS_QUERY = graphql(`
  query CustomersQuery {
    customer { first_name }
  }
`);

Great, now run npm run dev:codegen, edit the query, save the file, and wait a moment for the codegen to re-generate the type files. Enjoy typed variables and query responses.

Running Everything Together

We'll use concurrently to run the introspection, codegen, and Next.js dev server all together.

Install Dev Dependencies:

npm install -D concurrently

Rename the original dev script and add concurrently script to package.json as the new dev script:

"dev:next": "next dev",
"dev": "concurrently 'npm:dev:next' 'npm:dev:codegen' 'npm:dev:introspect'",

Run the dev server as before:

npm run dev

Benefits of Enhanced DX

By following these steps, you achieve the following benefits:

Code Auto-completion: Automatically suggests GraphQL queries and schema details as you type, reducing the need for constant reference checks.
Type Safety: Ensures that your GraphQL queries and mutations are type-safe, reducing runtime errors and improving maintainability.
Reduced Errors: With auto-completion and type hinting, the likelihood of making syntax or schema-related errors decreases significantly.

Conclusion

In this guide, we've shown you how to set up a Next.js project with GraphQL using urql, configure your environment for optimal DX with Hasura, and leverage TypeScript and VSCode tools to improve your coding experience. By integrating these technologies, you can streamline your development workflow, reduce errors, and build more robust applications.

By adopting these practices, you'll enhance your development workflow and enjoy a more productive and efficient coding experience. Happy coding!

Enforce Architectural Policies in JavaScript with ESLint

ptvty — Wed, 12 Jun 2024 11:46:34 +0000

As software projects grow in size and complexity, maintaining a clean and organized codebase becomes crucial. In large JavaScript and TypeScript codebases, ensuring consistent architectural practices can reduce technical debt and improve maintainability. ESLint's no-restricted-imports rule offers a powerful tool for enforcing these policies. This article explores how you can leverage this rule to create a cleaner codebase.

ESLint and `no-restricted-imports`

ESLint is a popular linter for JavaScript and TypeScript, providing a framework for identifying and reporting on patterns found in the code. Among its many rules, no-restricted-imports allows developers to specify import patterns that should be avoided within the codebase. By configuring this rule, you can:

Prevent the use of specific modules or files: This is useful for deprecating old utilities or avoiding problematic dependencies. Although this is not the focus of this article, you can read more about this in the ESLint Docs.
Enforce architectural boundaries: By restricting imports based on directory structure, you can enforce module boundaries, ensuring that the codebase complies with your architectural policy.

Some examples of such architectural policies include:

Feature modules should not directly import from other feature modules.
UI components should not directly access data layer modules, but only through a DAL module.
Modules should not import anything from a service directory except items exported from service/index.ts as service gateway.
Modules inside Services, Controllers, Providers, or any other conceptual components should honor their relations within the codebase.

Let's go over two simplified examples to grasp how it can help in your projects.

Basic Example

Given the following tree, let's say you implemented an internal service with a dozen of interconnected files including classes, functions, variables, etc. Now you do not want other services to be able to import any of the bells and whistles from the service directory. They should only import provided items from the service's API, api.js.

src
├── main.js
└── internal
    ├── api.js
    ├── constants.js
    ├── bells.js  
    └── whistles.js

// internal/constants.js
export const Pi = 3.14;

// internal/api.js
import { Pi } from "./constants.js";

export function getPi() {
    return Pi;
}

Setting Up ESLint

To get started, you need to have ESLint installed in your project. If you haven't already:

npm i eslint -D

Add a new script to your package.json for convenience:

"lint": "eslint"

Enforcing Architectural Policies

Next, configure your eslint.config.js file to include the no-restricted-imports rule:

export default [
  {
    rules: {
      'no-restricted-imports': [
        'error',
        {
          patterns: [
            {
              group: ['*/internal/**', '!*/internal/api.js'],
              message: 'Do not import from internal service directly. Use the public API instead.'
            }
          ]
        }
      ]
    }
  }
];

Strings you see in the group array follow gitignore syntax, allowing you to include and exclude files and directories as needed.

Results

You should expect an error if you import directly from any file in the internal directory except api.js.

// main.js
import { Pi } from "./internal/constants.js"; // ❌ Lint should fail
import { getPi } from "./internal/api.js";    // ✅ Lint should pass

Given a non-compliant usage you should get and error when you run npm run lint:

// main.js
import { Pi } from "./internal/constants.js";

> eslint
src\main.js
  1:1  error  './internal/constants.js' import is restricted from being used by a pattern. 
Do not import internal modules directly. Use the public API instead  no-restricted-imports

Hierarchal Architecture Example

For large codebases, it's crucial to enforce architectural boundaries. For example, you might want to ensure that a low-level module does not import from a high-level module. Additionally, you want to ensure that a high-level module does not import directly from a low-level module but through a mid-level module.

Enforcing Architectural Policies

To achieve these policies, we need to use the files and excludes properties in ESLint's flat config file and combine them with group patterns in no-restricted-imports:

hierarchy
├── high
|   ├── ui.js
|   └── ...
├── mid
|   ├── api.js
|   └── ...
└── low
    ├── constants.js
    └── ...

export default [
  {
    ignores: ['src/hierarchy/mid/**/*.js'],
    rules: {
      'no-restricted-imports': [
        'error',
        {
          patterns: [
            {
              group: ['**/low/**'],
              message: 'Low level modules can only be imported in mid level modules'
            },
          ]
        }
      ]
    }, 
  },

  {
    ignores: ['src/hierarchy/high/**/*.js'],
    rules: {
      'no-restricted-imports': [
        'error',
        {
          patterns: [
            {
              group: ['**/mid/**'],
              message: 'Mid level modules can only be imported in high level modules'
            },
          ]
        }
      ]
    }, 
  },

  {
    files: ['src/hierarchy/mid/**/*.js', 'src/hierarchy/low/**/*.js'],
    rules: {
      'no-restricted-imports': [
        'error',
        {
          patterns: [
            {
              group: ['**/high/**'],
              message: 'High level modules can not be imported in low or mid level modules'
            },
          ]
        }
      ]
    }, 
  },
];

Results

Here we are exporting 3 instructions with similar rules. In this configuration:

Any attempt to import from any of the files inside the low directory will be flagged, except by files in the mid directory.
Any attempt to import from any of the files inside the mid directory will be flagged, except by files in the high directory.
Any attempt by files inside low or mid directory to import from any of the files inside the high directory will be flagged.

Conclusion

Enforcing architectural policies in your codebase is essential for maintaining consistency, reducing technical debt, and ensuring long-term maintainability. ESLint's no-restricted-imports rule is an effective tool to help achieve these goals. By configuring this rule, you can:

Prevent unwanted dependencies by restricting specific import patterns.
Ensure that all developers adhere to the defined architectural boundaries.
Simplify code reviews by automating the enforcement of coding standards.

Involving your development team in defining these rules ensures they are practical and aligned with best practices. Regularly review and update your ESLint configuration as your project evolves to keep your codebase robust and well-organized.

By leveraging ESLint's no-restricted-imports rule, you can create a cleaner, more maintainable codebase, ultimately improving the overall quality of your software projects.

Let's Talk!

I wrote this article as my first post on DEV, to share a practical approach to maintaining a clean and organized codebase, which is something I've found crucial in my own projects. I hope you find it helpful in your development work. If you have any questions or thoughts, please leave them in the comments. I'd love to hear your feedback and continue the discussion!

DEV Community: ptvty

Next-Level Web Applications with On-Device Generative AI: A Look at Google Chrome's Built-In Gemini Nano LLM

Meet Gemini Nano

What is the window.ai API?

How to Use the window.ai API

Some Exciting Use Cases

Privacy-Focused Apps

Offline-First PWAs

Enhanced User Inputs

Assisted Writing

Accessibility and Improved Website Navigation

Improved Text Search

Chat With Your Data

Tips for Prompt Engineering

Why On-Device AI is Awesome

Things to Keep in Mind

Conclusion

Effortless GraphQL in Next.js: Elevate Your DX with Codegen and more

Introduction

Outline

Setting Up Hasura

Setting Up Prerequisites and creating a fresh Next.js Project

Configuring for Minimal GraphQL Endpoint Access

Setting Up Visual Studio Code for GraphQL IntelliSense

Setting Up Codegen for Typed Query Variables and Responses

Running Everything Together

Benefits of Enhanced DX

Conclusion

Enforce Architectural Policies in JavaScript with ESLint

ESLint and no-restricted-imports

Basic Example

Setting Up ESLint

Enforcing Architectural Policies

Results

Hierarchal Architecture Example

Enforcing Architectural Policies

Results

Conclusion

Let's Talk!

What is the `window.ai` API?

How to Use the `window.ai` API

ESLint and `no-restricted-imports`