DEV Community: Sebastian Wessel

Conclusion on using a QuickJS sandbox

Sebastian Wessel — Sun, 07 Jul 2024 13:24:06 +0000

Creating the QuickJS package has been a journey in addressing real needs I've encountered in my development career, especially as AI and LLMs become more prevalent in our workflows. It's designed to empower developers to push the boundaries of what's possible in JavaScript applications without compromising on security or code quality.

Whether you're building a complex web application, a desktop app, an AI-powered coding platform, or simply need a safe space to run and test untrusted scripts with Node.js-like capabilities, I believe this package offers the tools you need to execute JavaScript with confidence.

I encourage you to give the QuickJS package a try in your next project, especially if you're working with AI and LLMs. Experience the peace of mind that comes with secure code execution, coupled with the convenience of Node.js module support and robust testing capabilities.

As we continue to explore the frontiers of AI and secure code execution, the QuickJS package stands ready to support your innovative projects. I'm always open to feedback and suggestions from fellow developers. Let's make secure JavaScript execution easier and more accessible together, paving the way for the next generation of AI-powered applications!

Documentation: https://sebastianwessel.github.io/quickjs/
GitHub Repository: https://github.com/sebastianwessel/quickjs
Submit Feedback: Create an Issue on GitHub

Running Tests Inside QuickJS with TestRunner

Sebastian Wessel — Sun, 07 Jul 2024 13:23:56 +0000

The TestRunner is a lightweight testing library that allows you to write and run tests for JavaScript code with ease. It provides a simple interface for defining test suites, tests, and hooks with support for asynchronous code and configurable timeouts. The library also includes Chai's expect function for assertions.

Benefits of Running Tests in a Sandbox

Running tests in a sandbox environment like QuickJS provides several benefits:

Isolation: Each test runs in a separate environment, ensuring no side effects between tests.
Security: A sandboxed environment limits the potential for malicious code to affect the host system.
Consistency: Ensures that tests are run in a controlled environment, leading to more consistent and reliable results.

Importing the Library

To use the TestRunner library in your tests, you need to import it. You also need to ensure that the result of runTests is exported.

import 'testRunner'

Writing Tests

Developers should feel comfortable using the TestRunner as it follows the general approach used in the JavaScript testing ecosystem, similar to Mocha or Jasmine.

Defining a Test Suite

A test suite is defined using the describe function. Inside a test suite, you can define multiple tests using the it function.

describe('Sample Test Suite', () => {
  it('should pass this test', () => {
    expect(true).to.be.true;
  });

  it('should fail this test', () => {
    expect(false).to.be.true;
  });
});

Using Hooks

Hooks are functions that run before or after tests in a suite. The available hooks are:

beforeAll: Runs once before all tests in the suite.
afterAll: Runs once after all tests in the suite.
beforeEach: Runs before each test in the suite.
afterEach: Runs after each test in the suite.

describe('Test Suite with Hooks', () => {
  beforeAll(() => {
    console.log('Running beforeAll hook');
  });

  afterAll(() => {
    console.log('Running afterAll hook');
  });

  beforeEach(() => {
    console.log('Running beforeEach hook');
  });

  afterEach(() => {
    console.log('Running afterEach hook');
  });

  it('should pass this test', () => {
    expect(true).to.be.true;
  });
});

Asynchronous Tests and Hooks

The TestRunner supports asynchronous tests and hooks. Simply return a promise or use async/await in your test or hook functions.

describe('Asynchronous Test Suite', () => {
  beforeAll(async () => {
    await new Promise(resolve => setTimeout(resolve, 1000));
  });

  it('should pass this async test', async () => {
    const result = await new Promise(resolve => setTimeout(() => resolve(true), 1000));
    expect(result).to.be.true;
  });
});

Using Chai's `expect`

The TestRunner includes Chai's expect function for writing assertions in your tests. The expect function is part of Chai's BDD (Behavior-Driven Development) interface. You can learn more about Chai's expect function and its usage at the Chai documentation.

Examples of `expect` Usage

Here are some examples of using Chai's expect function in your tests:

it('should assert true is true', () => {
  expect(true).to.be.true;
});

it('should assert equality', () => {
  expect(1 + 1).to.equal(2);
});

Running Tests

To run the tests, you need to call runTests and export the result. You can specify a custom timeout for tests and hooks by passing the timeout value in milliseconds as an argument to runTests.

import 'testRunner'

describe('Sample Test Suite', () => {
  it('should pass this test', () => {
    expect(true).to.be.true;
  });

  it('should fail this test', () => {
    expect(false).to.be.true;
  });
});

export default await runTests();

Custom Timeout

You can set a custom timeout for all tests and hooks in the suite by passing the timeout value to runTests.

import 'testRunner'

describe('Sample Test Suite with Custom Timeout', () => {
  it('should pass this test', () => {
    expect(true).to.be.true;
  });

  it('should fail this test', () => {
    expect(false).to.be.true;
  });
});

export default await runTests(10000); // Set timeout to 10 seconds

Example Usage

Here is an example demonstrating the complete setup and usage of the TestRunner library.

import 'testRunner'

describe('Example Test Suite', () => {
  beforeAll(() => {
    console.log('Running global beforeAll hook');
  });

  afterAll(() => {
    console.log('Running global afterAll hook');
  });

  beforeEach(() => {
    console.log('Running global beforeEach hook');
  });

  afterEach(() => {
    console.log('Running global afterEach hook');
  });

  describe('Nested Suite', () => {
    beforeAll(() => {
      console.log('Running nested beforeAll hook');
    });

    it('should pass this test', () => {
      expect(true).to.be.true;
    });

    it('should fail this test', () => {
      expect(false).to.be.true;
    });
  });

  it('should pass this test outside nested suite', () => {
    expect(1 + 1).to.equal(2);
  });
});

export default await runTests();

Conclusion

The TestRunner library provides a straightforward way to define and run tests for your JavaScript code. With support for asynchronous operations, configurable timeouts, and Chai's expect function for assertions, you can ensure that your tests are robust and reliable. Happy testing!

Documentation: https://sebastianwessel.github.io/quickjs/
GitHub Repository: https://github.com/sebastianwessel/quickjs
Submit Feedback: Create an Issue on GitHub

Secure Data Transformation in ETL Pipelines

Sebastian Wessel — Sun, 07 Jul 2024 13:23:46 +0000

One particularly powerful use case for the QuickJS package is in the realm of data pipelines and ETL (Extract, Transform, Load) processes.

Imagine a scenario where you're building a flexible data transformation service. Your platform needs to handle incoming data and transform it into a specific format, but here's the catch: the transformation logic is provided by the users themselves in the form of custom JavaScript code.

This presents a significant challenge. On one hand, you want to offer users the flexibility to define their own transformation logic. On the other hand, you need to ensure that this user-provided code doesn't compromise the security or integrity of your system. This is where the QuickJS package truly shines.

Here's how you might implement such a system:

import { quickJS } from '@sebastianwessel/quickjs'

async function setupTransformationEnvironment() {
  const { createRuntime } = await quickJS()
  return createRuntime({
    allowFetch: false,  // Disable network access for security
    allowFs: false,     // Disable file system access
    env: {
      VERSION: '1.0',
      // Add any other environment variables needed for transformation
    },
  })
}

async function transformData(inputData, transformationCode) {
  const runtime = await setupTransformationEnvironment()
  const { evalCode } = runtime

  const wrappedCode = `
    const transform = (data) => {
      ${transformationCode}
    };
    export default transform(${JSON.stringify(inputData)});
  `

  try {
    const result = await evalCode(wrappedCode)
    return { success: true, transformedData: result }
  } catch (error) {
    return { success: false, error: error.message }
  }
}

// Example usage
const inputData = { name: "John Doe", age: 30, city: "New York" }
const userProvidedTransformationCode = `
  // User-defined transformation logic
  return {
    fullName: data.name,
    isAdult: data.age >= 18,
    location: data.city.toUpperCase()
  };
`

const result = await transformData(inputData, userProvidedTransformationCode)
console.log(result)

In this example:

We set up a secure runtime environment using QuickJS, disabling potentially dangerous features like network and file system access.
The transformData function takes input data and user-provided transformation code as parameters.
We wrap the user's code in a function and pass the input data to it.
The code is executed in the sandboxed environment, ensuring that it can't affect the rest of your system.
The transformed data is returned, or an error is caught and reported if the transformation fails.

This approach offers several benefits:

Security: User-provided code runs in a sandbox, protecting your system from malicious or poorly written scripts.
Flexibility: Users can define complex transformation logic tailored to their specific needs.
Control: You can precisely limit what the transformation code can do (e.g., no network requests, no file system access).
Error Handling: Errors in user code are caught and handled gracefully, preventing them from crashing your service.

By leveraging the QuickJS package in this way, you can build a robust, flexible, and secure data transformation service. This pattern can be extended to various other scenarios where you need to run user-provided code safely, such as custom report generation, dynamic data analysis, or even building a secure coding playground for educational purposes.

The ability to execute arbitrary JavaScript securely opens up a world of possibilities for building flexible, user-customizable systems without compromising on security. Whether you're dealing with ETL processes, building a low-code platform, or creating any system that needs to safely execute user-defined logic, the QuickJS package provides the tools you need to do so with confidence.

Documentation: https://sebastianwessel.github.io/quickjs/
GitHub Repository: https://github.com/sebastianwessel/quickjs
Submit Feedback: Create an Issue on GitHub

Empowering AI: The QuickJS Package for LLM Tool Calling

Sebastian Wessel — Sun, 07 Jul 2024 13:23:36 +0000

As AI continues to evolve, the ability to safely execute code becomes increasingly important. The QuickJS package is perfectly positioned to fill this gap, offering a powerful solution for implementing tool calling capabilities in AI applications.

Why QuickJS for AI Tool Calling?

When developing AI systems that interact with user-provided or dynamically generated code, security is paramount. The QuickJS package provides a secure sandbox that allows LLMs to execute JavaScript code safely, opening up a world of possibilities for AI-powered tools and applications.

Key Benefits for AI Developers:

Secure Execution: Run AI-generated or user-provided JavaScript code without risking your main application or server.
Flexible Integration: Easily integrate with various LLM frameworks and AI platforms.
Rich Environment: Provide a Node.js-like environment for more complex tool implementations.
Performance: Leverage the speed of QuickJS for quick code execution in AI workflows.
Customizable Sandbox: Control what capabilities are available to the AI, ensuring safe and appropriate tool use.

Example: Implementing an AI Code Executor with LangChain

Here's an example of how you can use the QuickJS package with LangChain to create a powerful AI code execution tool:

import { ChatOpenAI } from "@langchain/openai";
import { DynamicStructuredTool } from "@langchain/core/tools";
import { quickJS } from '@sebastianwessel/quickjs';
import { z } from "zod";

// Initialize QuickJS (this should be done once)
const initQuickJS = async () => {
  const { createRuntime } = await quickJS();
  return createRuntime({
    allowFetch: false,  // Disable network access for safety
    allowFs: true,      // Allow file system operations if needed
    env: {
      AI_VERSION: '1.0',
    },
  });
};

// Create a QuickJS runtime
const quickJSRuntime = await initQuickJS();

// Define the schema for our JavaScript execution tool
const jsExecutionSchema = z.object({
  code: z.string().describe("The JavaScript code to execute"),
});

// Create the JavaScript execution tool
const jsExecutionTool = new DynamicStructuredTool({
  name: "javascript_executor",
  description: "Executes JavaScript code and returns the result.",
  schema: jsExecutionSchema,
  func: async ({ code }) => {
    try {
      const { evalCode } = await quickJSRuntime;
      const result = await evalCode(code);
      return JSON.stringify(result);
    } catch (error) {
      return `Error executing code: ${error.message}`;
    }
  },
});

// Initialize the language model with the tool
const model = new ChatOpenAI({
  modelName: "gpt-3.5-turbo",
  temperature: 0,
});
const llmWithTools = model.bind({
  tools: [jsExecutionTool],
});

// Example usage
const query = "Calculate the 10th Fibonacci number using JavaScript.";
const result = await llmWithTools.invoke(query);

console.log("AI Response:", result.content);
console.log("Tool Calls:", result.additional_kwargs.tool_calls);

// If there's a tool call, we can execute it
if (result.additional_kwargs.tool_calls) {
  const toolCall = result.additional_kwargs.tool_calls[0];
  const toolResult = await jsExecutionTool.call(
    JSON.parse(toolCall.function.arguments)
  );
  console.log("Tool Execution Result:", toolResult);
}

This setup allows the AI to write and execute JavaScript code safely within the QuickJS sandbox, providing a powerful and flexible tool for various tasks. It combines the strengths of LangChain for AI interactions with the security and flexibility of the QuickJS package for code execution.

Enhancing AI Capabilities

By integrating the QuickJS package into your AI toolkit, you can:

Implement Complex Tools: Allow your AI to write and execute more sophisticated tools on the fly.
Safe Code Testing: Let AI assistants test and debug code in a controlled environment.
Dynamic Data Processing: Enable AI to process data using custom JavaScript functions.
Interactive Coding Tutorials: Create AI-powered coding education platforms with live code execution.
Secure Serverless Functions: Implement AI-generated serverless functions with built-in security.

Documentation: https://sebastianwessel.github.io/quickjs/
GitHub Repository: https://github.com/sebastianwessel/quickjs
Submit Feedback: Create an Issue on GitHub

Execute JavaScript in a WebAssembly QuickJS Sandbox

Sebastian Wessel — Sun, 07 Jul 2024 13:23:22 +0000

Typescript QuickJS Package: Empowering Secure JavaScript Execution in AI and Beyond

As an experienced developer who understands the needs of the community, I've created the QuickJS package to address a critical challenge in modern software development: executing JavaScript securely while maintaining flexibility and performance across various platforms. This TypeScript library offers a robust solution for running JavaScript code in a secure sandbox environment, with particular benefits for AI and large language model (LLM) applications.

The Genesis of the QuickJS Package

My journey began with QuickJS, an impressive small and efficient JavaScript engine originally written in C by Fabrice Bellard and Charlie Gordon. While QuickJS itself is a C program, I saw an opportunity to make it more accessible to the JavaScript and TypeScript community.

The foundation of my package is built upon the excellent work done by the quickjs-emscripten project (https://github.com/justjake/quickjs-emscripten). They've done the crucial work of compiling QuickJS to WebAssembly, enabling its use in web and Node.js environments. What I've created is a high-level abstraction layer around this WebAssembly implementation, designed with developer experience in mind.

A Sandbox Built for Developers, by a Developer

As someone who's been in the trenches of software development, I understand the importance of a tool that's not only powerful but also easy to use. The QuickJS package creates an isolated environment where untrusted code can run freely, separated from your main application, whether it's running in a browser or on a server.

Features Designed with You in Mind

When developing this package, I focused on bringing together features that I, as a developer, would want:

Node-like Module Support: I've included support for core modules similar to Node.js, including node:fs, node:assert, node:util, and node:path.
Fetch API: Because making HTTP requests is such a common need, I've made sure fetch is available within the sandbox.
Custom Module Support: Knowing that every project has unique needs, I've made it possible to use your own modules.
Virtual File System: This allows for file operations without risking your actual file system - a must-have for secure environments.
TypeScript Compatibility: As a TypeScript enthusiast, ensuring seamless integration with TypeScript projects was a priority.
User-Friendly API: I've put a lot of effort into making the API intuitive and easy to use.
Integrated TestRunner: Testing is crucial, so I've included a lightweight testing library right in the sandbox.

The QuickJS Package in Action

Here's a quick example of how you can use the package:

import { quickJS } from '@sebastianwessel/quickjs'

// General setup - do this once
const { createRuntime } = await quickJS()

// Create a runtime instance for code execution
const { evalCode } = await createRuntime({
  allowFetch: true,
  allowFs: true,
  env: {
    MY_ENV_VAR: 'env var value'
  },
})

// Execute code in the sandbox
const result = await evalCode(`
  import { join } from 'node:path'
  import { writeFileSync, readFileSync } from 'node:fs'
  import assert from 'node:assert'

  const fn = async () => {
    console.log(join('src','dist'))
    console.log(env.MY_ENV_VAR)

    // Using node:fs
    writeFileSync('/test.txt', 'Hello, QuickJS!')
    const content = readFileSync('/test.txt', 'utf8')
    assert.strictEqual(content, 'Hello, QuickJS!')

    // Using fetch
    const url = new URL('https://example.com')
    const f = await fetch(url)
    return f.text()
  }

  export default await fn()
`)

console.log(result)

Leveraging Modern Tools for a Better Developer Experience

In developing this package, I've leveraged some of the most exciting tools in the JavaScript ecosystem:

Bun: This all-in-one JavaScript runtime has been instrumental in the development process, offering speed and simplicity.
Hono: For handling HTTP requests, Hono has been a game-changer with its lightweight and performant approach.
Biome: This toolchain has helped maintain code quality and consistency throughout the project.

I've also used other modern tools like poolifier-web-worker for efficient multithreading, tshy for TypeScript package building, and autocannon for performance testing.

Real-World Applications

From my experience, I've seen this package shine in various scenarios:

Running user-provided scripts safely in web or desktop applications
Executing plugins or extensions securely
Creating sandboxed environments for coding challenges or educational platforms
Testing potentially unsafe code snippets in development environments
Implementing secure scripting capabilities in backend services
Powering AI-driven code generation and execution in LLM applications

Performance and Security: No Compromises

As a developer, I know the importance of both performance and security. That's why I've ensured that this package, leveraging the lightweight QuickJS engine compiled to WebAssembly, offers excellent performance without sacrificing security.

Getting Started

I've made sure that getting started with the QuickJS package is straightforward:

npm install @sebastianwessel/quickjs

For more detailed documentation and examples, check out the official documentation and the example repository.

Documentation: https://sebastianwessel.github.io/quickjs/
GitHub Repository: https://github.com/sebastianwessel/quickjs
Submit Feedback: Create an Issue on GitHub

How to Design a SurrealDB schema and create a basic client for TypeScript

Sebastian Wessel — Sun, 17 Sep 2023 13:04:15 +0000

In the midst of a dynamic landscape of exciting new projects, one name shines bright — SurrealDB.

It's not just another database - it's touted as 'The ultimate multi-model database.'.
Just last week, during the 'SurrealDB World' event, they celebrated the launch of their first production-ready version.

What makes SurrealDB unique is its exceptional flexibility in data storage.
Unlike traditional databases that force you to choose between fixed schemas or total chaos, SurrealDB lets you have the best of both worlds.
You can define a table as schema-less while still specifying schema information for known fields.
In this article, we'll explore SurrealDB's features and learn how to create a data schema and a basic client to make the most of this versatile database.

Defining Your Schema

When it comes to creating a schema definition, you have two primary methods at your disposal.

Using SurrealQL for Schema Definition

If you're already familiar with SQL, you'll find SurrealQL to be quite approachable since it shares similarities with standard SQL.

As described in the SurrealDB documentation, you can define your schema like this:



-- Let's start by creating a schema-full user table.
DEFINE TABLE user SCHEMAFULL;

-- Now, let's define some fields.
DEFINE FIELD firstName ON TABLE user TYPE string;
DEFINE FIELD lastName ON TABLE user TYPE string;
DEFINE FIELD email ON TABLE user TYPE string
  ASSERT string::is::email($value);

With SurrealQL, you can structure your schema just the way you want it, making it a powerful tool for designing your data model.

As mentioned earlier, the same principles apply when defining a table as SCHEMALESS instead of SCHEMAFULL.

Here's an example in SurrealQL:



-- Let's create a schemaless user table.
DEFINE TABLE user SCHEMALESS;

-- Now, we'll proceed to define some fields.
DEFINE FIELD firstName ON TABLE user TYPE string;
DEFINE FIELD lastName ON TABLE user TYPE string;
DEFINE FIELD email ON TABLE user TYPE string
  ASSERT string::is::email($value);

When it comes to field definition, SurrealQL offers a rich array of features. Some of the key ones include:

Specifying the data type
Setting up validation rules
Defining default values
Applying data transformations
Establishing a list of possible values, similar to enums

For more detailed information, be sure to explore the official documentation.

Leveraging Surrealist.app

When it comes to working with SurrealDB, there's one tool that I wholeheartedly recommend: Surrealist.
This software is a game-changer for simplifying your SurrealDB experience.

Surrealist offers a user-friendly interface that streamlines your interactions with SurrealDB. Not only does it come in a convenient browser version, but it also offers the powerful Surrealist Desktop application.

Surrealist Desktop takes your SurrealDB workflow to the next level by introducing a range of additional features, including the highly versatile Designer.

The Surrealist Designer empowers you to effortlessly craft your table schema and manage entity relations. It's not just for beginners; the intuitive UI makes it accessible to newcomers, while experts can appreciate its ability to visualize even the most intricate data structures.

Whether you're just starting or diving into complex projects, Surrealist is your trusted companion for making the most of SurrealDB. Give it a try and see how it enhances your database design journey.

Generating Zod Schemas and TypeScript Clients from SurrealDB

Being a TypeScript developer, my journey with a database doesn't end at just interacting with it.
Naturally, I want a TypeScript program that seamlessly connects to SurrealDB.

I soon found myself at a point where I wanted to automate the creation of a simple client, leveraging the information already present in the database.

So, I embarked on a weekend coding session and crafted a small CLI tool. This tool works its magic by extracting schema information directly from SurrealDB, and it doesn't stop there. It also generates corresponding Zod schemas and a TypeScript client for basic CRUD operations.
The client is designed to work seamlessly with the official SDK, making the integration process smooth and efficient.

With this tool, I aimed to simplify the process of working with SurrealDB, allowing TypeScript developers to dive into their projects with confidence, knowing they have a solid foundation to build upon.

The intention is, to provide a solid starting point for development, and to speed up the processed.

You can find the project on GitHub:
https://github.com/sebastianwessel/surrealdb-client-generator

You have multiple configuration options at your disposal, which can be set either as command-line interface (CLI) options or within a configuration file.
What's more, you can even use both methods simultaneously.
In such cases, the CLI options take precedence.

Getting started with the tool is a breeze:



npx surql-gen



Options:
  -V, --version                          output the version number
  -f, --file [schemaFile]                a SurrealQL file containing the definitions (default: "myschema.surql")
  -c, --config [configFile]              SurrealDB connection url (default: "surql-gen.json")
  -s, --surreal [surreal]                SurrealDB connection url (default: "ws://127.0.0.1:8000")
  -u, --username [username]              auth username (default: "root")
  -p, --password [password]              auth password (default: "root")
  -n, --ns [ns]                          the namspace (default: "test")
  -d, --db [db]                          the database (default: "test")
  -o, --outputFolder [outputFolder]      output folder (default: "client_generated")
  -g, --generateClient [generateClient]  generate client (default: true)

For further information, just check out the readme file in the repository.

Understanding Generated Files

In your designated output folder, it's essential to grasp how the generated files are organized for efficient management:

`_generated` Subfolder

Anything residing beneath the _generated subfolder is subject to potential overwriting during subsequent runs of the generator. This serves as the workspace where dynamic updates can occur.

`schema` Subfolder

The schema subfolder plays a pivotal role. It contains the schema and type definitions, offering customization possibilities. Unlike the _generated folder, everything within schema is generated only once and remains untouched or deleted in subsequent runs of the generator.

`client` Subfolder

When it comes to accessing the generated client code, you'll find it neatly tucked away within the client subfolder.

This organized structure ensures that your generated files are readily accessible while maintaining the integrity of your project's foundation.

Using the Client

The generated client is designed with simplicity in mind, adhering to the efficient repository pattern. In this pattern, each table is treated as a distinct entity, complete with its dedicated repository. These repositories, in turn, consolidate actions specific to the respective entity.

Using the client is straightforward and user-friendly. You'll always encounter methods in the format of get[Entity Name]Repository(db: Surreal), requiring only the database instance as an argument.

For a quick illustration, let's take a peek at how we can access all the generated methods for a project entity:



import { Surreal } from 'surrealdb.js'

import { getProjectRepository } from './client_generated/client/project/getProjectRepository.js'

const main = async () => {
  const db = new Surreal()
  await db.connect('ws://0.0.0.0:8000', {
    auth: { user: 'root', pass: 'root' },
  })
  await db.use({
    ns: 'voyage',
    db: 'voyage',
  })

  const rep = getProjectRepository(db)

  const project = await rep.createProject({
    name: 'Test project',
    logo: '/mylogo',
  })
  console.log('project', project)

  const projects = await rep.getAllProjects()
  console.log('projects', projects)

  const updated = await rep.updateProject(project.id, { name: 'Renamed !' })
  console.log('updated', updated)

  const selected = await rep.getProjectById(project.id)
  console.log('selected', selected)

  await rep.deleteProject(project.id)
  console.log('deleted')

  const projectsDel = await rep.getAllProjects()
  console.log('projects', projectsDel)

  await db.close()
}

main()

All return values are automatically typed based on the Zod schema, simplifying your code and ensuring consistency.
Even if you customize a schema, TypeScript types adapt seamlessly, eliminating the need for manual updates.
This streamlined process keeps your codebase error-free and agile, saving you time and effort.

Alternative

There is the possibility, to disable the generation of the TypeScript client.
The intention is, to only generate Zod schema, which than can simply used in Cirql - SurrealDB ORM & Query Builder or in your own choice.

Final words

In Closing

SurrealDB is undeniably a remarkable addition to the world of databases, and it's a platform that deserves your attention. With Surrealist, the entry barrier for beginners is lowered to a minimum, offering a professional UI enriched with fantastic features.

I trust that my small tool will serve as a valuable resource for TypeScript developers looking to dive into the world of SurrealDB, making it more accessible and enjoyable.

If you find this tool helpful, I kindly ask you to show your appreciation by giving it a star on GitHub.

And, of course, if you encounter any issues or have suggestions for improvements, please don't hesitate to open an issue on GitHub. Your feedback is invaluable.

For further inquiries about SurrealDB, don't hesitate to explore their comprehensive documentation or engage with the community on their Discord channel.

Happy coding - Cheers and bye bye! 👋

Looking to dive deeper into SurrealDB?
Don't miss out on the comprehensive series I've recently penned

SurrealDB - The Magic Database to Keep on Your Radar

Sebastian Wessel ・ Sep 8 '23

#database #programming #tutorial #news

PURISTA - Tests with Jest, Sinon.js and Testcontainers

Sebastian Wessel — Tue, 12 Sep 2023 08:48:46 +0000

In the journey of crafting the PURISTA TypeScript backend framework, the need for automated software testing became increasingly apparent.
PURISTA's extensive amount of adapters and integrations with third-party solutions makes it impractical to conduct manual testing repeatedly.
In this article, we dive into the pivotal role of automated testing in ensuring the reliability and stability of PURISTA's offerings.

Moreover, enabling developers to effortlessly test their code built on top of PURISTA stands as a pivotal aspect of the framework.

Jest

Jest offers a comprehensive test suite where you can efficiently organize, implement, and execute your tests. With approximately 1490 contributors, Jest has played a pivotal role in simplifying the process of writing tests, as demonstrated below:

import { asyncOperation, syncOperation } mytestfile.test.js

describe('My testfile works', ()=>{
  it('is exected to be ok', () => {
    const result = syncOperation()
    expect(result).toBe('OK')
  })

  it('works with async await', async ()=>{
    const result = asyncOperation()
    await expect(result).resolves.toBe('OK')
  })
})

As the number on GitHub shows: This awesome tool is used by 10.4 millions!
This unbelievable number shows, how important it is.

A massive shout-out to Jest and all its dedicated contributors! Consider starring it on GitHub, giving a shout-out on social media, or mentioning it in your readme file to show your appreciation. 🌟👏

We use Jest together with @swc/jest, which speeds up our test runs dramatically.
They don't lie with "Super-fast alternative for babel-jest or ts-jest without type checking".
It's the second great tool by Vercel we use at PURISTA.

Sinon.js

When writing tests, you probably will come to the point where you need to mock, fake or spy on something. Even if the test framework is providing some helpers for it, you should have a look at Sinon.js.

At PURISTA, we primarily leverage Sinon.js to deliver a seamless testing experience for developers.
We provide for example the getLoggerMock function, which returns an easy way to test logging, without poluting the test runner output.

This function looks like this:

export const getLoggerMock = (sandbox?: SinonSandbox) => {
  const info = sandbox?.stub() || stub()
  const error = sandbox?.stub() || stub()
  const warn = sandbox?.stub() || stub()
  const debug = sandbox?.stub() || stub()
  const trace = sandbox?.stub() || stub()
  const fatal = sandbox?.stub() || stub()

  const mock: Logger = {
    info,
    error,
    warn,
    debug,
    trace,
    fatal,
    getChildLogger: () => mock,
  }

  return {
    stubs: {
      info,
      error,
      warn,
      debug,
      trace,
      fatal,
    },
    mock,
  }
}

As you can see, we've created stubs, which are a form of mock implementations. You can use them to check whether a stub is called, how many times it's called, if it's called with specific parameters, and more. Additionally, Sinon.js offers sandboxing, a highly recommended feature. It prevents your tests from interfering with each other, ensuring clean and reliable results.

As an example on how it might look:

import { Logger } from '@purista/core'
import { createSandbox } from 'sinon'

function syncOperation(logger: Logger) {
   logger.info('info log')
   return 'OK'
}

describe('My testfile works', ()=>{
  const sandbox = createSandbox()

  afterEach(() => {
    sandbox.reset()
  })

  it('is exected to be ok', () => {
    const loggerMock = getLoggerMock()

    const result = syncOperation(loggerMock.mock)
    expect(result).toBe('OK')
    expect(loggerMock.stub.info.calledOnceWith('info log')).toBeTruthy()
  })

})

Sinon.js is unbelievably helpful, and the 328 contributors are doing an incredible job. It's no surprise that Sinon.js is trusted by over 574k developers. Show them some love with a big shout-out and a star on GitHub!

If you haven't used Sinon.js before, it's definitely worth giving it a try.

Testcontainers

To be honest, Testcontainers wasn't on our radar initially.
However, we soon realized the need for a viable solution to conduct integration tests for:

Testcontainers was the gamechanger here.
It is super easy to spin up a docker container and run tests against a real running system.

To show you, how simple it is:

Here is the example, on how start a full RabbitMQ before we run our tests:

import { GenericContainer, StartedTestContainer } from 'testcontainers'

const AMQP_PORT = 5672

describe('@purista/amqpbridge', () => {
  let container: StartedTestContainer

  beforeAll(async () => {
    container = await new GenericContainer('rabbitmq:alpine')
      .withExposedPorts({ host: AMQP_PORT, container: AMQP_PORT })
      .start()
  })

  afterAll(async () => {
    await container.stop()
  })

It significantly enhanced our local testing process. Furthermore, Testcontainers is a versatile tool available for multiple programming languages.

A heartfelt thank you to all the contributors of Testcontainers for this incredible project! We deeply appreciate the outstanding work you've put into it.

If you're searching for content ideas for the next article, consider crafting a cool piece about Testcontainers and its practical applications. Dive into how to harness its power effectively.

Now that we've explored the powerful tooling required to build PURISTA itself, let's shift our focus to the components under the hood that make PURISTA tick.

Be sure to check out the upcoming article in this series!

PURISTA: Build with rimraf, esbuild, Turbo & git-cliff

Sebastian Wessel — Mon, 11 Sep 2023 21:15:26 +0000

In our previous article, we laid the foundation with a glimpse of our coding setup.

It's now time to spotlight the indispensable build tools that power its development.

rimraf

Huge thanks to Isaacs!
Rimraf comes to the rescue, providing a reliable solution for deep, recursive removal of folders and files.
At PURISTA, we rely on rimraf to maintain pristine build output directories.

Turbo

PURISTA is organized in a monorepo.
During the development and build process, Turbo is used to execute different tasks and steps on multiple packages with one command.

We aren't harnessing the full potential and capabilities of Turbo at the moment since PURISTA currently requires only a few straightforward build tasks.

If you have more complex tasks, especially those with interdependencies between steps, it's worth exploring https://turbo.build

Esbuild

In its early stages, PURISTA was built using the standard TypeScript compiler.
However, this approach excelled in creating CommonJS packages but turned into a nightmare when it came to generating ESM builds.

esbuild - the rescue!
No longer struggling with configs, file extensions or similar.

With esbuild, it was becoming simple and fast.

We created a small script:

const esbuild = require('esbuild')

esbuild
  .build({
    entryPoints: ['./src/index.ts'],
    outfile: './lib/esm/index.mjs',
    bundle: true,
    format: 'esm',
    splitting: false,
    platform: 'node',
    sourcemap: false,
    target: 'node18',
    minify: true,
    packages: 'external',
  })
  .catch(() => process.exit(1))

esbuild
  .build({
    entryPoints: ['./src/index.ts'],
    outfile: './lib/cjs/index.cjs',
    bundle: true,
    format: 'cjs',
    splitting: false,
    platform: 'node',
    sourcemap: false,
    target: 'node18',
    minify: true,
    packages: 'external',
  })
  .catch(() => process.exit(1))

Since this approach only generates JavaScript files, and we aim to provide proper TypeScript types to PURISTA users, we utilize the TypeScript compiler specifically for building types.

tsc --emitDeclarationOnly --declarationMap false --declaration --outDir lib/types

In package.json, only these additions were necessary:

{
  "directories": {
      "lib": "lib/index.js"
  },
  "main": "./lib/cjs/index.cjs",
  "module": "./lib/esm/index.mjs",
  "exports": {
      ".": {
          "import": "./lib/esm/index.mjs",
          "require": "./lib/cjs/index.cjs",
          "types": "./lib/types/index.d.ts"
      }
  },
  "typings": "lib/types/index.d.ts",
  "files": [
      "lib"
  ]
}

A big thank you ❤️ to all contributors and maintainers of esbuild!
It solved the issue with CommonJS/ESM for us and produces size optimized outputs for all PURISTA packages.

git-cliff

Changelogs are nice and everybody likes them, but nobody wants to maintain them 😜.
Here, git-cliff by Orhun Parmaksız is one simple to use solution.

One simple command:

git-cliff > CHANGELOG.md

And your changelog gets generated from your git history.

Undoubtedly one of the time-saving tools, you should check out at https://git-cliff.org

In the upcoming article of this series, we will delve deeper into the test setup. Stay tuned!

PURISTA - Thanks to amazing open-source software

Sebastian Wessel — Mon, 11 Sep 2023 20:17:57 +0000

Welcome to our series on the unsung heroes behind PURISTA!

Ever wondered about PURISTA?
It's not just another Typescript backend framework for simply building HTTP-endpoints.
It's a versatile solution that embraces diverse techniques for highly distributed deployments, taking inspiration from event-driven architecture.
If you're interested, you're invited to take a look at the official website http://purista.dev.

But hold on, this series isn't about the framework itself. Instead, we're here to shine a spotlight on the incredible libraries, tools, software, and passionate contributors that make PURISTA possible.

In this series, we will introduce all the tools, explain how we use them, why we find them essential, and endeavor to illustrate why they might also be a suitable solution for you.

General setup

Most developers should be more or less familiar with the basic setup.

VSCode as a code editor with ESLint plugin.

GitHub is uses as repository, issue & project tracker, and also for hosting the official website.

Coding Setup

In addition to Node.js and TypeScript, we rely on two indispensable tools available as npm modules:

ESLint and Prettier

Maintaining a clean, readable, and consistent codebase is essential, and ESLint and Prettier are the perfect tools for the job.
Just enable "lint on save" in VSCode, and you won't have to worry about it. They're a "must-have," even for small personal projects.

See https://eslint.org and https://prettier.io

ESLint is extended by some awesome plugins and extensions:

eslint-plugin-simple-import-sort by Simon Lydell @SimonLydell
eslint-plugin-node by Toru Nagashima - Dev.to: @mysticatea & Twitter: @mysticatea
eslint-plugin-json by Azeem Bande-Ali
eslint-plugin-import
eslint-config-standard

In our upcoming article, we'll delve deep into the inner workings of PURISTA's build pipeline. Stay tuned for an in-depth exploration!

SurrealDB - Improve data integrity by adding schema information

Sebastian Wessel — Fri, 08 Sep 2023 20:46:26 +0000

In the concluding chapter of this series, we'll delve into the intriguing world of SurrealDB's schema possibilities and explore how they can elevate our data integrity to new heights.

You can define fields for tables. We will take the user table as an example here.

Our initial definition was:

-- Define the user table
DEFINE TABLE user SCHEMALESS;

We will add now some field definitions:

-- Define the user table
DEFINE TABLE user SCHEMALESS;
DEFINE FIELD name ON TABLE user TYPE string;
DEFINE FIELD email ON TABLE user TYPE string;
DEFINE FIELD firstName ON TABLE user TYPE string;
DEFINE FIELD lastName ON TABLE user TYPE string;
DEFINE FIELD createdAt ON TABLE user TYPE datetime;

We're on the right track so far. We've defined the necessary fields, and we've also specified their types.
But there are some improvements, we can make.

Now, let's focus on the createdAt attribute. In this case, we should establish a default value that automatically captures the creation timestamp when the entity is generated.

DEFINE FIELD createdAt ON TABLE user TYPE datetime DEFAULT time::now();

We should also enhance the email field.
There are two improvements we can make.
First, we should specify the field type as more than just a generic string, as we know it must adhere to the email pattern.

DEFINE FIELD email ON TABLE user TYPE string ASSERT string::is::email($value);

Secondly, we want to ensure, that the email is globally unique. To ensure this, we define a unique index.

DEFINE INDEX uniqueEmailIndex ON TABLE user COLUMNS email UNIQUE;

Here's our ultimate definition for our user table:

-- Define the user table
DEFINE TABLE user SCHEMALESS;
DEFINE FIELD name ON TABLE user TYPE string;
DEFINE FIELD email ON TABLE user TYPE string ASSERT string::is::email($value);
DEFINE FIELD firstName ON TABLE user TYPE string;
DEFINE FIELD lastName ON TABLE user TYPE string;
DEFINE FIELD createdAt ON TABLE user TYPE datetime DEFAULT time::now();

DEFINE INDEX uniqueEmailIndex ON TABLE user COLUMNS email UNIQUE;

Concerning the relationships we've established between various entities, it's important to avoid duplicates between the same entities.
In this context, we can create a unique index to prevent multiple connections of a specific user to a particular tenant.

-- prevent linking a user to the same tenant multiple times
DEFINE INDEX tenant_memberIndex
    ON TABLE tenant_member
    COLUMNS in, out UNIQUE;

Final thoughts

As we conclude this series, we've only just scratched the surface of SurrealDB's capabilities, with a strong focus on its graph structures.

SurrealDB has an exciting set of features and upcoming announcements that make it a truly fascinating unicorn in the database world.

The handling of unique record-IDs and the possibilities they unlock are nothing short of amazing.
The freedom to choose between a relational database, key-value store, NoSQL/document database, graph database, or specialized time series database while using one database simplifies life significantly.
Tailoring how you store and access data based on your specific needs and mixing things up as required is the way forward.

While SurrealDB is still in its early stages and not officially production-ready, I encourage you to give it a try in one of your upcoming small (or side) projects.

The adventure with SurrealDB may just be the unique and rewarding experience you've been looking for.

SurrealDB - Query and combine data via relations

Sebastian Wessel — Fri, 08 Sep 2023 20:46:15 +0000

In the previous article of this series, we set up the foundation and added some sample data.

Now, we'll dive into retrieving and merging that data.
I strongly encourage you to explore the SurrealDB documentation. It will make understanding the upcoming examples much smoother.

See: SELECT and RELATE

Query tenant(s)

To start, we'll keep it simple and begin by fetching all the tenants.

-- fetch all tenants
SELECT * from tenant;

Result:

[
    {
        "id": "tenant:car",
        "name": "Car Enthusiasts"
    },
    {
        "id": "tenant:cat",
        "name": "Cat Owners"
    },
    {
        "id": "tenant:musician",
        "name": "Musicians"
    }
]

Add available roles

Our next step is to find out which roles are generally accessible within the context of a particular tenant.

This information is stored in the tenant_role relation.
We can now extend our query as shown below.

-- select the tenant car enthusisast
-- fetch the related roles and return it as availableRoles
SELECT *,
  (SELECT * FROM <-tenant_role<-role) as availableRoles
FROM tenant:car;

Result:

[
    {
        "availableRoles": [
            {
                "id": "role:admin",
                "name": "Administrator"
            },
            {
                "id": "role:reader",
                "name": "Content Reader"
            },
            {
                "id": "role:author",
                "name": "Content Author"
            }
        ],
        "id": "tenant:car",
        "name": "Car Enthusiasts"
    }
]

Now, let's tidy up the output. Since we're aware that we're fetching just one tenant, we can use the ONLY statement between FROM and tenant:car to directly obtain a single result object. Additionally, we'll limit the output to only display the role IDs.

-- return only id´s of roles and return as single object
SELECT *,
  (SELECT id FROM <-tenant_role<-role).id as availableRoles 
FROM ONLY tenant:car;

Result:

{
    "availableRoles": [
        "role:admin",
        "role:reader",
        "role:author"
    ],
    "id": "tenant:car",
    "name": "Car Enthusiasts"
}

Add tenant members

In the next step, we will also return the users, which are car enthusiasts.

-- add members of tenant car
SELECT *,
      (SELECT * FROM <-tenant_member<-user) as members,
      (SELECT id FROM <-tenant_role<-role).id as availableRoles 
FROM ONLY tenant:car;

Result:

{
    "availableRoles": [
        "role:admin",
        "role:reader",
        "role:author"
    ],
    "id": "tenant:car",
    "members": [
        {
            "email": "john.doe@example.com",
            "firstName": "John",
            "id": "user:1",
            "lastName": "Doe",
            "name": "John Doe"
        }
    ],
    "name": "Car Enthusiasts"
}

Add tenant member roles

We will now add the role information to each user who is a member of the given tenant.
We will change

(SELECT * FROM <-tenant_member<-user) as members,

(
 SELECT 
  (SELECT * FROM ONLY <-person) as person,
  (SELECT id FROM ->member_role->role).id as roles,
 FROM <-tenant_member
) as members

As you can observe, we've made a change in how we resolve data. Now, we don't go all the way up to the user through <-tenant_member<-user. Instead, we stop at the relation entry itself with <-tenant_member.

Recall our data schema. We've established a relationship between users and tenants. This relationship, in turn, has its own connections to one or even multiple roles.

Think of it as a crossroads. If you head straight ahead, you'll discover user information. But if you veer left or right, you'll stumble upon the roles assigned to that user within a specific tenant.
The full query:

SELECT *, (
      SELECT 
            (SELECT * FROM ONLY <-user) as user,
            (SELECT id FROM ->member_role->role).id as roles
      FROM <-tenant_member
      ) as members,
      (SELECT id FROM <-tenant_role<-role).id as availableRoles 
FROM ONLY tenant:car;

Result:

{
    "availableRoles": [
        "role:admin",
        "role:reader",
        "role:author"
    ],
    "id": "tenant:car",
    "members": [
        {
            "roles": [
                "role:author"
            ],
            "user": {
                "email": "john.doe@example.com",
                "firstName": "John",
                "id": "user:1",
                "lastName": "Doe",
                "name": "John Doe"
            }
        }
    ],
    "name": "Car Enthusiasts"
}

Add tenant permissions for roles of members

In most scenarios, it may not be practical to display the permissions for every user within a tenant at this stage.

However, for the sake of learning and demonstration, I'll illustrate the impressive capabilities of SurrealDB.

We will need to add something like this:

(SELECT * FROM ->member_role->role->role_permission->permission) as permissions

Which will add something like this:

"permissions": [
  {
      "id": "permission:update",
      "name": "update content"
  },
  {
      "id": "permission:read",
      "name": "read content"
  },
  {
      "id": "permission:create",
      "name": "create content"
  }
],

In our example, everything appears neat and straightforward.
However, our user currently has just one role assigned.
When a user holds multiple roles, they might end up with the same permission through different roles. This could lead to duplicate entries in our results.

Fortunately, SurrealDB offers a set of handy helper functions. We'll employ array::group to eliminate these duplicates from our results.

The complete query:

SELECT *, (
      SELECT 
            (SELECT * FROM ONLY <-user) as user,
            (SELECT id FROM ->member_role->role).id as roles,
            array::group((SELECT * FROM ->member_role->role->role_permission->permission)) as permissions
      FROM <-tenant_member
      ) as members,
      (SELECT id FROM <-tenant_role<-role).id as availableRoles 
FROM ONLY tenant:car;

Result:

{
    "availableRoles": [
        "role:admin",
        "role:reader",
        "role:author"
    ],
    "id": "tenant:car",
    "members": [
        {
            "permissions": [
                {
                    "id": "permission:update",
                    "name": "update content"
                },
                {
                    "id": "permission:read",
                    "name": "read content"
                },
                {
                    "id": "permission:create",
                    "name": "create content"
                }
            ],
            "roles": [
                "role:author"
            ],
            "user": {
                "email": "john.doe@example.com",
                "firstName": "John",
                "id": "user:1",
                "lastName": "Doe",
                "name": "John Doe"
            }
        }
    ],
    "name": "Car Enthusiasts"
}

Final query 🎉

Let's remove the permissions and the ONLY Statement.
Our query, for fetching single or multiple tenants, is:

SELECT *, (
      SELECT 
            (SELECT * FROM ONLY <-user) as user,
            (SELECT id FROM ->member_role->role).id as roles
      FROM <-tenant_member
      ) as members,
      (SELECT id FROM <-tenant_role<-role).id as availableRoles 
FROM tenant;

Query user(s)

When we initiate a query for a user, our goal is to retrieve the user's details.

Additionally, we aim to determine which tenants the user belongs to and understand the roles associated with each of these tenant memberships.

We will utilize the same relationships as we did for the tenant query. The only distinction here lies in our approach: we will traverse most of the graph relationships in the reverse direction.
Because of this, I will skip the detailed explanation.

Our user query:

-- query user(s) and return related tenants, roles and permissions

SELECT *, (
      SELECT 
            (SELECT * FROM ONLY ->tenant) as tenant,
            (SELECT id FROM ->member_role->role).id as roles,
            array::group((SELECT * FROM ->member_role->role->role_permission->permission)) as permissions
      FROM ->tenant_member
      ) as tenants
FROM user;

Result:

[
    {
        "email": "john.doe@example.com",
        "firstName": "John",
        "id": "user:1",
        "lastName": "Doe",
        "name": "John Doe",
        "tenants": [
            {
                "permissions": [
                    {
                        "id": "permission:update",
                        "name": "update content"
                    },
                    {
                        "id": "permission:read",
                        "name": "read content"
                    },
                    {
                        "id": "permission:create",
                        "name": "create content"
                    }
                ],
                "roles": [
                    "role:author"
                ],
                "tenant": {
                    "id": "tenant:car",
                    "name": "Car Enthusiasts"
                }
            }
        ]
    }
]

Query information for a specific user within a particular tenant

As an addition to the previous queries, I will provide a query, which you might need in similar way in real world scenarios.

-- query the information for a specific user within a particular tenant and return user information, tenant information, roles and permissions

SELECT
      (SELECT * FROM ONLY <-user) as user,
      (SELECT *, (SELECT id FROM <-tenant_role<-role).id as availableRoles FROM ONLY ->tenant) as tenant,
      (SELECT id FROM ->member_role->role).id as roles,
      array::group((SELECT * FROM ->member_role->role->role_permission->permission)) as permissions
FROM ONLY tenant_member
WHERE in='user:1' and out='tenant:car'

Result:

{
    "permissions": [
        {
            "id": "permission:update",
            "name": "update content"
        },
        {
            "id": "permission:read",
            "name": "read content"
        },
        {
            "id": "permission:create",
            "name": "create content"
        }
    ],
    "roles": [
        "role:author"
    ],
    "tenant": {
        "availableRoles": [
            "role:admin",
            "role:reader",
            "role:author"
        ],
        "id": "tenant:car",
        "name": "Car Enthusiasts"
    },
    "user": {
        "email": "john.doe@example.com",
        "firstName": "John",
        "id": "user:1",
        "lastName": "Doe",
        "name": "John Doe"
    }
}

SurrealDB - Structuring Data for Multi-Tenant Role & Permission Systems

Sebastian Wessel — Fri, 08 Sep 2023 20:45:58 +0000

Let's dive into the vision of what we're aiming to create.

Our goal is to develop a versatile platform where multiple tenants and users can seamlessly coexist.
To illustrate, consider our initial set of tenants on this platform: car enthusiasts, cat owners, and musicians.

Here's where it gets interesting:
A single user can be a member of one or more of these tenants simultaneously.
This means that one user might proudly wear the hats of both a cat owner and a car enthusiast, while another user could be equally passionate about music and cars.
Moreover, within each tenant, users can hold distinct permissions.
For instance, a user might enjoy the full read and write access within the car enthusiasts' realm, but might only possess read privileges in the domain of musicians.
In order to streamline the management of permissions, we will organize them into user roles.
We also want to be able to create user roles for each tenant individually.

Define the base data tables

Let's transform the foundational data structure into SurrealQL.
While SurrealQL typically doesn't require explicit table definitions, doing so can enhance our understanding of the process.
Additionally, we'll incorporate schema definitions down the line to bolster data integrity.



-- Define the user table
DEFINE TABLE user SCHEMALESS;

-- Define the tenant table
DEFINE TABLE tenant SCHEMALESS;

-- Define the permission table
DEFINE TABLE permission SCHEMALESS;

-- Define the role table
DEFINE TABLE role SCHEMALESS;

Insert permissions

In our tutorial, we will have permission create, read, update and delete.
Inserting data is similar to regular SQL. You can use the INSERT statement, or the SurrealDB statement CREATE.



-- insert the create permission
CREATE permission:create SET name = 'create content';

-- insert the read permission
CREATE permission:read SET name = 'read content';

-- insert the update permission
CREATE permission:update SET name = 'update content';

-- insert the delete permission
CREATE permission:delete SET name = 'delete content';

Insert roles

Creating the roles itself is similar to creating a permission. However, in this context, we also aim to associate roles with permissions.
SurrealDB provides the RELATE statement. This will bring a whole set of graph-data-features.



-- insert the reader role
CREATE role:reader SET name = 'Content Reader';

-- create a relation from role reader to read permission
RELATE role:reader->role_permission->permission:read;

We'll keep track of the relationship between roles and permissions in a table called role_permission. This table has two essential fields: in and out.
Think of the data in this table as a kind of roadmap stored as a directed graph.
In our example, this connection goes from roles to permissions.
Remember this detail because it will become crucial when we retrieve data later on.

With this knowledge, we can add some more roles.



-- insert the creator role
CREATE role:author SET name = 'Content Author';

-- create a relation from role author to permissions
RELATE role:author->role_permission->permission:read;
RELATE role:author->role_permission->permission:create;
RELATE role:author->role_permission->permission:update;



-- insert the admin role
CREATE role:admin SET name = 'Administrator';

-- create a relation from role admin permissions
RELATE role:admin->role_permission->permission:read;
RELATE role:admin->role_permission->permission:create;
RELATE role:admin->role_permission->permission:update;
RELATE role:admin->role_permission->permission:delete;

Insert tenants

As we have now permissions and roles, we can now create our tenants.
It's pretty straightforward.
We create the tenant entry itself, and we link the roles to a specific tenant.
Keep the direction from role to tenant in mind.



-- create the tenant for our car enthusiasts
CREATE tenant:car SET name = 'Car Enthusiasts';

RELATE role:admin->tenant_role->tenant:car;
RELATE role:author->tenant_role->tenant:car;
RELATE role:reader->tenant_role->tenant:car;

-- create the tenant for our cat owners
CREATE tenant:cat SET name = 'Cat Owners';

RELATE role:admin->tenant_role->tenant:cat;
RELATE role:author->tenant_role->tenant:cat;
RELATE role:reader->tenant_role->tenant:cat;

-- create the tenant for our musicians
CREATE tenant:musician SET name = 'Musicians';

RELATE role:admin->tenant_role->tenant:musician;
RELATE role:author->tenant_role->tenant:musician;
RELATE role:reader->tenant_role->tenant:musician;

Insert users

To wrap things up, let's add some user entries to our database. We begin by creating a user entity entry. SurrealDB, by default, returns the entire newly created entry, allowing us to save this result in a variable for future use.

Next, we use this stored result to establish a relationship between the user and a tenant. Creating this relationship follows a similar process to regular entry creation, and once again, we receive a full entity as a result, which we can store in another variable.

Finally, in the last step, we establish a relationship from the user-to-tenant relationship to a role entry, completing the connection within our database.



-- create the user
LET $createdUser = (CREATE user:1 SET 
  name='John Doe',
  firstName = 'John',
  lastName = 'Doe',
  email='john.doe@example.com');

-- link the user John Doe to the tenant "car enthusiasts"
LET $memberRelation = (RELATE $createdUser->tenant_member->tenant:car);

-- link the member relationship to a role
RELATE $memberRelation->member_role->role:author;

-- finally, return the newly created user
RETURN $createdUser;

Structure

Here is a diagram of how the data is structured:

Here is the full code snipped.
You can use surrealist.app to run the SurrealQL.



-- Define the user table
DEFINE TABLE user SCHEMALESS;

-- Define the tenant table
DEFINE TABLE tenant SCHEMALESS;

-- Define the permission table
DEFINE TABLE permission SCHEMALESS;

-- Define the role table
DEFINE TABLE role SCHEMALESS;

-- insert the create permission
CREATE permission:create SET name = 'create content';

-- insert the read permission
CREATE permission:read SET name = 'read content';

-- insert the update permission
CREATE permission:update SET name = 'update content';

-- insert the delete permission
CREATE permission:delete SET name = 'delete content';

-- insert the reader role
CREATE role:reader SET name = 'Content Reader';

-- create a relation from role reader to read permission
RELATE role:reader->role_permission->permission:read;

-- insert the creator role
CREATE role:author SET name = 'Content Author';

-- create a relation from role author to permissions
RELATE role:author->role_permission->permission:read;
RELATE role:author->role_permission->permission:create;
RELATE role:author->role_permission->permission:update;

-- insert the admin role
CREATE role:admin SET name = 'Administrator';

-- create a relation from role admin permissions
RELATE role:admin->role_permission->permission:read;
RELATE role:admin->role_permission->permission:create;
RELATE role:admin->role_permission->permission:update;
RELATE role:admin->role_permission->permission:delete;

-- create the tenant for our car enthusiasts
CREATE tenant:car SET name = 'Car Enthusiasts';

RELATE role:admin->tenant_role->tenant:car;
RELATE role:author->tenant_role->tenant:car;
RELATE role:reader->tenant_role->tenant:car;

-- create the tenant for our cat owners
CREATE tenant:cat SET name = 'Cat Owners';

RELATE role:admin->tenant_role->tenant:cat;
RELATE role:author->tenant_role->tenant:cat;
RELATE role:reader->tenant_role->tenant:cat;

-- create the tenant for our musicians
CREATE tenant:musician SET name = 'Musicians';

RELATE role:admin->tenant_role->tenant:musician;
RELATE role:author->tenant_role->tenant:musician;
RELATE role:reader->tenant_role->tenant:musician;

-- create the user
LET $createdUser = (CREATE user:1 SET 
  name='John Doe',
  firstName = 'John',
  lastName = 'Doe',
  email='john.doe@example.com');

-- link the user John Doe to the tenant "car enthusiasts"
LET $memberRelation = (RELATE $createdUser->tenant_member->tenant:car);

-- link the member relationship to a role
RELATE $memberRelation->member_role->role:author;

-- finally, return the newly created user
RETURN $createdUser;

We now have a user named "John Doe" who is a member of the "car enthusiasts" tenant. He has been assigned the role of "Author," which grants him the permissions to "read," "create," and "update."

In the next part of this tutorial, we will take a closer look at how we can access this information.

DEV Community: Sebastian Wessel

Conclusion on using a QuickJS sandbox

Running Tests Inside QuickJS with TestRunner

Benefits of Running Tests in a Sandbox

Importing the Library

Writing Tests

Defining a Test Suite

Using Hooks

Asynchronous Tests and Hooks

Using Chai's expect

Examples of expect Usage

Running Tests

Custom Timeout

Example Usage

Conclusion

Secure Data Transformation in ETL Pipelines

Empowering AI: The QuickJS Package for LLM Tool Calling

Why QuickJS for AI Tool Calling?

Key Benefits for AI Developers:

Example: Implementing an AI Code Executor with LangChain

Enhancing AI Capabilities

Execute JavaScript in a WebAssembly QuickJS Sandbox

Typescript QuickJS Package: Empowering Secure JavaScript Execution in AI and Beyond

The Genesis of the QuickJS Package

A Sandbox Built for Developers, by a Developer

Features Designed with You in Mind

The QuickJS Package in Action

Leveraging Modern Tools for a Better Developer Experience

Real-World Applications

Performance and Security: No Compromises

Getting Started

How to Design a SurrealDB schema and create a basic client for TypeScript

Defining Your Schema

Using SurrealQL for Schema Definition

Leveraging Surrealist.app

Generating Zod Schemas and TypeScript Clients from SurrealDB

Understanding Generated Files

_generated Subfolder

schema Subfolder

client Subfolder

Using the Client

Alternative

Final words

In Closing

SurrealDB - The Magic Database to Keep on Your Radar

Sebastian Wessel ・ Sep 8 '23

PURISTA - Tests with Jest, Sinon.js and Testcontainers

Jest

Sinon.js

Testcontainers

PURISTA: Build with rimraf, esbuild, Turbo & git-cliff

rimraf

Turbo

Esbuild

git-cliff

PURISTA - Thanks to amazing open-source software

General setup

Coding Setup

ESLint and Prettier

SurrealDB - Improve data integrity by adding schema information

Final thoughts

SurrealDB - Query and combine data via relations

Query tenant(s)

Add available roles

Add tenant members

Add tenant member roles

Add tenant permissions for roles of members

Final query 🎉

Query user(s)

Query information for a specific user within a particular tenant

SurrealDB - Structuring Data for Multi-Tenant Role & Permission Systems

Define the base data tables

Insert permissions

Insert roles

Insert tenants

Insert users

Structure

Using Chai's `expect`

Examples of `expect` Usage

`_generated` Subfolder

`schema` Subfolder

`client` Subfolder