DEV Community: Olivier

Supabase helper for better RPC function typing with jsonb fields

Olivier — Thu, 09 May 2024 01:45:58 +0000

The problem

When you have those tables used for say vector search... you usually have a jsonb field for metadata like this:



create table
  documents (
    id text not null,
    vector extensions.vector null,
    content text not null,
     ---v--- this field
    metadata jsonb null, 
    constraint documents_pkey primary key (id)
  )

Where let's say you always have this in your jsonb metadata



 type Metadata = { 
    sourceUrl: string
    otherField: string
 }

But when you do a supabase call say supabase.rpc('match_vec_doc', args)... even if you create client supabse with Database from the typings provided... that metadata field will be Json and not of type Metadata.. which messes with your rest of your code!

So here is some help functions in addition to the type pulling you can get from



supabase gen types typescript --project-id [PROJECT_ID] > database.types.ts

The solution

Create a database_types.ts file in say you lib/supabase folder, and import from the generated types (you probably have some of this for Tables, Views):



import { type Database } from '@/database.types'

export type Tables<T extends keyof Database['public']['Tables']> =
  Database['public']['Tables'][T]['Row']
export type Views<T extends keyof Database['public']['Views']> =
  Database['public']['Views'][T]['Row']
export type Enums<T extends keyof Database['public']['Enums']> =
  Database['public']['Enums'][T]

export type Insert<T extends keyof Database['public']['Tables']> =
  Database['public']['Tables'][T]['Insert']

export type Functions = Database['public']['Functions']

Then add a MetadataField type mapping



// (... continue)
// Keys are the types for each metadata field here for document
export type MetadataField = {
  documentMetadata: { 
    sourceUrl: string
    otherField: string
  }
  defaultMetadata: unknown // this is the default for all other functions
 }

// Define a type that maps function names to keys in MetadataField
// Here I have 2 functions that both return metadata from the documents tables.. I have the k in keyof Functions to ensure we load all functions to avoid type errors
export type MetadataFieldMapping = {
  [k in keyof Functions]: string
} & {
  match_fts_doc: 'documentMetadata'
  match_vec_doc: 'documentMetadata'
  hybrid_search_doc: 'documentMetadata'
  match_fts_chunk: 'chunkMetadata'
  match_vec_chunk_text: 'chunkMetadata'
}

Then add an RPC type that you will use in your helper replacement function for rpc calls. It has a fallback



//  Define the RPC type using a generic parameter for dynamic metadata field
export type RPC = {
  [FunctionName in keyof MetadataFieldMapping]: {
    Args: Functions[FunctionName]['Args']
    Returns: Array<
      Functions[FunctionName]['Returns'][number] & {
        metadata: MetadataField[MetadataFieldMapping[FunctionName] extends keyof MetadataField
          ? MetadataFieldMapping[FunctionName]
          : 'defaultMetadata'] // Fallback to 'defaultMetadata' if the mapping does not exist
      }
    >
  }
}

The helper function



// Helper function to simplify RPC calls
export async function supabaseRPC<
  MethodName extends keyof MetadataFieldMapping
>(methodName: MethodName, args: RPC[MethodName]['Args']) {
  const supabase = createClient()
  return supabase.rpc<MethodName, RPC[MethodName]>(methodName, args)
}

How to use it

Instead of



supabase.rpc('match_vec_doc', args)

Use



supabaseRPC('match_vec_doc', args)

Example

You can see the metadata field is now properly typed. Note this example the schema is different than above.. but you get it.

Vercel AI SDK & LangChain streaming

Olivier — Mon, 29 Apr 2024 22:45:08 +0000

If you got stuck like me for hours on how to use the new Vercel AI SDK with Langchain to stream, the trick is not to use the LangChainStream from Vercel's here https://sdk.vercel.ai/docs/api-reference/providers/langchain-stream

But to use the .stream with an outputparser here
https://sdk.vercel.ai/docs/guides/providers/langchain

Here is a custom LangChainStream function

langChainStreamCustom.ts

import { createStreamDataTransformer } from 'ai'

export const LangChainStreamCustom = (
  stream: any,
  { onCompletion }: { onCompletion: (completion: string) => Promise<void> }
): ReadableStream => {
  let completion = ''
  const transformStream = new TransformStream({
    async transform(chunk, controller) {
      completion += new TextDecoder('utf-8').decode(chunk)
      controller.enqueue(chunk)
    },
    async flush(controller) {
      await onCompletion(completion)
        .then(() => {
          controller.terminate()
        })
        .catch((e: any) => {
          console.error('Error', e)
          controller.terminate()
        })
    }
  })
  stream.pipeThrough(transformStream)
  return transformStream.readable.pipeThrough(createStreamDataTransformer())
}

Then use it like all the other OpenAIStream AnthropicStream .. etc that Vercel SDK provides.

//....
response = llm.stream({}) // assuming this comes from a typical LangChain
const stream = LangChainStreamCustom(response, {
    onCompletion: async (completion: string) => {
      console.log('COMPLETE!', completion)
    }
})

return new StreamingTextResponse(stream)

Enjoy.

LoReFT and pyreft for surgical fine-tuning

Olivier — Sun, 07 Apr 2024 15:23:13 +0000

Here’s trying to understand and summarise the paper "ReFT: Representation Finetuning for Language Models"

https://arxiv.org/abs/2404.03592
https://github.com/stanfordnlp/pyreft

The paper proposes Representation Finetuning (ReFT) as a more parameter-efficient alternative to PEFTs like adapters and LoRA for adapting large language models to downstream tasks. Key ideas:

ReFT methods train task-specific interventions that modify hidden representations of a frozen base model, leveraging the insight from interpretability work that representations encode rich semantics.
The Low-rank Linear Subspace ReFT (LoReFT) variant uses 10-50x fewer parameters than leading PEFTs by intervening on representations in a learned low-dimensional subspace.
LoReFT provides state-of-the-art efficiency-performance tradeoffs on commonsense reasoning, arithmetic, instruction following, and language understanding tasks. On instruction tuning Llama-2 7B, it nearly matches GPT-3.5 using only 0.004% extra parameters.
Editing representations may be more powerful than modifying weights as done in PEFTs. LoReFT did struggle more on arithmetic reasoning, possibly due to long output lengths.

To make ReFT easy to use, the authors also introduce pyreft, a library that enables:

Fine-tuning any HuggingFace pretrained LM with ReFT
Configuring ReFT hyperparameters via config files
Easily sharing fine-tuned models on HuggingFace

Here is some pseudo-code showing a basic usage of pyreft to fine-tune a Llama-7B model:

from pyreft import get_reft_model, ReftConfig, ReftTrainerForCausalLM

# Load pretrained LM
model = AutoModelForCausalLM.from_pretrained("llama-7b-hf") 

# Configure LoReFT intervention
reft_config = ReftConfig(
  representations={
    "layer": 15, 
    "component": "block_output",
    "intervention": LoReftIntervention(embed_dim=model.hidden_size, rank=1)
  }  
)

# Wrap model with ReFT  
reft_model = get_reft_model(model, reft_config)

# Load training data
data_module = make_supervised_data_module(...)

# Configure training
trainer = ReftTrainerForCausalLM(
  model=reft_model,
  args=TrainingArguments(...),  
  **data_module
)

# Train model
trainer.train()

This fine-tunes a Llama-7B model by training a rank-1 LoReFT intervention at layer 15, modifying only 0.00006% of the model parameters. Once trained, the reft_model can be used for inference on downstream tasks.

By establishing ReFT as a promising new paradigm for LM adaptation, this work points to representation editing as a powerful lever for both model efficiency and interpretability. The pyreft library enables researchers and practitioners to easily experiment with and build on these ideas.

LangGraph idea generation

Olivier — Sun, 24 Mar 2024 06:01:46 +0000

TL;DR: Using a well written blog about langGraph and a Claude-3 prompt, I have a guide and starting point to build my own system using LangChain.

LangGraph takes a minute to wrap your head around but is brilliant. Rotem Weiss wrote a brilliant blog that explains it with a simple example from his GPT-Newspaper.

I used that blog's content (yes the full content) in combination with Claude-3-opus and my own idea (A research tool) that I needed to build a LangGraph for, and got a great response!

Copy-Paste the text from the blog in Claude then copy-paste this:

Given the logic of how LangGraph works given that example.
Help me come up with a LangGraph setup (help me with the 
logic of the nodes, the edges and the conditional edges 
knowing that we can define as many as we want and we also
define the functions and every fully customizable)... 
for a system that will Research a topic in XYZ. I have a
knowledge base that can be searched using a hybrid search 
(vector and keyword), I have the internet to search, I have a 
query deconstructor (to help break the user's research 
question down into multiple queries), I have a research 
planner that will help determine the tasks, I have a 
summarizer, and then a final_responder .. add any other 
functions or roles you can think of and give me the outline 
of the graph.

Here is the response I got!

Here's a proposed LangGraph setup for a system that researches a topic in XYZ based on the components you mentioned:

Nodes (Functions):

query_deconstructor: Breaks down the user's research question into multiple sub-queries.
research_planner: Determines the tasks and steps needed to answer the sub-queries.
knowledge_base_searcher: Searches the internal knowledge base using hybridSearch (vector and keyword).
internet_searcher: Searches the internet for relevant information.
information_filter: Filters and selects the most relevant information from the search results.
information_synthesizer: Combines and synthesizes the filtered information into coherent sections.
section_critic: Critiques and provides feedback on the synthesized sections.
section_reviser: Revises the sections based on the feedback from the critic.
summarizer: Summarizes the revised sections into a concise overview.
final_responder: Generates the final response to the user's research question.

Edges:

query_deconstructor -> research_planner
research_planner -> knowledge_base_searcher
research_planner -> internet_searcher
knowledge_base_searcher -> information_filter
internet_searcher -> information_filter
information_filter -> information_synthesizer
information_synthesizer -> section_critic
section_critic -> section_reviser (conditional edge)
section_critic -> summarizer (conditional edge)
section_reviser -> section_critic
summarizer -> final_responder

Conditional Edges:

section_critic -> section_reviser: If the critic finds issues or suggests improvements, the section is sent to the reviser for refinement.
section_critic -> summarizer: If the critic approves the section, it is sent to the summarizer.

Additional Functions/Roles:

data_visualizer: Creates visualizations and infographics based on the researched information to enhance the final output.
citation_manager: Manages and formats citations for the information used in the research.
glossary_generator: Generates a glossary of key terms and concepts related to the researched topic.
executive_summary_generator: Produces an executive summary of the research findings.

Outline of the Graph:

Start with the query_deconstructor to break down the user's research question.
Pass the deconstructed queries to the research_planner to determine the tasks and steps.
The research_planner initiates searches in the knowledge_base_searcher and internet_searcher.
The search results are passed to the information_filter for selection of the most relevant information.
The filtered information is sent to the information_synthesizer to create coherent sections.
The synthesized sections are critiqued by the section_critic.
If the section_critic suggests improvements, the section is sent to the section_reviser for refinement and then back to the section_critic for re-evaluation.
If the section_critic approves the section, it is passed to the summarizer.
The summarizer condenses the revised sections into a concise overview.
The summarized content is sent to the final_responder to generate the final response to the user's research question.
Additional functions like data_visualizer, citation_manager, glossary_generator, and executive_summary_generator can be integrated at appropriate points in the graph to enhance the final output.

Honestly, that's not bad for the fact that LangGraph is new and Claude 3 has absolutely no knowledge of LangChain's LangGraph.

Making sure you get JSON in your LLM response

Olivier — Wed, 10 Jan 2024 22:28:13 +0000

Using language models to return JSON is very useful.

For OpenAI you can just pass the response_format: { type: "json_object" } such as:

const completion = await openai.chat.completions.create({
    messages: [
      {
        role: "system",
        content: "You are a helpful assistant designed to output JSON.",
      },
      { role: "user", content: "Who won the world series in 2020?" },
    ],
    model: "gpt-3.5-turbo-1106",
    response_format: { type: "json_object" }, // <==== here
  });

For other models its not always reliable. It might say "Here is the JSON object you wanted... ". Or even add things at the end like. "I hope this helps!" 😡

Here is a function that will make sure you only get the JSON.

function parseJSONFromString(input: string): any {
    // Regular expression to find a JSON object or array
    const regex = /(\{.*\}|\[.*\])/s;

    // Extract JSON string using the regular expression
    const match = input.match(regex);

    // Check if a match was found
    if (match) {
        try {
            // Parse and return the JSON object/array
            return JSON.parse(match[0]);
        } catch (error) {
            // Handle parsing error (invalid JSON)
            console.error("Invalid JSON:", error);
            return null;
        }
    } else {
        // No JSON object/array found
        console.warn("No JSON object or array found in the string.");
        return null;
    }
}

// Example usage
const exampleString = "here is your JSON object: { \"key\": \"value\" }";
const result = parseJSONFromString(exampleString);
console.log(result);

Path mapping with Vercel Functions node.js typescript

Olivier — Thu, 08 Dec 2022 18:35:41 +0000

TLDR; Problem: path mappings in typescript Vercel Functions don't work. Solution: Compile your typescript before handing it over to Vercel! Works in local and production.

So you are super excited to port over your serverless functions to Vercel (Serverless) Functions and of course you used Typescript and path mappings (aka the import from @mylib stuff)... but then you realized path mappings don't work getting the error Can't find module x when you make the api call. You first think its your tsconfig.json setup or something to do with filename case sensitivity and git .. but it isn't you.. it's me.. I mean, it's Vercel

Indeed, if you actually read the docs you will see it says path mappings don't work: https://vercel.com/docs/concepts/functions/serverless-functions/runtimes/node-js#using-typescript-with-the-node.js-runtime

Most options are supported aside from "Path Mappings" and "Project References".

This is specifically for Vue builds because Next has its own setup for serverless functions in Vercel.

Here is a solution

Basically we are going to use a different folder to for our typscript version of the Vercel Functions and then compile them with the output directory being the /api folder that Vercel asks you to use for its serverless functions.

Create a source folder say /api-src for your serverless functions
Create a tsconfig.json with path mappings, in that folder because that is what the typescript compiler will read, add the outDir property to the /api (that's because Vercel wants serverless functions in the root /api folder
Install typescrypt and tsc-alias yarn global add typescript tsc-alias

tsc-alias is so that it replaces the @lib with the absolute path, once compiled to javascript.

Run tsc && tsc-alias from the /api-src folder

A /api folder gets created (because we told it to in the outDir value in the tsconfig.json in the /api-src folder) and the code gets compiled to javascript. Notice the .map files, those are because we want debugging to be easier. That is set by sourceMap : true value in the tsconfig.

Now you can run vercel dev and no more Can't find module x problems.

Try it out! Let me know if I missed something!

PS:

Consider adding dom to your lib array in tsconfig to avoid errors in compilations with some modules.
Consider moving the .ts files to a subfolder api-src/src so separate code from tooling. This will also help with project references if you have any
As an alternative to tsc-alias you can use tscpaths, which works well in conjunction with tsc-watch for live development: tsc-watch --build --onSuccess "tscpaths -p tsconfig.json -s ./src"

5 git tips for beginners

Olivier — Wed, 26 Jan 2022 21:54:46 +0000

Practical commands for efficiency and a clean tree

1. Shortcuts (aka Alias)

You will find tons of alias ideas out there, the ones I found using the most are these. You can edit them using nano ~/.gitconfig in mac/linux.
So instead of typing git checkout <branch> you can type git co <branch>

[alias]
   co = checkout
   br = branch
   st = status
   # create a new branch from current branch and check it out
   cob = checkout -b
   # stage all changes and commit (add a message)
   a = !git add -A && git commit -m
   unstage = reset HEAD --
   last = log -1 HEAD
   hist = log --pretty=format:'%h %ad | %s%d [%an]' --graph --date=short
   # Meld the currently unstaged changes into the last commit
   fixup = !git add -A && git commit --amend --no-edit

I will be using these throughout this article.

3. Bug and feature branches

From your main or dev branch use git cob bug/123 to create a branch that will be used to work on issue #123 .. you can use bug/123 or issue/123 if you want to distinguish between bugs, tasks, features. This will organize them in folders when using git gui tools because of the forward slash.

4. Change your git editor to nano

Unless you where born in the 1980s you likely hate vi or vim editor, you can change it with:

git config --global core.editor "nano"

Use Ctrl+o to save (ie o for output), Ctrl+x to quit, Ctrl+w to search

5. Clean up your commits

When creating a bug branch called say bug/123 you will likely have tons of wip ie. work in progress commits. If you want to squash them all down to 1 commit before publishing the branch, then you can do this with git rebase -i
You want to go from this:

To this:

# Open interactive rebase with the last 6 commits
git rebase -i HEAD~6

That will open nano or vi with the ability to tell git what to do:

What we want is to squash all the commits together.
The top commit is the oldest one so you need to pick that one because squash will squash with the previous commit. If you don't one to squash them all onto that commit (maybe you wanted the next or previous one as the starting point, just exit and change the HEAD to HEAD~5 or ~7. Caution: make sure to comment out all the lines with # so that the rebase does nothing!

Then save (Ctrl+o in nano, or <Esc> :wq in vim). It will then let you pick the commit message for that rebase, just comment out (#) all of them and type your own or just pick the best one.

Tip: If you have already published the branch (and all those noisy commits) you are working on, then rebasing it locally won't change it remotely. So delete the remote branch first git push origin --delete bug/123 (assuming you use origin as name of your remote) then publish it after the rebase, or else the next time you git pull it will pull those commits back into your local branch.

🤗 Thanks for reading. Open to feedback and other tips!

10 javascript basics interns should know before the job

Olivier — Sat, 15 Jan 2022 18:05:49 +0000

tldr; learn typescript and es6 |

I have spent a lot of time doing crash courses on JS and TS basics instead of using the time to work on the actual code. Here is what I would recommend anyone know before getting an internship or job for js related work.

1. Typescript . Yes, learn typescript before getting any javascript job. It's a steep learning curve and you will struggle so much trying to understand it if you don't have the basics. See this course on egghead.io or this longer course on udemy by our friend Max. And try and remember: typescript doesn't run at runtime!

2. Arrow functions

const coolFunction = () => returnValue // FAVOUR THIS
const coolFunction = function() { return returnValue } // AVOID THIS, UNLESS NEEDED

3. Template literals

let fruit = "oranges"
const stringValue = `I love ${fruit}` // I love oranges

const fancier = `I love ${favFruit()?.answers?.[0]?.value ?? "fruit"}` // See below for what this means

You could use || instead of ?? read more about nullish coalescing

4. Property shorthand

let a = "something"
let b = "something else"
const myObject = { a, b } // same as { a: a, b: b}

5. Destructuring assignment

let { name, email } = contact // same as name = contact.name..

// or go deeper - careful as contact needs 
// to exist and wont be set as variable, only address will
let { contact : { address}} = person // address = person.contact.address

6. Spread operators
Merge arrays and objects easily

let stuff = [ "bye", false, 3 ]
var mergedStuff = [ 1, 2, ...stuff ] // [ 1, 2, "bye", false, 3 ]

7. Optional chaining
Only use if ... then when you need it. Use optional chaining instead.

// this is a getter, ie computed type variable
// learn that too!
get pronouns() { 
// safely returns undefined rather than 
// blowing up with "can't get property x of undefined"
 return person?.details?.pronouns 
}

// How to use it with functions and arrays:
let email = getSomething()?.email
// You could also do this:
let { email } = getSomething();

let deepThing = myFunction()?.someArrayProp?.[0]?.detail

8. Common JS methods
Don't shy away from the MDN webdocs, see e.g. .some

let transformedArray = myArray.map(...)
let anArray = myArray.filter(...) // filters out
let aBoolean = myArray.some(...) // .includes, .many, .every
let anItem = myArray.find(...) // returns first item

9. Lodash
Mainly, _.get, _.set, _.uniq, _.omit, _.difference
You will find it in lots of codebases you work on, but many of these are available in vanilla js.

10. JS Doc


/**
 * Documenting stuff matters
 * @param thing - An important input
 * @returns otherthing - Clearly expected result
 */
const gardenFunction = (thing: string) => otherthing

Combining these learnings you need to be able to write and understand:

type Contact = {
    readonly address: string;
    readonly email?: string;
};

type Person = {
    readonly name: string;
    readonly contact: Contact;
};

const listOfPeople: ReadonlyArray<Person> = [];

/**
 * Find a person's email by name (case insensitive).
 *
 * @param name Name you are looking for.
 * @returns The person's email or `undefined` if not found.
 */
const findPersonEmailByName = (name: string) =>
    listOfPeople.find(
        person => person.name.toLocaleLowerCase() === name.toLocaleLowerCase(),
    )?.contact.email;

Quick conversion of Enums into Tables for Hasura

Olivier — Sun, 14 Feb 2021 18:24:41 +0000

tldr; I have a ton of Enums in my pretty schema but now Hasura tells me they want tables instead! Here is a script #regexForTheWin

Why?

Hasura recommends Tables versus Enum types see why here

How it works.js

Given a sql string with CREATE TYPEs. .. in it, convert them to CREATE TABLE.

Don't forget

Tell hasura to use those tables as enum tables and don't forget to update the fields that used to reference your enums by pointing to the table and adding foreign keys. For example:

ALTER TABLE myschema.users ADD CONSTRAINT
  myschema."usersStatus_fkey" FOREIGN KEY (status) REFERENCES myschema."Status";

Here is the gist of it:

What you get

Using a GraphQL schema to generate your Hasura database

Olivier — Sat, 13 Feb 2021 19:22:20 +0000

tldr; You have huge amount (30+) models/tables in a SDL-type graphql schema (like Fauna)? Use Prisma migrate to generate a postgres database (or update your schema) in Hasura, dump it, clean it, and track it all again in Hasura.

Say you have a (large) schema written in SDL'ish like this

model Post {
  id        Int      @default(autoincrement()) @id
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
  title     String
  content   String?
  published Boolean  @default(false)
  author    User     @relation(fields: [authorId], references: [id])
  authorId  Int
}
model User { ... }
## 40+ other models!

that you would like to use to generate a new database/schema in Hasura (i.e. the Postgres database). This is how we will do it:

Use Prisma Migrate to populate the Postgres Database
Use Hasura metadata "reload" so that Hasura gets back in sync with the postgres database
Track the tables and relationships in Hasura
Realize its not exactly what you want so dump it and clean it, and re-track it.

🎉 Voilà!

PS: I am not affiliated to Fauna, Prisma or Hasura.

OK let's do this...

Getting Started

Make sure you have Hasura or Hasura Cloud (its free) project running. In this example we use Hasura Cloud.

1. Bring Prisma into to your project

This is just to use prisma migrate. This is based on this start guide from prisma.

Create a folder called prisma_migrate or prisma_stuff anything so that the prisma init later doesn't mess up your project's tooling.

$ mkdir prisma_migrate; cd prisma_migrate

Install prisma and typescript

$ npm install prisma typescript ts-node @types/node --save-dev

Add a tsconfig.json in your prisma_migrate folder

{
  "compilerOptions": {
    "sourceMap": true,
    "outDir": "dist",
    "strict": true,
    "lib": ["esnext"],
    "esModuleInterop": true
  }
}

Initiatilize prisma to get the starter files you need. It will create a prisma directory inside your prisma_generate folder for you and put schema.prisma in there and also .env <-- this is why we created a sub folder, else that would override your own .env

$ npx prisma init

2. Point Prisma to your Postgres database

Copy the HASURA_GRAPHQL_DATABASE_URL variable in your Hasura project settings and paste it in the DATABASE_URL variable of the .env file that Prisma generated
Now if you want prisma to generate this into a new schema, just add a ?schema=myapp_test that will create a new schema in your postgres and Hasura will pick it up later. (This avoids overrides of any existing shcema you have in Hasura), so your .env file looks something like this (see the end):

DATABASE_URL="postgresql://postgres:XXXXX@potatoes.ce6gsvwmjbvx.us-east-2.rds.amazonaws.com:5432/postgres?schema=potatoes_dev"

Notice that I am using AWS RDS to host my Postgres database, but you could use anything that Hasura supports.

3. Get your SDL schema

OK. We are ready to edit the schema.prisma file by adding our existing schema. This needs to be compatible with Prisma (see prisma's guide for more). So if you have say a Fauna DB SDL schema, make sure to adjust it (still quicker than writing your own SQL migration files in Hasura!)

model Post {
  id        Int      @default(autoincrement()) @id
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
  title     String
  content   String?
  published Boolean  @default(false)
  author    User     @relation(fields: [authorId], references: [id])
  authorId  Int
}
model Profile {
  id     Int     @default(autoincrement()) @id
  bio    String?
  user   User    @relation(fields: [userId], references: [id])
  userId Int     @unique
}
model User {
  id      Int      @default(autoincrement()) @id
  email   String   @unique
  name    String?
  posts   Post[]
  profile Profile?
}

4. Generate the schema/database in postgres

Now that you have your schema in a happy Prisma SDL format you can run prisma migrate to leverage Prisma's magic in generating Postgres tables.

$ npx prisma migrate dev --name init --preview-feature

That will create the 3 tables in Postgres.

5. Reload your Hasura metadata

You can now reload your schema in Hasura cloud using the cli or the console learn more here.

6. Track all the new tables and relationships

Go to your new schema in the Hasura console track the new tables and relationships. See how here.

Before

After

Track the relationships

🚀 You can now use GraphQL with all the queries and mutations ready!

7. Some gotchas

7.1 Coming from Faunadb

Missing id fields: if you are coming from Fauna, you will need to add your own id fields and create_at and update_at as Fauna generates the _id and _ts itself. I wrote a little script that parsed the .graphql file and injects it at the top of each model/type, something like this:

lines.push("id           Int      @id @default(autoincrement())")
lines.push("created_at   DateTime @default(now())")
lines.push("updated_at   DateTime @updatedAt")

Changing Long and Date/Time fields: Fauna uses Long, and DateTime. We need to change those, here is a handy script (assuming you are parsing your file and line is each line):

if (line.includes(":")) {
    const regex = /([^:]*):\s*(\[?)([^@!\]#/]*)(\]?)!?/m;
    const subst = `$1\t\t$3$2$4? `;
    line = line.replace(regex, (m:any,g1:string,g2:string,g3:string,g4:string) => {
        if(g3.trim() == "Time") {
            return `${g1}\t\tDateTime${g2}${g4}? @db.Time() `
        } else if(g3.trim() == "Date") {
            return `${g1}\t\tDateTime${g2}${g4}? @db.Date `
        } else {
            g3 = g3.replace("Long","Int");
            return `${g1}\t\t${g3}${g2}${g4}? `
        }
    });
    // remove optional list (i.e. []?) not allowed in prisma
    line = line.replace("[]?","[]");
    return line;
  }

7.2 Prisma don't like `uuid()`

Prisma's uuid() function as a @default gets ignored by Hasura, so you will need to update the metadata in Hasura to use their gen_random_uuid() instead.
Once you change the schema with a new function, running the prisma migrate command will cause Prisma Migrate to panic with a ✔ Drift detected: Your database schema is not in sync with your migration history. message, forcing a full reset. So I suggest just using Prisma migrate for an initial port of your schema.

8. Still having issues?

You might find it easier after doing the first prisma migrate to dump the schema to a .sql file and then do some batch editing to change things like id INT to id uuid etc., here is how:

8.1 Dump the schema

Use Postman or curl to POST to the pd_dump of Hasura:

// endpoint: POST https://xxxxxxx.hasura.app/v1alpha1/pg_dump
// body:
{
  "opts": ["-O", "-x", "--schema-only", "--schema", "potatoes_dev"],
  "clean_output": true
}

8.2 Edit the .sql file

The above will return a dump of the schema, starting with CREATE SCHEMA...

You can edit the file to "replace all" for things like table names and relationships, just keep in mind the following:

When changing from INT to UUID for your table id:

you no longer need the CREATE SEQUENCE.. for those id columns,
make sure to replace the foreign keys for example if the Author table has a uuid as id, then the Post table's authorId column needs to change from authorId int to authorId uuid in its CREATE TABLE... section

Prisma uses A and B for many-to-many relationships. I'd suggest replacing those with more readable names like PostId and TagId for the _PostTag relationship table.

-- Ugly and unreadable! --
CREATE TABLE "_PostTag" (
"A" integer NOT NULL,
"B" integer NOT NULL
)

-- Way better --
CREATE TABLE "_PostTag" (
"postId" integer NOT NULL,
"tagId" integer NOT NULL
)

8.3 Delete the schema and run the edited .sql file

You can now delete the schema from within the Hasura console, and run the above sql file from either the console or using any Postgres admin tool like pgAdmin. pgAdmin gives great feedback incase you didn't do a proper clean up or rename of items in our sql file.

8.4 Reload the schema in Hasura

Refresh the Hasura console and notice the schema is there, track the tables and relationsips as described in 6. above