DEV Community: Kaushik Varanasi

I wrote a Datadog to Grafana monitor translator

Kaushik Varanasi — Tue, 29 Sep 2026 06:50:40 +0000

I spent a week writing a translator from Datadog monitors to Grafana alert rules. It compiled cleanly, produced sensible-looking PromQL, and passed 18 unit tests.

Then I loaded the output into an actual Grafana and watched an alert fire, drop below its threshold, and keep firing. Forever.

This is what I learned, in the order the mistakes found me.

Why translate monitors at all

OpenTelemetry won instrumentation. Telemetry is portable now — same OTLP, same conventions, pointable anywhere. The industry read that as "the backends are commoditised."

It isn't quite true. The lock-in moved up a layer.

A team with 900 Datadog monitors cannot leave Datadog. Not because the data won't walk — it will. Because of the person-years encoded in DQL: the composite monitors, the anomaly detectors someone tuned in 2023, the notification routing, and the institutional knowledge that monitor #447 firing means check replica lag first.

Every vendor's answer to this is "we're OpenTelemetry-native, migration is easy." That's a claim about the data layer. It's true, and it's irrelevant. The question that actually decides the renewal — what happens to my 900 monitors — gets no answer at all.

So I tried to answer it.

Chapter 1: The query is 20% of a monitor

Here's a Datadog monitor:

avg(last_10m):avg:system.cpu.user{env:prod} by {host} > 90

Translating that to PromQL is trivial. An LLM does it in one shot, correctly:

avg by (host) (avg_over_time(system_cpu_user{env="prod"}[600s])) > 90

Ship it and you have destroyed five behaviours, none of which appear anywhere in the query string:

Datadog option	What it does	Grafana
`no_data_timeframe: 15`	alert after 15 min without data	fires on the first empty evaluation
`critical_recovery: 75`	real hysteresis — alert at >90, recover at <75	recovers at the alert threshold, so it flaps in the band
`evaluation_delay: 60`	wait 60s for late-arriving points	no equivalent
`new_group_delay: 600`	don't page for a freshly-booted host	no equivalent
`require_full_window`	refuse to fire on a partial window	always evaluates whatever came back

The alert still exists after your migration. It just fires differently. And you discover that during an incident, which is why nobody attempts this.

This is the whole reason the project exists: the query is about 20% of a monitor, and the other 80% is undocumented evaluation semantics.

So the translator carries every one of those into an intermediate representation, and anything the target cannot express becomes a caveat attached to the output. Never a silent drop.

Chapter 2: PromQL comparisons filter

Now the bug from the opening.

The emitted rule looked right:

avg by (host) (avg_over_time(system_cpu_user{env="prod"}[600s])) > 90

I drove CPU to 95. It fired. I dropped it to 78. It stayed firing. I checked the data directly:

avg_over_time = 78
threshold     = 90
rule state    = firing
rule health   = ok

PromQL comparison operators filter. When the condition is false, the query returns no series. And Grafana cannot distinguish:

no series because the condition is false
no series because the exporter died

Both are NoData. So with noDataState: Alerting — which is exactly what Datadog's notify_no_data: true should map to — the rule fires the moment your metric drops below the threshold, and never clears.

Every monitor in the migration with no-data alerting enabled would have been permanently stuck on.

The fix is Grafana's own two-node shape. The query returns a value; a separate threshold expression evaluates it:

data:
  - refId: A
    datasourceUid: my-prom
    model:
      expr: avg by (host) (avg_over_time(system_cpu_user{env="prod"}[600s]))
      instant: true
  - refId: B
    datasourceUid: __expr__
    model:
      type: threshold
      expression: A
      conditions:
        - evaluator: { type: gt, params: [90] }
condition: B

Now "below threshold" is a false condition and "exporter died" is NoData. Two different states, with two different configured behaviours.

If you write Grafana alert rules by hand and you put the comparison in the PromQL, go and check them. This is not specific to migration.

Chapter 3: `for` is not `no_data_timeframe`

While fixing that, I found a second one sitting underneath it.

I had mapped Datadog's no_data_timeframe onto Grafana's for duration. They sound similar. They are completely different things:

no_data_timeframe — how long to wait before alerting on missing data
for — how long a breached threshold must hold before firing

Conflating them meant every monitor with notify_no_data: true sat in Grafana's pending state for fifteen minutes during a real outage, while Datadog would have fired at the next evaluation.

The correct value is 0s, and not as a cop-out. Datadog's time rollup is already baked into the PromQL range selector — avg_over_time(...[600s]). Any non-zero for applies the window twice: average over ten minutes, then hold for ten more.

// Wrong. Two different concepts, one variable.
const forDuration = ir.noData.timeframeSeconds
  ? `${ir.noData.timeframeSeconds}s`
  : `${ir.evaluation.windowSeconds}s`;

// Right. The window is in the range selector; `for` must not repeat it.
const forDuration = '0s';

no_data_timeframe genuinely has no Grafana equivalent. It's reported as a caveat rather than approximated.

Chapter 4: Datadog's own monitors broke my parser

I built a realistic estate in a live Datadog account — 31 monitors across golden signals, host saturation, Kubernetes, Postgres, Kafka, SQS, and business metrics — and started streaming telemetry into it.

Then Datadog auto-provisioned nine of its own recommended monitors once it detected the integrations.

Those nine turned out to be a far better test corpus than anything I had written, because they use syntax I didn't know existed:

max(next_3d):forecast(
  (1 - avg:system.disk.free{*} by {host,device}
     / avg:system.disk.total{*} by {host,device}) * 100,
  "seasonal", 1, interval="60m", seasonality="weekly"
) >= 99

Keyword arguments. interval="60m", seasonality="weekly", direction='above', count_default_zero='true'. My lexer had never seen an = that wasn't part of >=. None of my hand-written fixtures contained one.

And it failed badly. A LexError escaped the compiler and killed the entire run — one unparseable monitor took down all forty. For a tool whose entire job is reporting honestly on a large estate, that's the worst possible failure mode.

} catch (err) {
  // ANY failure reading the query is a refusal, not a crash.
  findings.push(caveat('unparseable_query',
    `${err.name}: ${err.message}`,
    { level: Level.BLOCKER, ddValue: dd.query }));
  return finish(dd, null, findings);
}

The lesson I'd pass on: your fixtures lie. Mine were realistic, reviewed, and still wrong in ways only a real account exposed. If you're building anything that parses a vendor's DSL, get real input early — and note that Datadog hands you a free corpus of professionally-written monitors the moment telemetry arrives.

The Datadog API taught me four more constraints by rejecting my monitors outright:

forecast monitors need a future window (next_1w), not last_*
that window must be between 12h and 3mo
forecast supports only min/max aggregators, not avg
service checks require a grouping (.by("host"))

Chapter 5: Refusing is a feature

Against the 40 live monitors:

40 monitors
├─    0  translated exactly            0.0%
├─   31  translated with caveats       77.5%
└─    9  UNTRANSLATABLE                22.5%

Nine refused, each by name with an actionable reason:

3  anomalies()    proprietary model — the band can't be derived from the query
2  forecast()     proprietary extrapolation
1  outliers()     proprietary inter-series clustering
1  log monitor    different query grammar
1  composite      references other monitors by id
1  service check  no query — aggregates check statuses

This is the part I'd defend hardest. Asked to translate anomalies(), a language model will happily emit a plausible static threshold. It looks right. It is not right, and nobody audits the alert that still exists.

A confidently-wrong translation of your seven most important monitors is strictly worse than a missing one, because the missing one gets noticed.

Chapter 6: Why deterministic, and not an LLM

For a single monitor, an LLM is fine. Genuinely — it'll do it correctly right now.

Three reasons the pipeline is deterministic anyway:

Auditability. "Why did monitor 447 translate this way?" — "the model decided" is not an answer when that rule guards production.
Consistency at scale. 900 monitors through an LLM is 900 slightly different decisions. Monitors 47 and 612 share an idiom and translate two different ways, which becomes unmaintainable the first time someone edits one.
Diffability. The output gets compared month over month. Anything whose key ordering wanders produces noise that hides real change.

Tested, not asserted:

test('DETERMINISM: identical input produces byte-identical output', () => {
  const once  = stableStringify(monitors.map(compileMonitor).map(emit));
  const twice = stableStringify(monitors.map(compileMonitor).map(emit));
  assert.equal(once, twice);

  // And key order must not depend on input key order.
  const shuffled = monitors.map(m =>
    Object.fromEntries(Object.keys(m).reverse().map(k => [k, m[k]])));
  assert.equal(once, stableStringify(shuffled.map(compileMonitor).map(emit)));
});

There is one legitimate place for a model: proposing translations for the minority that fail deterministic compilation — with a differ as the judge, and nothing shipping until parallel evaluation agrees. Arithmetic decides; the model assists.

Chapter 7: Proving it, instead of claiming it

Every caveat the compiler emits started life as something I believed about Grafana. That's not good enough when the caveat is the product.

So the repo ships a docker-compose harness — Prometheus, Grafana, and a controllable exporter whose values a test drives directly. No API keys: Grafana provisions alert rules from files on disk.

The headline claim under test was the hysteresis caveat:

Datadog recovers at a separate threshold, so an alert at >90 recovering at <75 stays firing between 75 and 90. Grafana recovers at the alert threshold, so this monitor will flap in that band.

Datadog monitor: critical 90, warning 80, critical_recovery 75.

1. CPU -> 95                                  firing   after  9s
2. CPU -> 78  (inside Datadog's 75-90 hold)   inactive after 10s
3. stop sending data entirely                 firing   after  0s

Step 2 is the difference. At 78, Datadog would still be holding. Grafana clears.

The caveat is now observed rather than remembered.

A confession on that harness: three separate failures during this were my test, not the translator. Each time I changed where I read state from without changing what I expected to find. Grafana reports two vocabularies and they are easy to conflate:

rule level      firing | pending | inactive
instance level  Alerting | Pending | Normal | NoData | Error

Threshold behaviour must be read per instance — the rule is avg by (host), so a stale series from an earlier run on a different host keeps the rule firing and makes your test lie. But NoData is raised as a separate DatasourceNoData instance carrying none of your labels, so it's only visible at rule level.

What's still not verified

The Grafana side is measured. The Datadog side is still partly reasoned.

I have not sat with a live monitor, cut the data, and recorded exactly when no_data_timeframe fires. I have not walked a metric through a recovery band on the Datadog side and watched it hold. Those claims come from understanding rather than observation, and the honest thing is to say so in the README rather than let a confident-sounding table imply otherwise.

That's next, and the account now exists to do it.

Also missing, and it's the real product: the differ. Both engines evaluating the same telemetry for thirty days, producing "847 identical, 41 differ only in evaluation delay, 12 untranslatable." That table is what someone takes into a renewal conversation. Everything above is just the compiler that sits in front of it.

Try it

git clone https://github.com/Rocketgraph/translate
cd translate
npm test                                        # 25 tests, zero dependencies
node bin/ddtranslate.mjs fixtures/sre-standard.json

And to watch the behaviour rather than trust me:

cd demo
docker compose up -d --build
node provision.mjs ../fixtures/sre-standard.json
node verify.mjs

MIT. Zero runtime dependencies — npm test needs nothing installed beyond Node.

Issues and PRs welcome, particularly if you have monitors it refuses that you think it shouldn't. That list is the most interesting part of the project.

Disclosure: I build an observability product. This is not it — it translates to Grafana and Prometheus, and works fine for people who never buy anything from me.

🪄✨How I built this Twitter scheduler using React and Hasura🔥

Kaushik Varanasi — Sat, 20 Jan 2024 12:00:05 +0000

TL;DR 🔥

In this tutorial, you'll learn how to build a tweet scheduler. Enter your Twitter credentials and select a time and a date and your tweet will be tweeted at that time.

Rocketgraph: An open-source replacement to Firebase 🚀

Just a quick background about us. Rocketgraph lets developers build web applications in minutes.
We do this by providing:

PostgresDB hosted on AWS RDS
Hasura console to GraphQLise your PostgresDB, manage your permissions, crons, schedulers and a lot more.
Built-in authentication that supports email/password, social, magic link and OTP
Serverless functions for any additional backend code.

Before we start

We are going to be using Hasura on-off schedulers to trigger Twitter API at the appropriate time and Rocketgraph's edge functions to setup the backend. So we need to create a project on Rocketgraph. It's free.

Signup

And create a project

It will setup a Hasura console. For now you don't need to know how to use it. Since we will be using it via the Hasura API.

Let's set it up 🆙

We'll be using React.js and shadcn UI for the front-end and Node.js for our serverless functions.

create a folder for project, client and server:

mkdir twitter-app && cd twitter-app
mkdir client server

Setting up Lambda functions

With Rocketgraph you can easily create lambda functions that act as your backend. To setup, note the project name from your project console on Rocketgraph:

npm init -y
npm install express body-parser axios axios-oauth-1.0a
npm install @codegenie/serverless-express

The last install is for express on the AWS Lambdas. You don't need to worry about that as Rocketgraph takes care of Lambda deployments.

Create a file index.js and add the following code

const serverlessExpress = require('@codegenie/serverless-express')
const app = require('./app')
exports.handler = serverlessExpress({ app })

Now create a file named app.js and add the following code:

// jshint esversion: 6

const express = require("express");
const bodyParser = require("body-parser");
const axios = require("axios");
const addOAuthInterceptor = require("axios-oauth-1.0a");

const app = express();
const port = 3000

app.use(express.json());

const APP_URL = "https://guhys2s4uv24wbh6zoxl3by4xa0im.lambda-url.us-east-2.on.aws/tweet";



app.post("/tweet", function (req, res) {
    // Create a client whose requests will be signed
    const client = axios.create();
    // Specify the OAuth options
    const payload = req.body.payload;
    const options = {
        algorithm: 'HMAC-SHA1',
        key: `${payload.key}`,
        secret: `${payload.secret}`,
        token: `${payload.token}`,
        tokenSecret: `${payload.tokenSecret}`,
    };

    const data = {
        "text": `${payload.tweet}`
    };

    console.log("addOAuthInterceptor is not a function: ", addOAuthInterceptor)
    // Add interceptor that signs requests
    addOAuthInterceptor.default(client, options);

    console.log("params: ", req.body);
    client.post("https://api.twitter.com/2/tweets", 
    data)
    .then(response => {
        console.log("resp: ", response, req.body.key);
        res.status(200).send(`<h2> the result is : Tweeted </h2>` );
    })
    .catch(err => {
        console.log(err);

        res.status(500).send(`<h2> the result is : Error </h2>` );
    });
});

app.post("/schedule-tweet", function (req, res) {
    const body = JSON.parse(req.body);
    console.log("body", body)
    const jsonb = {
        "type": "create_scheduled_event",
        "args": {
            "webhook": `${APP_URL}`,
            "schedule_at": `${body.schedule_at}`,
            "include_in_metadata": true,
            "payload": {
                "tweet": `${body.tweet}`,
                "algorithm": 'HMAC-SHA1',
                "key": `${body.key}`,
                "secret": `${body.secret}`,
                "token": `${body.token}`,
                "tokenSecret": `${body.tokenSecret}`,
            },
            "comment": "sample scheduled event comment"
        }
    }
    console.log(req.body);

    axios.post("https://hasura-vlklxvo.rocketgraph.app/v1/metadata", jsonb, {
        "headers" : {
            "Content-Type": "application/json",
            "X-Hasura-Role":"admin",
            "X-hasura-admin-secret":"********"
        },   
    }).then(function (response) {
        console.log("response: ", response);
        res.status(200).send({
            message: "Your result is tweeted"
        })
    }).catch(function (error) {
        console.log("Error from Hasura: ", error);
        res.status(500).send(error)
    });
})

app.listen(port, () => console.log(`calculator listening on port ${port}!`))

module.exports = app;

Notice the APP_URL? Leave it empty for now. We will get this URL once we deploy the function to prod. So now let's go ahead and do that.

Also replace X-hasura-admin-secret and HASURA_URL with the ones shown on your project's Hasura tab

Deploy the function ⛈️

Rgraph provides a cute CLI to assist you to develop your functions to AWS lambda:

npm install -g rgraph@0.1.10

And with your project name, execute this to deploy your function:

rgraph deploy <function name>

Build the frontend 👼

Create project

cd client
npx create-next-app@latest my-app --typescript --tailwind --eslint

Add shadcn ui

npx shadcn-ui@latest init

Add necessary components for our app

npx shadcn-ui@latest add button popover calendar card input label

Create UI

In src/app/page.tsx

"use client"

import { Button } from "@/components/ui/button"
import {
  Card,
  CardContent,
  CardDescription,
  CardFooter,
  CardHeader,
  CardTitle,
} from "@/components/ui/card"
import { Input } from "@/components/ui/input"
import { Label } from "@/components/ui/label"
import React from "react"
import { useState } from "react"
import { DatePickerDemo } from "./date"


type ScheduleTweetInput = {
  key: String,
  secret: String,
  token: String,
  tokenSecret: String,
  tweet: String,
  date: any,
  time: any,
}

const sendTweet = async (twitterData: ScheduleTweetInput) => {
  const { date, time, tweet, key, secret, token, tokenSecret } = twitterData;
  console.log(time.split(":"))
  date.setHours(...time.split(":"))
  console.log(twitterData, date);
  const isoDate = date.toISOString()
  console.log(isoDate);
  const url = "https://guhys2s4uv24wbh6zoxl3by4xa0imsdb.lambda-url.us-east-2.on.aws/schedule-tweet"
  const postData = {
      "tweet": tweet,
      "schedule_at": isoDate,
      "key": key,
      "secret": secret,
      "token": token,
      "tokenSecret": tokenSecret
  }


  let headersList = {
    "Accept": "*/*",
    "User-Agent": "Thunder Client (https://www.thunderclient.com)",
    "Content-Type": "application/json"
   }

   let bodyContent = JSON.stringify(postData);

   let response = await fetch(url, { 
     method: "POST",
     body: bodyContent,
     headers: headersList,
     mode: "no-cors"
   });

   let data = await response.text();
   console.log(data, response.status, response.json);
   console.log("Status", response.status, response);
}

export default function DemoCreateAccount() {
  const [key, setKey] = useState("");
  const [token, setToken] = useState("");
  const [tokenSecret, setTokenSecret] = useState("");
  const [tweet, setTweet] = useState("");
  const [secret, setSecret] = useState("");
  const [date, setDate] = React.useState<Date>()
  const [time, setTime] = useState<any>('10:00');

  console.log(key, "key")

  return (
    <Card>
      <CardHeader className="space-y-1">
        <CardTitle className="text-2xl">Twitter scheduler</CardTitle>
        <CardDescription>
          Enter your credentials below to schedule a tweet
        </CardDescription>
      </CardHeader>
      <CardContent className="grid gap-4">
        <div className="grid gap-2">
          <Label htmlFor="email">API Key</Label>
          <Input id="email" type="email" placeholder="API Key" onChange={(e) => setKey(e.target.value)}/>
        </div>
        <div className="grid gap-2">
          <Label htmlFor="email">API Secret</Label>
          <Input id="email" type="email" placeholder="API Secret" onChange={(e) => setSecret(e.target.value)}/>
        </div>
        <div className="grid gap-2">
          <Label htmlFor="email">Access Token</Label>
          <Input id="email" type="email" placeholder="Access Token" onChange={(e) => setToken(e.target.value)} />
        </div>
        <div className="grid gap-2">
          <Label htmlFor="email">Access Secret</Label>
          <Input id="email" type="email" placeholder="Access Secret" onChange={(e) => setTokenSecret(e.target.value)}/>
        </div>
        <div className="grid gap-2">
          <Label htmlFor="email">Tweet</Label>
          <Input id="email" type="email" placeholder="Tweet your tweet here" onChange={(e) => setTweet(e.target.value)}/>
        </div>
        {DatePickerDemo(setDate, date)}
        <input aria-label="Time" type="time" onChange={(e) => setTime(e.target.value)}/>
      </CardContent>
      <CardFooter>
        <Button className="w-full" onClick={() => sendTweet({
          key,
          token,
          tokenSecret,
          date,
          time,
          tweet,
          secret,
        })}>Schedule tweet</Button>
      </CardFooter>
    </Card>
  )
}

Create a file called src/app/date.tsx

"use client"

import * as React from "react"
import { CalendarIcon } from "@radix-ui/react-icons"
import { format } from "date-fns"

import { cn } from "@/lib/utils"
import { Button } from "@/components/ui/button"
import { Calendar } from "@/components/ui/calendar"
import {
  Popover,
  PopoverContent,
  PopoverTrigger,
} from "@/components/ui/popover"

export function DatePickerDemo(onChange: any, date: any) {

  return (
    <Popover>
      <PopoverTrigger asChild>
        <Button
          variant={"outline"}
          className={cn(
            "w-[240px] justify-start text-left font-normal",
            !date && "text-muted-foreground"
          )}
        >
          <CalendarIcon className="mr-2 h-4 w-4" />
          {date ? format(date, "PPP") : <span>Pick a date</span>}
        </Button>
      </PopoverTrigger>
      <PopoverContent className="w-auto p-0" align="start">
        <Calendar
          mode="single"
          selected={date}
          onSelect={onChange}
          initialFocus
        />
      </PopoverContent>
    </Popover>
  )
}

Deploy

And deploy the front-end on Vercel. Hooray, your app is live 🥳

Configuring permissions on Hasura for Pokemon data

Kaushik Varanasi — Wed, 13 Dec 2023 10:03:32 +0000

Now, go to public and click on track

Now the permissions

Open the Hasura console from your Rocketgraph Hasura tab. We set permissions here so we know which pokemon belongs to whom. Create a role user with the following permissions on the permissions tab.

For select:

Do without any checks

For update:

We also need to preset column user_id so that it automatically sets it up from our rocketgraph-js-sdk:

Build a pokemon collectibles app in React.js in minutes 👨‍💻

Kaushik Varanasi — Wed, 13 Dec 2023 10:03:22 +0000

What we'll be building?

We'll be building a Pokemon collectibles app. You can have users signup and a database of Pokemon to choose from. After a user collects a Pokemon, it's his. It's first come first serve.

For this, we'll need

Authentication for users
DB to store, users and Pokemon
Pokemon data
A mapping or a foreign key to track which Pokemon belongs to who
Realtime notification system for tracking Pokemon collection
Front-end in React.js

Luckily we have Rocketgraph which provides this functionality right out of the box. Let's leverage that to build this much faster.

💡To read this article you need to have a basic understanding of React.js and PostgreSQL

Rocketgraph : A complete backend that beats Firebase and is open-source

A little background. Rocketgraph provides a complete backend. It comes with a Postgres DB, Hasura console to manage your Postgres and add GraphQL layer to your data, Authentication and Serverless functions.

So to sum it up we provide auth for your web apps, GraphQL for realtime things like messages/notifications/comments etc and Serverless functions for anything you want to offload. Deploy your serverless functions by running rgraph deploy <project name> in your repo.

So what the hack is GraphQL?

GraphQL is a language specification by Meta to achieve realtime queries on your data by asking for exactly what you need. This is different from traditional API approach where the query is coded into the backend and front-end has very little control over what/how to ask for data.

Think of it like a JSON query. You ask for what data you want inn a json-like query and it will return exactly those fields.

In this article we'll leverage the power of GraphQL, React Apollo and Hasura to build a real-time system for rating and commenting on movies. We can use this same system to book movie tickets too.

TLDR Version

If you just want to see the code. Here is the code base for this article. Here you can see more examples. This is the open-source software behind Rocketgraph

Keep reading

Before you start

Signup on Rocketgraph and create a project. This will also spin up an AWS RDS Postgres DB that we'll use to store our pokemon data. Once it's complete,

Cool, let's start from the basics

Create a react project and develop the front-end. Forget about the backend, we'll add it later.

mkdir pokemon-app && cd pokemon-app

Next scaffold a basic react application.

npx create-react-app ./

Install react router to be able to navigate between pages. Install react-apollo and graphql for real-time stuff as mentioned above.

yarn add react-router-dom
yarn add @apollo/client graphql

Next add some Rocketgraph libraries

yarn add @rocketgraphql/react-apollo
yarn add @rocketgraphql/rocketgraph-js-sdk

Delete the redundant files such as logo from the project and update the index.js file to as follows:

// src/index.js
import React from "react";
import ReactDOM from "react-dom";
import { Login } from "./components/login";
import { Signup } from "./components/signup";
import { BrowserRouter as Router, Routes, Route } from "react-router-dom";
import App from "./App";
import { RApolloProvider } from "@rocketgraphql/react-apollo";
import { auth } from "./utils/rockets";

ReactDOM.render(
  <React.StrictMode>
      <RApolloProvider auth={auth} gqlEndpoint="https://hasura-vlklxvo.rocketgraph.app/v1/graphql">
        <Router>
          <Routes>
            <Route path="/login" element={<Login />}/>
            <Route path="/signup" element={<Signup />}/>
            <Route path="/" element={<App />} />
          </Routes>
        </Router>
      </RApolloProvider>
  </React.StrictMode>,
  document.getElementById("root")
);

And replace your gqlEndpoint value with that on your Hasura console here:

App.js

import './App.css';

import { gql, useMutation, useSubscription } from "@apollo/client";


const GET_TODOS = gql`
  subscription {
    pokemon(limit: 100, order_by: {id: asc}) {
      id
      identifier
      user_id
    }
  }
`;

const COLLECT = gql`
  mutation MyMutation($name: String!) {
    update_pokemon(where: {identifier: {_eq: $name}}, _set: {identifier: $name}) {
      affected_rows
    }
  }
`

const PokemonComponent = (poke, func) => {
  return (
    <>
      <li key={poke.id}>{poke.identifier}
        {
          poke.user_id ?
          " collected"
          : <button onClick={() => {console.log(poke); func(poke.identifier)}}>collect</button>
        }
      </li>
    </>
  )
}
function App() {
  const { data, loading } = useSubscription(GET_TODOS);
  console.log("data, loading", data, loading);
  const [collectPoke, { poke_mutate, poke_loading }] = useMutation(COLLECT);
  const collectPokemon = (poke_name) => {
    collectPoke({ variables: { name: poke_name} });
}
  return (
    <div className="App">
        {!data || !data.pokemon || !data.pokemon.length ? (
          "no data"
        ) : (
          <ul id="todosList">
            {data.pokemon.map((poke) => {
              return PokemonComponent(poke, collectPokemon);
            })}
          </ul>
        )}
    </div>
  );
}

export default App;

components/signup:

import React, { useState } from "react";
import { useNavigate } from 'react-router-dom';
import { auth } from "../utils/rockets";

export function Signup(props) {
  const [email, setEmail] = useState("");
  const [password, setPassword] = useState("");
  const navigate = useNavigate();

  async function handleSubmit(e) {
    e.preventDefault();

    // login
    try {
        await auth.register({email, password});
        navigate("/")
      } catch (error) {
        alert("error signing up");
        console.error(error);
        return;
    }

  }

  return (
    <div>
      <h1>Signup</h1>
      <form onSubmit={handleSubmit}>
        <input
          id="inputUserEmail"
          type="email"
          placeholder="email"
          value={email}
          onChange={(e) => setEmail(e.target.value)}
        />
        <input
          id="inputUserPassword"
          type="password"
          placeholder="password"
          value={password}
          onChange={(e) => setPassword(e.target.value)}
        />
        <button id="signupButton">Signup</button>
      </form>
    </div>
  );
}

Aaaand, we are done!!

You can host your app on Vercel, use tailwind to make it look more professional.

Load Pokemon data into Postgres

Kaushik Varanasi — Mon, 11 Dec 2023 10:25:24 +0000

Load the Pokemon data

Let's load the Pokemon data from the open-source db. We'll need a small standalone Python project for that.

Create a folder called pokemon-python and a file named load_pokemon.py:

import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('postgresql://postgres:password@xxxxx.xxxxx.us-east-2.rds.amazonaws.com:5432/postgres')
df=pd.read_csv('https://raw.githubusercontent.com/veekun/pokedex/master/pokedex/data/csv/pokemon.csv')
df.to_sql('pokemon', engine)

cd pokemon-python

# create a venv
pip install virtualenv
python3 -m venv env

# To activate
source env/bin/activate

For an easier and robust deployment of Postgres instances, checkout Rocketgraph

pip3 install pandas
pip3 install sqlalchemy

And for good measure

pip3 freeze > requirements.txt

Now simply run

python3 load_pokemon.py

if you get an error such as

ModuleNotFoundError: No module named 'psycopg2'

simply install it:

pip install psycopg2-binary

Once you run that command, psql into your PostgresDB to ensure that Pokemon data is loaded:

psql postgresql://postgres:password@restless-pine.xxxxxxxxxxx.us-east-2.rds.amazonaws.com:5432/postgres

And:

SELECT * FROM pokemon LIMIT 10;

You should see a few Pokemon there. Congrats, you loaded the Pokemon data and setup a front-end succesfully.

I built this postgres logger

Kaushik Varanasi — Sat, 09 Dec 2023 10:54:59 +0000

Postgres Logger

I used pgAudit to setup logging on my AWS RDS Postgres instance.

Disclaimer

I built (Rocketgraph)(https://rocketgraph.io/) which provides a complete backend. That includes Authentication, Hasura console for GraphQL, Postgres DB and serverless functions. So it let's you build web applications in minutes rather than in weeks.

see it in action

Try it out

Rocketgraph hosted

You can signup on Rocketgraph and create a project. The first project is free for 7 days and doesn't require CC.

This spins up a Postgres DB on AWS RDS instance with pgAudit configured.

Then, follow these steps to enable pgAudit:

psql postgresql://postgres:password@project-name.xxxxxxx.us-east-2.rds.amazonaws.com:5432/postgres

Provided in your Rocketgraph dashboard

Create a role:

CREATE ROLE rds_pgaudit;

Then set log level:

ALTER DATABASE postgres set pgaudit.log="ALL";

And enable extension:

CREATE EXTENSION pgAudit;

self-hosted

To use pgAudit on your own Postgres instance, I wrote a detailed article here to setup: https://blog.rocketgraph.io/posts/install-pgaudit

Rocketgraph is open-source. Don't forget to star us here: https://github.com/RocketsGraphQL/rgraph

How Rocketgraph is built using Rocketgraph

Kaushik Varanasi — Wed, 09 Aug 2023 06:51:21 +0000

TL;DR

I built a PAAS Rocketgraph. It lets you build production grade web applications in minutes rather than in weeks. I believe in the idea so much that the entire Rocketgraph infrastructure runs on Rocketgraph.

You can find the latest release here.

We do this by providing you the following functionality:

20 GB Postgres DB hosted on AWS RDS
Pre-populated with some tables for common use cases
Authentication using email/password
Authentication using Social logins, Magic link and OTP as well.
Server less functions.
Hasura console for an amazing GraphQL experience.

So as you can see, it covers most of the use cases. You'll only have to write the front-end for your web applications. You can see me build a todos application here

Rocketgraph: The complete backend

Build production grade web applications in minutes, not weeks. Rocketgraph is an open-source backend as a service that let's you build web applications much faster. It's like Firebase but much better.

When you create a project on rocketgraph, you get PostgresDB to hold your data, Hasura console that "GraphQLises" your data, authentication, serverless functions and much more.

You can check out the detailed blog here, documentation and the example setups

I have been building it alone for the last 2 years and only spoke about it here, here and here. It was extremely difficult and lonely here.

How do you do it?

We do this by leveraging the power of GraphQL. Basically, we provide an authentication method to your Javascript using a library called rocket-js-sdk like so:

rocketgraph.signUp({ email, password, provider: "local"})

Then every GraphQL request you make will be embedded with a JWT once you wrap our provider at the root level of your tree like so:

<RApolloProvider auth={auth} gqlEndpoint="https://hasura-vf0gs3q.rocketgraph.app/v1/graphql">
    <App />
</RApolloProvider>

Once you have configured that, subscribe to your Postgres tables like:

const GET_TODOS = gql`
  subscription {
    instances {
      id
      created_at
      name
    }
  }
`;

export default function ProjectsList() {
  const { data, loading } = useSubscription(GET_TODOS);
  console.log("data, loading: ", data, loading);
}

You'd only get the instances belonging to that particular user. Of-course you'd need to enable public/user access which is really simple. Please follow the steps here.

Hows Rocketgraph built

So to hold the users data, I use AWS RDS. The I run the hasura-batteries software that is connected to this RDS instance. This runs as a docker container alongside the hasura GraphQL engine as described in this file here.

Now in my frontend, I wrapped it with the RApolloProvider as described above. Now when the user logs in, hasura-batteries already takes care of setting and refreshing the JWT tokens. Now everytime I need to fetch the user's details, for example all the projects of the user I just do:

const GET_TODOS = gql`
  subscription {
    instances {
      id
      created_at
      name
    }
  }
`;

export default function ProjectsList() {
  const { data, loading } = useSubscription(GET_TODOS);
  console.log("data, loading: ", data, loading);
}

That's literally it. This brought down the code written by 25% since most of it was redundant stuff like authentication, setting the JWTs, fetch methods. Now all of that suff is gone out the window. Simple, clean, elegant GraphQL requests like the good ol' JSON.

Be sure to give us a star. Check out Rocketgraph. Please let me know if you have any questions. Oh, and it's free for 1 week and no CC required.

What is Rocketgraph

Kaushik Varanasi — Thu, 03 Aug 2023 10:23:57 +0000

Check out Rocketgraph

And the latest v0.2.0 stable release: https://github.com/RocketsGraphQL/rgraph/releases/tag/v0.2-stable

Finally the messages component

Kaushik Varanasi — Thu, 03 Aug 2023 10:21:19 +0000

Now modify the code in Message.js component to the following:

import React, {useEffect, useState} from "react";
import { useSearchParams } from 'react-router-dom';

import { Input } from '@chakra-ui/react'
import { Button, ButtonGroup } from '@chakra-ui/react'
import { Card, CardHeader, CardBody, CardFooter, Heading, Text, Flex, Avatar, Box } from '@chakra-ui/react'
import { gql, useQuery, useMutation, useSubscription } from "@apollo/client";


const ADD_MESSAGE = gql`
    mutation MyMutation($message: String!, $from_email: String!, $to_email: String!) {
        insert_messages_one(object: {from_email: $from_email, message: $message, to_email: $to_email}) {
            from_email
            message
            to_id
            to_email
        }
    }
`;

const GET_MESSAGES = gql`
    subscription MySubscription {
        messages {
            id
            message
            from_email
            to_email
        }
    }
`;

// CREATE FUNCTION check_author_active()
//     RETURNS trigger AS $BODY$
//     BEGIN
//     SELECT id INTO NEW.to_id FROM users WHERE email = NEW.to_email;
//     RETURN NEW;
//     END;
//     $BODY$ LANGUAGE plpgsql;

// CREATE TRIGGER insert_article BEFORE INSERT OR UPDATE ON messages FOR EACH ROW EXECUTE PROCEDURE check_author_active();


export default function DrawerExample() {
    const [searchParams] = useSearchParams();
    const [messages, setMessages] = useState([]);
    console.log(searchParams.get('email')); // ▶ URLSearchParams {}
    const currentUser = "kaushik@moneysave.io";
    const { data, loading } = useSubscription(GET_MESSAGES);
    console.log("messages: ", data, loading);
    const [addMessage, { messages_mutate, message_loading }] = useMutation(ADD_MESSAGE);
    const senderAddress = searchParams.get('email');
    const [message, setMessage] = useState("");
    const sendMessage = (text, from_email, to_email) => {
        addMessage({ variables: { message: text, from_email: from_email, to_email: to_email } });
    }
    useEffect(() => {
        if (data && data.messages.length) {
            setMessages(data?.messages);
        }
    }, [data])
    return (
      <>
        <div className="message-body">
            <div className="messages">
                {
                    messages.map((message) => {
                        return (
                            <div key={message.id} className={message.to_email == currentUser ? "message-received" : "message-sent"} m={4}>
                                <div className="message-text">
                                    <Card>
                                        <CardHeader>
                                            <Flex spacing='4'>
                                            <Flex flex='1' gap='4' alignItems='center' flexWrap='wrap'>
                                                <Avatar name='Segun Adebayo' src='https://bit.ly/sage-adebayo' />

                                                <Box>
                                                <Heading size='sm'>{message.from_email}</Heading>
                                                <Text>Creator, Chakra UI</Text>
                                                </Box>
                                            </Flex>
                                            </Flex>
                                        </CardHeader>
                                        <CardBody>
                                            <Text>{message.message}</Text>
                                        </CardBody>
                                    </Card>
                                </div>
                            </div>
                        )
                    })
                }


            </div>
            <div className="main-bottom">
                <Input m={2} placeholder='Basic usage' onChange={(e) => {setMessage(e.target.value)}}/>
                <Button m={2} colorScheme='blue' onClick={() => {sendMessage(message, "kaushik@rocketgraph.io", "kaushik");}}>Button</Button>
            </div>
        </div>
      </>
    )
}

You should be able to send messages and receive on this component.

Be sure to check out Rocketgraph

Create event triggers for PostgresDB

Kaushik Varanasi — Thu, 03 Aug 2023 10:21:04 +0000

Postgres Triggers

SO what are Postgres triggers and why are they so great. To be honest I also didn't know about them until I wrote this application. And realising just how powerful they are made me rethink my view on web-development.

Basically Postgres triggers are functions that are triggered when an event(CREATE, READ, UPDATE, DELETE) happens. You can do powerful joins and interpolations using these triggers. They are completely on the backend and you can use them to set restrictions on the data.

So let's say you want to embed to_id field automatically into every message. to_id is the id of the user the message is addressed to. Since this is not available on the frontend, we will do it from the backend.

Go to the SQL tab in your Hasura console:

And add the following code:

CREATE FUNCTION create_to_id()
    RETURNS trigger AS $BODY$
    BEGIN
    SELECT id INTO NEW.to_id FROM users WHERE email = NEW.to_email;
    RETURN NEW;
    END;
    $BODY$ LANGUAGE plpgsql;

CREATE TRIGGER insert_id BEFORE INSERT OR UPDATE ON messages FOR EACH ROW EXECUTE PROCEDURE create_to_id();

We are telling SQL to get the id from email and insert that into the new record before inserting the record. This way we get the to id information in the table for using later.

Connecting the Frontend and backend for the messaging application

Kaushik Varanasi — Thu, 03 Aug 2023 10:20:45 +0000

Enter GQL

So to fetch the data from our backend we'll be using GraphQL. So let's install the necessary libraries.

yarn add @apollo/client
yarn add @rocketgraphql/rocketgraph-js-sdk
yarn add @rocketgraphql/react-apollo

Now modify the Signup.js page to be able to register new users:

import React, { useState } from "react";
import { useNavigate } from 'react-router-dom';
import { auth } from "../utils/config";

import {
    Flex,
    Box,
    FormControl,
    FormLabel,
    Input,
    Checkbox,
    Stack,
    Link,
    Button,
    Heading,
    Text,
    useColorModeValue,
  } from '@chakra-ui/react';

  export default function SimpleCard() {
    const [ email, setEmail ] = useState()
    const [ password, setPassword ] = useState()
    const navigate = useNavigate();

    const signup = async () => {
      // login
      try {
        await auth.register({email, password});
        navigate("/")
      } catch (error) {
        alert("error signing up");
        console.error(error);
        return;
      }
    }
    return (
      <Flex
        minH={'100vh'}
        align={'center'}
        justify={'center'}
        bg={useColorModeValue('gray.50', 'gray.800')}>
        <Stack spacing={8} mx={'auto'} maxW={'lg'} py={12} px={6}>
          <Stack align={'center'}>
            <Heading fontSize={'4xl'}>Create your account</Heading>
            <Text fontSize={'lg'} color={'gray.600'}>
              to enjoy all of our cool <Link color={'blue.400'}>features</Link> ✌️
            </Text>
          </Stack>
          <Box
            rounded={'lg'}
            bg={useColorModeValue('white', 'gray.700')}
            boxShadow={'lg'}
            p={8}>
            <Stack spacing={4}>
              <FormControl id="email">
                <FormLabel>Email address</FormLabel>
                <Input type="email" onChange={(e) => setEmail(e.target.value)}/>
              </FormControl>
              <FormControl id="password">
                <FormLabel>Password</FormLabel>
                <Input type="password" onChange={(e) => setPassword(e.target.value)}/>
              </FormControl>
              <Stack spacing={10}>
                <Stack
                  direction={{ base: 'column', sm: 'row' }}
                  align={'start'}
                  justify={'space-between'}>
                  <Checkbox>Remember me</Checkbox>
                  <Link color={'blue.400'}>Forgot password?</Link>
                </Stack>
                <Button
                  onClick={() => signup()}
                  bg={'blue.400'}
                  color={'white'}
                  _hover={{
                    bg: 'blue.500',
                  }}>
                  Sign up
                </Button>
              </Stack>
            </Stack>
          </Box>
        </Stack>
      </Flex>
    );
  }

Notice that we are using the Rocketgraph's auth.register() method. This will take care of handling the signup and setting the user cookies for us.

And similarly for the login:

import React, { useState } from "react";
import { useNavigate } from 'react-router-dom';
import { auth } from "../utils/config";

import {
    Flex,
    Box,
    FormControl,
    FormLabel,
    Input,
    Checkbox,
    Stack,
    Link,
    Button,
    Heading,
    Text,
    useColorModeValue,
  } from '@chakra-ui/react';

  export default function SimpleCard() {
    const [ email, setEmail ] = useState()
    const [ password, setPassword ] = useState()
    const navigate = useNavigate();

    const signup = async () => {
      // login
      try {
        await auth.signIn({email, password, provider: "local"});
        navigate("/")
      } catch (error) {
        alert("error signing up");
        console.error(error);
        return;
      }
    }
    return (
      <Flex
        minH={'100vh'}
        align={'center'}
        justify={'center'}
        bg={useColorModeValue('gray.50', 'gray.800')}>
        <Stack spacing={8} mx={'auto'} maxW={'lg'} py={12} px={6}>
          <Stack align={'center'}>
            <Heading fontSize={'4xl'}>Login to your account</Heading>
            <Text fontSize={'lg'} color={'gray.600'}>
              to enjoy all of our cool <Link color={'blue.400'}>features</Link> ✌️
            </Text>
          </Stack>
          <Box
            rounded={'lg'}
            bg={useColorModeValue('white', 'gray.700')}
            boxShadow={'lg'}
            p={8}>
            <Stack spacing={4}>
              <FormControl id="email">
                <FormLabel>Email address</FormLabel>
                <Input type="email" onChange={(e) => setEmail(e.target.value)}/>
              </FormControl>
              <FormControl id="password">
                <FormLabel>Password</FormLabel>
                <Input type="password" onChange={(e) => setPassword(e.target.value)}/>
              </FormControl>
              <Stack spacing={10}>
                <Stack
                  direction={{ base: 'column', sm: 'row' }}
                  align={'start'}
                  justify={'space-between'}>
                  <Checkbox>Remember me</Checkbox>
                  <Link color={'blue.400'}>Forgot password?</Link>
                </Stack>
                <Button
                  onClick={() => signup()}
                  bg={'blue.400'}
                  color={'white'}
                  _hover={{
                    bg: 'blue.500',
                  }}>
                  Sign up
                </Button>
              </Stack>
            </Stack>
          </Box>
        </Stack>
      </Flex>
    );
  }

Now test that you are able to retrieve the users list via the following code in App.js:

import logo from './logo.svg';
import Profiles from "./components/Profiles"
import './App.css';

function App() {
  return (
    <div className="App">
      <Profiles/>
    </div>
  );
}

export default App;

And create a component named Profiles.js in /components:

import {
    Heading,
    Avatar,
    Box,
    Center,
    Image,
    Flex,
    Text,
    Stack,
    Button,
    useColorModeValue,
    useRangeSlider,
  } from '@chakra-ui/react';
  import { gql, useQuery, useMutation, useSubscription } from "@apollo/client";

  const GET_TODOS = gql`
    subscription {
      users {
        id
        email
      }
    }
  `;

  export default function SocialProfileWithImage() {
    const { data, loading } = useSubscription(GET_TODOS);
    const bgColor800 = useColorModeValue('white', 'gray.800');
    const bgColor900 = useColorModeValue('#151f21', 'gray.900');
    const users = data?.users;
    return (
      <Center py={6}>
        {
          users && users.map((user) => {
            return (
              <Box
              maxW={'270px'}
              w={'full'}
              bg={bgColor800}
              boxShadow={'2xl'}
              rounded={'md'}
              overflow={'hidden'}>
              <Image
                h={'120px'}
                w={'full'}
                src={
                  'https://images.unsplash.com/photo-1612865547334-09cb8cb455da?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=634&q=80'
                }
                objectFit={'cover'}
              />
              <Flex justify={'center'} mt={-12}>
                <Avatar
                  size={'xl'}
                  src={
                    'https://images.unsplash.com/photo-1500648767791-00dcc994a43e?ixlib=rb-1.2.1&q=80&fm=jpg&crop=faces&fit=crop&h=200&w=200&ixid=eyJhcHBfaWQiOjE3Nzg0fQ'
                  }
                  alt={'Author'}
                  css={{
                    border: '2px solid white',
                  }}
                />
              </Flex>

              <Box p={6}>
                <Stack spacing={0} align={'center'} mb={5}>
                  <Heading fontSize={'2xl'} fontWeight={500} fontFamily={'body'}>
                    {user.email}
                  </Heading>
                  <Text color={'gray.500'}>Frontend Developer</Text>
                </Stack>

                <Button
                  w={'full'}
                  mt={8}
                  bg={bgColor900}
                  color={'white'}
                  rounded={'md'}
                  onClick={() => {window.location = `/messages?email=${user.email}`} }
                  _hover={{
                    transform: 'translateY(-2px)',
                    boxShadow: 'lg',
                  }}>
                  Message
                </Button>
              </Box>
            </Box>
            )
          })
        }

      </Center>
    );
  }

How to build the backend for the messaging app in minutes 🤯

Kaushik Varanasi — Thu, 03 Aug 2023 10:20:08 +0000

Backend on Rocketgraph

Head over to the Hasura console. You can find it here on the 2nd tab in Rocketgraph:

use the password to login to the console. And head over to the data column and create the following columns:

Permissions

We have to add user permissions so that the users can view only their messages.

Head over to the permissions tab in messages and create a role called user. A user is a person's role who is logged in.

Create insert permissions as follows:

Notice the column presets. WE are letting Hasura to inject custom variables automatically into our columns, so we can use them later.

Next allow select over the messages as follows:

We are telling Hasura to only allow messages to the user if he is the sender or receiver of the message.

And save: