DEV Community: Hassan El Mghari

How I built NotesGPT – a full-stack AI voice note app

Hassan El Mghari — Wed, 21 Feb 2024 21:20:22 +0000

Last week, I launched notesGPT, a free and open source voice note app that has 35,000 visitors, 7,000 users, and over 1,000 GitHub stars so far in the last week. It allows you to record a voice note, transcribes it uses Whisper, and uses Mixtral via Together to extract action items and display them in an action items view. It’s also fully open source and comes equipped with authentication, storage, vector search, action items, and is fully responsive on mobile for ease of use.

I’m going to to walk you through exactly how I built it.

Architecture and tech stack

This is a quick diagram for the architecture. We’ll be discussing each piece in more depth and also showing code examples as we go.

Here’s the overall tech stack I used:

Convex for the database and cloud functions
Next.js App Router for the framework
Replicate for Whisper transcriptions
Mixtral with JSON mode for the LLM
Together.ai for inference and embeddings
Convex File Storage for storing voice notes
Convex Vector search for vector search
Clerk for user authentication
Tailwind CSS for styling

Landing Page

The first piece of the app is the landing page you see when you navigate to notesGPT.

The first things users see is this landing page which along with the rest of the app, was built with Next.js and with Tailwind CSS for styling. I enjoy using Next.js since it makes it it easy to spin up web apps and just write React code. Tailwind CSS is great too since it allows you to iterate quickly on your web pages while staying in the same file as your JSX.

Authentication with Clerk and Convex

When the user clicks either of the buttons on the homepage, they get directed to the sign in screen. This is powered by Clerk, an easy authentication solution that integrates well with Convex, which is what we’ll be using for our entire backend including cloud functions, database, storage, and vector search.

Clerk and Convex are both easy to setup. You can simply create an account on both services, install their npm libraries, run npx convex dev to setup your convex folder, and create a ConvexProvider.ts file as seen below to wrap your app with.

'use client';

import { ReactNode } from 'react';
import { ConvexReactClient } from 'convex/react';
import { ConvexProviderWithClerk } from 'convex/react-clerk';
import { ClerkProvider, useAuth } from '@clerk/nextjs';

const convex = new ConvexReactClient(process.env.NEXT_PUBLIC_CONVEX_URL!);

export default function ConvexClientProvider({
  children,
}: {
  children: ReactNode;
}) {
  return (
    <ClerkProvider
      publishableKey={process.env.NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY!}
    >
      <ConvexProviderWithClerk client={convex} useAuth={useAuth}>
        {children}
      </ConvexProviderWithClerk>
    </ClerkProvider>
  );
}

Check out the Convex Quickstart and the Convex Clerk auth section for more details.

Setting up our schema

You can use Convex with or without a schema. In my case, I knew the structure of my data and wanted to define it so i did so below. This also gives you a really nice type-safe API to work with when interacting with your database. We’re defining two tables – a notes table to store all voice note information and actionItems table for extracted action items. We’ll also define indexes to be able to quickly query the data by userId and noteId.

import { defineSchema, defineTable } from 'convex/server';
import { v } from 'convex/values';

export default defineSchema({
  notes: defineTable({
    userId: v.string(),
    audioFileId: v.string(),
    audioFileUrl: v.string(),
    title: v.optional(v.string()),
    transcription: v.optional(v.string()),
    summary: v.optional(v.string()),
    embedding: v.optional(v.array(v.float64())),
    generatingTranscript: v.boolean(),
    generatingTitle: v.boolean(),
    generatingActionItems: v.boolean(),
  })
    .index('by_userId', ['userId'])
    .vectorIndex('by_embedding', {
      vectorField: 'embedding',
      dimensions: 768,
      filterFields: ['userId'],
    }),
  actionItems: defineTable({
    noteId: v.id('notes'),
    userId: v.string(),
    task: v.string(),
  })
    .index('by_noteId', ['noteId'])
    .index('by_userId', ['userId']),
});

Dashboard

Now that we have our backend and authentication setup along with our schema, we can take a look at fetching data. After signing into the app, users can view their dashboard which lists all of the voice notes they’ve recorded.

To do this, we first define a query in the convex folder that uses auth to take in a userId, verify it’s valid, and returns all the notes that match a user’s userId.

export const getNotes = queryWithUser({
  args: {},
  handler: async (ctx, args) => {
    const userId = ctx.userId;
    if (userId === undefined) {
      return null;
    }
    const notes = await ctx.db
      .query('notes')
      .withIndex('by_userId', (q) => q.eq('userId', userId))
      .collect();

    const results = Promise.all(
      notes.map(async (note) => {
        const count = (
          await ctx.db
            .query('actionItems')
            .withIndex('by_noteId', (q) => q.eq('noteId', note._id))
            .collect()
        ).length;
        return {
          count,
          ...note,
        };
      }),
    );

    return results;
  },
});

After this, we can call this getNotes query with a user’s authentication token via a function that convex provides to display all the user’s notes in the dashboard. We’re using server side rendering to fetch this data on the server then passing it into the <DashboardHomePage /> client component. This also ensures that the data stays up to date on the client as well.

import { api } from '@/convex/_generated/api';
import { preloadQuery } from 'convex/nextjs';
import DashboardHomePage from './dashboard';
import { getAuthToken } from '../auth';

const ServerDashboardHomePage = async () => {
  const token = await getAuthToken();
  const preloadedNotes = await preloadQuery(api.notes.getNotes, {}, { token });

  return <DashboardHomePage preloadedNotes={preloadedNotes} />;
};

export default ServerDashboardHomePage;

Recording a voice note

Initially, users won’t have any voice notes on their dashboard so they can click the “record a new voice note” button to record one. They’ll see the following screen that will allow them to record.

This will record a voice note using native browser APIs, save the file in Convex file storage, then send it to Whisper through Replicate to be transcribed. The first thing we do is define a createNote mutation in our convex folder that will take in this recording, save some information in the Convex database, then call the whisper action.

export const createNote = mutationWithUser({
  args: {
    storageId: v.id('_storage'),
  },
  handler: async (ctx, { storageId }) => {
    const userId = ctx.userId;
    let fileUrl = (await ctx.storage.getUrl(storageId)) as string;

    const noteId = await ctx.db.insert('notes', {
      userId,
      audioFileId: storageId,
      audioFileUrl: fileUrl,
      generatingTranscript: true,
      generatingTitle: true,
      generatingActionItems: true,
    });

    await ctx.scheduler.runAfter(0, internal.whisper.chat, {
      fileUrl,
      id: noteId,
    });

    return noteId;
  },
});

The whisper action is seen below. It uses Replicate as the hosting provider for Whisper.

export const chat = internalAction({
  args: {
    fileUrl: v.string(),
    id: v.id('notes'),
  },
  handler: async (ctx, args) => {
    const replicateOutput = (await replicate.run(
      'openai/whisper:4d50797290df275329f202e48c76360b3f22b08d28c196cbc54600319435f8d2',
      {
        input: {
          audio: args.fileUrl,
          model: 'large-v3',
          translate: false,
          temperature: 0,
          transcription: 'plain text',
          suppress_tokens: '-1',
          logprob_threshold: -1,
          no_speech_threshold: 0.6,
          condition_on_previous_text: true,
          compression_ratio_threshold: 2.4,
          temperature_increment_on_fallback: 0.2,
        },
      },
    )) as whisperOutput;

    const transcript = replicateOutput.transcription || 'error';

    await ctx.runMutation(internal.whisper.saveTranscript, {
      id: args.id,
      transcript,
    });
  },
});

Also, all these files can be seen in the Convex dashboard under “Files”.

Generating action items

After the user finishes recording their voice note and it gets transcribed via whisper, the output is then passed into Together AI. We show this loading screen in the meantime.

We first define a schema that we want our output to be in. We then pass this schema into our Mixtral model hosted on Together.ai with a prompt to identify a summary of the voice note, a transcript, and generate action items based on the transcript. We then save all this information to the Convex database. To do this, we create a Convex action in the convex folder.

// convex/together.ts

const NoteSchema = z.object({
  title: z
    .string()
    .describe('Short descriptive title of what the voice message is about'),
  summary: z
    .string()
    .describe(
      'A short summary in the first person point of view of the person recording the voice message',
    )
    .max(500),
  actionItems: z
    .array(z.string())
    .describe(
      'A list of action items from the voice note, short and to the point. Make sure all action item lists are fully resolved if they are nested',
    ),
});

export const chat = internalAction({
  args: {
    id: v.id('notes'),
    transcript: v.string(),
  },
  handler: async (ctx, args) => {
    const { transcript } = args;
      const extract = await client.chat.completions.create({
        messages: [
          {
            role: 'system',
            content:
              'The following is a transcript of a voice message. Extract a title, summary, and action items from it and answer in JSON in this format: {title: string, summary: string, actionItems: [string, string, ...]}',
          },
          { role: 'user', content: transcript },
        ],
        model: 'mistralai/Mixtral-8x7B-Instruct-v0.1',
        response_model: { schema: NoteSchema, name: 'SummarizeNotes' },
        max_tokens: 1000,
        temperature: 0.6,
        max_retries: 3,
      });
      const { title, summary, actionItems } = extract;

      await ctx.runMutation(internal.together.saveSummary, {
        id: args.id,
        summary,
        actionItems,
        title,
      });
});

When Together.ai responds, we get this final screen which lets users toggle between their transcript and a summary on the left, and see and check off the action items on the right.

Vector Search

The final piece of the app is vector search. We’re using Together.ai embeddings to embed the transcripts and make it possible for folks to search in the dashboard based on the semantic meaning of the transcripts.

We do this by creating a similarNotes action in the convex folder that takes in a user’s search query, generates an embedding for it, and finds the most similar notes to display on the page.

export const similarNotes = actionWithUser({
  args: {
    searchQuery: v.string(),
  },
  handler: async (ctx, args): Promise<SearchResult[]> => {
        // 1. Create the embedding
    const getEmbedding = await togetherai.embeddings.create({
      input: [args.searchQuery.replace('/n', ' ')],
      model: 'togethercomputer/m2-bert-80M-32k-retrieval',
    });
    const embedding = getEmbedding.data[0].embedding;

    // 2. Then search for similar notes
    const results = await ctx.vectorSearch('notes', 'by_embedding', {
      vector: embedding,
      limit: 16,
      filter: (q) => q.eq('userId', ctx.userId), // Only search my notes.
    });

    return results.map((r) => ({
      id: r._id,
      score: r._score,
    }));
  },
});

Conclusion

Just like that, we’ve built a production-ready full-stack AI app ready with authentication, a database, storage, and APIs. Feel free to check out notesGPT to generate action items from your notes or the GitHub repo for reference. And if you had any questions, shoot me a DM and I’d be more than happy to answer it!

What is Developer Advocacy and how to get a job in it?

Hassan El Mghari — Tue, 18 Jan 2022 05:17:14 +0000

You might have seen Developer Relations (also referred to as DevRel or Developer Advocacy) around Twitter lately and been confused on what it was. Or maybe you have a fuzzy understanding of DevRel and want to learn more about it to try to get a job in it.

Either way, this blog post is for you. We're going to be talking about what developer relations is, exploring its 3 main areas, then talking about how to land a job in this field.

The 3 Pillars of DevRel

Before we discuss the pillars of DevRel, let's give a quick overview of what it is. DevRel is a role that exists at developer tool companies (companies whose target market is developers). Developer Advocates help raise awareness for a product and collect feedback from the dev community to improve it.

DevRel has 3 main categories: Community, Content, and Product. Some Devrels do one of these while others are focused on a couple; it ultimately depends on people's skillsets and what the company needs. Both as a startup hiring for devrel and as someone who wants to get into it, it's important to understand these areas to know which you want to focus on. Let's explore each one in more depth.

Community

Community-focused DevRel usually centers around growing and managing a community of developers. This is typically a Slack or Discord community that's focused on a product or technology. Managing a community consists of greeting new members, answering questions, designing the community in an inclusive way, and making sure your members are getting value from the community. This can also include livestreams, events, and other activities that help the community grow.

Being a technical community builder is a very hot skill to have nowadays due to the shift developer tool companies are making in the way they obtain their customers.

Content

Content-focused DevRel is about spreading awareness about a product through written and spoken content. This can involve writing blog posts, making videos, teaching workshops, recording podcasts, and more.

Product

Product-focused DevRel is about owning the developer exeprience of the product. In other words, it's about improving how it feels for a developer to use a specific product. This can be done by talking to users to understand how they're using it, then making improvements to the documentation and creating guides based on their answers.

This can also extend to the product ecosystem so working on code examples, templates, and integrations is also common.

Getting a DevRel job

This is a relatively new role. To stand out though, you should try to have 2 out of 3 of these.

You can code. For this, you can show off some of your side projects.
You know how to build communities. You can show off helping build a community or organization if you have any experience in this area.
You're a good content creator. You can show off your talks, YouTube videos, tweets, and blog posts for this one.

While there aren't pre-requisites for the job, it's very helpful to have prior work in these areas to show off and get your career in developer advocacy started. Ultimately, it's for engineers who enjoy working with developers, creating content, and managing communities.

Conclusion

As long as you can demonstrate the skills above, you'll likely have no issues finding a role. I also suggest having a website or writing up a document with links to your work to include in your applications. I have a list of some startups that are hiring here.

I hope this clarified some things for you regarding what DevRel is. If you have any questions, feel free to shoot me a DM on Twitter!

Leading developer relations at a Silicon Valley Startup

Hassan El Mghari — Mon, 17 Jan 2022 01:19:51 +0000

Moving to San Fransisco and working for a startup in Silicon Valley has been a dream of mine for a while. After all, it is the startup tech hub of the world. As a fresh college graduate last year, I got a chance to make it a reality.

I want to tell the story of leading developer relations at an early stage devtool startup called Fig. It all started with a Twitter DM, that lead to an interview, that a few weeks later, led to me moving across the country to San Fransisco.

The work

To set the scene, we were a small and scrappy team of 6 having recently raised a seed round of a few million dollars. And we just needed to execute.

The fast pace was no joke. In the first week, we spent about 12h a day at the office for onboarding, that then decreased to ~10.5h a day. The founders spent even more time working. We prioritized tasks on a week to week basis which led to me being able to work on a wide variety of things. In terms of the 3 pillars of developer advocacy, I was lucky to do work in every pillar.

What I did

What I did while I was at Fig can be broken down into five primary areas:

1. Discord Community

I helped grow the Fig Discord community by DMing new users and answering people's questions. At one point, I sent a personalized DM to every single user that joined until it become unscalable.

I also livestreamed myself contributing to Fig's open source repo weekly on the Discord to encourage our users to contribute and did a livestream with Nader + Fig's CEO. I ended up helping them scale their Discord community from 1k members to over 2k over the two months I was there.

2. Twitter Account

I came up with our Twitter strategy and executed, posting a variety of tweets over an average of 5 times a week. Fig was a very visual product so I also recorded a lot of short videos and GIFs to show it off.

We also ran promotions and giveaways on Twitter. Fig was invite-only at the time so we partnered with popular developer influencers to give away hundreds of Fig invites and increase our userbase. I ended up growing their Twitter from 2k to over 4.5k followers.

3. Open Source Contributions

Another thing I did was manage our open source repo. Along with a part-time team member, I reviewed dozens of PRs with some back and forth with our contributors to make sure we were pushing quality code and following best practices.

I also submitted several PRs myself – a total of 63 commits and 19k lines of code. Some of this code was generated using CLI parsers and scripts that I wrote. You can look through my commits here.

4. Writing code

Yet another thing I helped with was the frontend for our Fig settings app. I redesigned and reimplemented it, fixed some bugs, and added features to make it easier for users to customize their settings.

I also helped with creating parsers for popular CLI tools like curl and GCC. I did this to programatically grab all the different options and arguments of a CLI tool and generate a completion spec so Fig could autocomplete for them.

5. Developer Experience

The final thing I did was help improve the overall developer experience of the product. I collected feedback from users each week through Discord chats, Twitter DMs, zoom calls, and my livestreams and relayed it over to our engineering team.

I also revamped our entire documentation to improve the UI, base it on the Divio system and write a few extra guides. Naturally, our docs used Next.js and were hosted on Vercel :)

Lessons Learned

Overall, my work contributed to us getting thousands of more members in our Discord and Twitter, which lead to more OSS contributors, more Github stars, and significantly more users. I'm proud of what I did at Fig.

It was a hectic and rewarding couple months and even though it didn't work out in the end, I'm extremely thankful to Fig to giving me my start in the world of SF startups and for everything I learned.

They taught me how to ruthlessly prioritize and focus on the biggest pain points first. They taught me that a plan means nothing without solid execution. And they taught me that unexpected events occur and you need to be ready to deal with them.

How to Develop Your Talent Stack

Hassan El Mghari — Fri, 11 Dec 2020 02:16:13 +0000

From a young age, we're taught that we need to be the best at one thing to be successful. But, this idea is becoming increasingly untrue in this age of infinite leverage.

This path of being the best (top 1%) at a particular thing is an uphill battle. You’re directly competing with everyone else who wants to be the best at your skill. It requires lots of hard work, talent, and luck, and even then, it may not work out.

Luckily, there is another way: Become very good at a few skills you can connect well. In other words, learn a set of complimentary skills that make you uniquely qualified. Scott Adams called this a talent stack.

Before we get into how to build a talent stack, let us examine how Scott, a man with no extraordinary skills, created a \$75 million net worth.

Scott Adam's Story

Adam Scott is the writer and illustrator of Dilbert, a popular comic strip. He publicly admits that he's not the best artist, writer, or comedian in the world. He says he was just a little better than average at each of these skills.

What he did have is a strong work ethic, a sense of humor, some writing skills and some drawing skills.

Individually, these skills are common. But together, they become dangerous. They become his talent stack.

“The idea of a talent stack is that you can combine ordinary skills until you have enough of the right kind to be extraordinary. You don’t have to be the best in the world at any one thing. All you need to succeed is to be good at a number of skills that fit well together.” - Scott Adams

How can you build your talent stack?

Explore your talents, curiosity, and passions. It may take some and trial & error to find the intersection of what you love, what you're good at, and what the world needs.

Take some time to think about your end goal. Do you want to climb the corporate ladder in a particular field or industry? Do you want to freelance? Start a company? Your talent stack should be complimentary to what you want to do.

Embrace this process and take your time. After some exploration, you want to end up with 2 skills to focus on.

Some of my personal favorites:

Code + ML/AI
Sales + Design
Storytelling + Data Analytics
Code + Design
Design + Marketing

Add a third and you'll be unstoppable.

The trick is to become very good at a skill before you add another one. Not the best, but comfortably above average (top 25%). You don't want to be a jack of all trades and master of none. You want to master a few, taking your time as you focus on sharpening your skills.

My Talent Stack

Here's the talent stack that I'm developing right now:

Coding (full-stack web development)
Writing (communicating my thoughts clearly)
Business Strategy (analyzing businesses and identifying trends)

Coding is very important to me because I like to build products and be able to code up quick prototypes. I'm getting better at this through building side projects.

Writing is also crucial because I want to be able to scale myself. It allows me to share my thoughts, get feedback from others, and build an audience. I’m developing this skill through writing on twitter and this blog that you’re reading right now.

Lastly, business strategy is an important skill to have as a startup founder. You need to be able to predict where the market is going and pivot accordingly. I’m improving at this through reading books, case studies, and keeping up with funding deals.

I'm passionate about coding and startups, so that's what I chose to focus my talent stack on. I recently sold my latest startup and plan to start another eventually. But in the near future, my goal is to join a high-growth startup as a software engineer. I believe it's the best way to rapidly develop my talent stack.

Conclusion

Building your talent stack will give you a unique edge for particular roles or tasks. In doing so, you won't be competing with everyone else anymore. You won't be playing a zero sum game.

Instead, you'll be focusing on yourself, playing positive sum games, and building your own path.

Thanks for reading! Feel free to DM me on Twitter and I'd be more than happy to chat about your talent stack.

Build or sell: The two critical skills in tech

Hassan El Mghari — Mon, 07 Dec 2020 05:59:40 +0000

If you want to be invaluable in the tech industry, you need to learn to build or sell. These two skills are arguably the most important in the tech space since every crucial role falls under one of them.

Let's dive into what they mean, why they're important, and what they can look like, specifically in the tech industry.

Building

Building is the process of adding business value in some way. This can manifest itself in different forms, including:

Building new features for an application
Writing code to build an MVP
Designing the next iteration of a mobile app on Figma
Building a website with a no-code tool like Bubble or Wordpress
Managing a service

Selling

Selling on the other hand is the process of convincing someone to do something. Selling will look different depending on the business model and stage of the company. It could be:

Selling to customers in the form of copywriting, outbound sales, or online marketing
Designing landing pages or sales funnels
Selling the company's vision to investors when raising money through pitching
Selling upper management on what features or products to prioritize as a product manager
Selling the company's culture to prospective employees to attract talent through recruiting

Conclusion

At the end of the day, every critical role in a tech company will fall under building or selling. Learn how to do one very well, and you'll become an asset to any business you join. If you're a startup founder, you should be able to do both.

"Learn to sell, learn to build. If you can do both, you will be unstoppable." - Naval

Just knowing what category your role falls under and how it fits into the bigger picture of your organization is invaluable. You'll have more insight into choosing the right problems to work on and how you can add value to your company.

Actionable advice: Learn to build (coding, designing, managing) or sell (copywriting, marketing, sales, PM). Become really good at one of them, then learn one skill on the opposite category to build your talent stack and become unstoppable.

How I became a certified Scrum Master in 3 days

Hassan El Mghari — Fri, 11 Sep 2020 20:12:52 +0000

I wrote this guide to share exactly how I studied and passed the Professional Scrum Master I certification, and how you can too. Let's get started!

What is Scrum?

Scrum is a framework that helps teams work together to solve a complex problem. It's primarily used in software development and encourages teams to self-organize, keep progress transparent, and continuously plan, reflect, and adapt.

Whether you're just learning what scrum and agile mean for the first time, or you already work in a team that uses these principles, it may be worth it to get scrum master certified to both learn these principles and prove your knowledge to employers. My goal for taking it was to get familiar with the agile methodology and serve as an accomplishment on my resume for job hunting.

This certification costs $150, has no requirements, and is taken online.

Note that the certification I passed was the PSM I certification, not the CSM. They are different - I generally recommend the PSM as it doesn't require you to complete an expensive training like the CSM.

My Process

I dedicated three days for studying - here is everything I did to study for the exam broken down by day. I spent 4-5 hours per day on these tasks.

Day 1

Read the Scrum Guide, an 18 page document written by the founders of Scrum. This is the main document that the exam is based on
Re-read the Scrum Guide, taking detailed notes this time
Go over the Scrum glossary and take notes. These terms are referred to in the exam so make sure you know them

Day 2

Review Scrum Guide notes
Review Scrum Glossary notes
Do the Scrum Open Assessment. This is a 30 question practice exam from the official Scrum organization [Scored 75%]
Do Mikhail Lapshin's practice quiz in learning mode [Scored 78%]
Do Mikhail Lapshin's practice quiz in real mode [Scored 93%]
Valentin Depsa's Udemy Course, sections 1-9. This was great to visualize the material from the Scrum Guide and see it in practice
Valentin Depsa's practice exam #1 [Scored 85%]

Day 3

Review Scrum Guide notes
Review Scrum Glossary notes
Review notes from practice exams
Valentin Depsa's practice exam #2 [Scored 82%]
Do the Scrum Open Assessment a second time [Scored 92% this time]
Take the exam [passed with 97.5%]

Tips for the exam

If you follow what I did, you'll be more than ready for the exam. Here are a few additional thoughts I want to stress:

Look over your notes each day to reinforce the material
Note what questions you got wrong from practice exams (either in flashcards or jot them down) to review so you don't make the same mistakes on your certification
The main material of the exam is the Scrum Guide. I highly recommend reading this at least 2-3 times and knowing it well

Conclusion

You now have all the tools and resources you need, along with a detailed process you can follow or refer to. Go get that certification!