DEV Community: trannguyenhung011086

Build a small Q&A bot with OpenAI

trannguyenhung011086 — Thu, 03 Oct 2024 07:46:33 +0000

Recently I registered a developer account with OpenAI to play around cause it is quite popular these days and I am curious to see what I can use it for. Technically I am not able to understand how OpenAI is doing all those magic behind the scene, I feel it is very useful and time effective when using it properly for a business purpose on a high level.

So taking inspiration from a course on Educative (https://www.educative.io/courses/build-open-ai-applications-using-javascript) which is a very nice introduction and gives some basic ideas of how we can use OpenAI with JavaScript, I have just made a new repo to implement a simple application to parse and analyze documents as a Q&A bot.

Some use cases I can think of:

You are a headhunt and you want to have a quick scan of a candidate's resume to get his/her most notable skills.
You are a customer support and you want to upload a Knowledge base document to let users easily ask questions for it.
Actually, OpenAI can process many kind of documents. But for a better learning scope, this project focuses on parsing resume pdf files only.

Folder Structure

/my-monorepo
│
├── /apps
│   ├── /backend    # Express.js backend
│   │   ├── src
│   │   │   ├── index.ts
│   │   │   └── routes.ts
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   └── .env
│   └── /frontend   # React.js or any frontend framework
│       ├── src
│       │   ├── index.tsx
│       │   └── App.tsx
│       ├── package.json
│       └── tsconfig.json
├── package.json    # Monorepo-level package.json with workspace settings
└── .gitignore

Basically, I use a mono repo approach with nom workspace to develop backend and frontend parts separately. And eventually they will be deployed as separate services just like in real world projects.

Backend

The backend is an Express.js app which expose an endpoint POST /api/retrieve-answer to pass the question for the document to multiple methods from langchain library to retrieve the final answer.

Overall flow will be like this:

The structure is like a basic Express app with middleware, route handlers, commands, etc. to handle the above workflow.

├── src
│   ├── assets
│   ├── commands
│   ├── env.ts
│   ├── index.ts
│   ├── methods
│   ├── middlewares
│   ├── repos
│   ├── routeHandlers
│   ├── routes.ts
│   ├── services
│   └── types

And for demo purpose, I do not use a real data store to store files persistently like Postgres. Instead those files are stored as public assets and exposed as static files via Express server.

Another thing is to avoid consuming too much OpenAi credits when invoking their API, I implement a simple LRU cache in memory createLRUCacheProvider to cache the answer to return for known questions.

export function createLRUCacheProvider<T>({ ttl, itemLimit }: LRUCacheProviderOptions): LRUCacheProvider<T> {
  const cache = new Map<string, CacheEntry<T>>();

  return {
    has: (key: string) => {
      const entry = cache.get(key);

      if (!entry) {
        return false;
      }

      if (entry.expiry < Date.now()) {
        cache.delete(key);
        return false;
      }

      // Move accessed item to the end (most recently used)
      cache.delete(key);
      cache.set(key, { ...entry, expiry: Date.now() + ttl });
      return true;
    },
    get: (key: string) => {
      const entry = cache.get(key);

      if (!entry) {
        return undefined;
      }

      if (entry.expiry < Date.now()) {
        cache.delete(key);
        return undefined;
      }

      // Move accessed item to the end (most recently used)
      cache.delete(key);
      cache.set(key, { ...entry, expiry: Date.now() + ttl });

      return entry.value;
    },
    set: (key: string, value: T) => {
      const entry = cache.get(key);
      if (entry) {
        cache.delete(key);
      }

      // Get the first item (least used)
      if (cache.size >= itemLimit) {
        const lruKey = cache.keys().next().value;
        if (lruKey) cache.delete(lruKey);
      }

      cache.set(key, {
        value,
        expiry: Date.now() + ttl,
      });
    },
  };
}

const lruCache = createLRUCacheProvider({
  ttl: 86_400_000 * 3, // 3 days
  itemLimit: 50,
});

Frontend

Frontend will be a HTML web page, on which we will render a form where the user can provide questions and click the "Get answer!" button. Once we get the answer from OpenAI using the PDF document, we will render the answer on the web page with a button to copy the answer to the clipboard. This is built with React and Vite.

Components:

ProfileList: display a list of candidate's profiles to select. The list will be fetched on page load, and the first record is used as default selection.
PDFViewer: render the PDF file for selected profile.
QuestionForm: render a form for user to input questions for the selected profile.

And the UI is something like this:

Full source code can be found at https://github.com/trannguyenhung011086/resume-analyzer

There are still many improvements to make for this project like:

Add an endpoint to allow uploading pdf files
Add a controller to store pdf files on Postgres
Add methods to store questions and answers on Postgres
Use Redis to cache answers from OpenAI
Use rate limiter to avoid spamming calls to the endpoint
Use server push event or websocket to stream answer response to frontend

Overall, this is a fun learning time to scratch the surface of OpenAI usage even though I haven't had a real project to use it yet. But embracing it will definitely open new opportunities in the future.

Explore Google Cloud functions for micro tasks

trannguyenhung011086 — Sat, 23 May 2020 05:15:39 +0000

Given tasks of integrating our current system with multiple third-party services for customer support purposes at RealStake, I often make use of available webhooks to distribute data to multiple places. But there are some trouble like bringing up separate Express app for such goals or how we centralize logs. Therefore I decide to explore how to utilize Google cloud functions in current tasks.

Under the hood, Google cloud function is already using Express library, so it is really easy to write functions as mini Express apps then deploy right away from local machines or set up Github actions for it. So connecting the dots, I come up with below flow.

The basic idea is the separate of concerns from updating data in our internal system and third party results via cloud functions.

For example, the requirements are after user signing up successfully via the tracking_id stored in browser cookies, such data needs to be sent to AccessTrade (a marketing affiliate provider). Then after our customer support team confirm such users are legitimate, we trigger an update for user status on AccessTrade for reporting.

First, I make an index.js file to export our Express app as a single cloud function.

Next is creating our Express app for necessary routes interacting with AccessTrade service.

I will not go into details of business route implementation because it is exactly the same approach when we normally built REST api with Express app.

Then I think of how we centralize error log format in case we put in more third party service cloud functions in the future. It is convenient that Google already has provided a plugin for the famous Winston logging library.

I really love this library because we can define and format log to a universal JSON format and stream it to JSONpayload of Google cloud logging system.

Note: to test cloud functions locally, we can use https://cloud.google.com/functions/docs/functions-framework.

So far so good, I can make use of lightweight cloud functions and have a base structure for integrating different third party services later.

However, to push my learning further, I am considering using Google Cloud PubSub to apply event-based trigger patterns. The main idea is using a cloud function A as publiser to publish data to a defined topic, then another cloud function B will subscribe to it and update data back to our internal system.

But that will be for my later post.

How I schedule jobs for my pet project with Agenda JS

trannguyenhung011086 — Wed, 18 Dec 2019 08:11:13 +0000

While playing with my small Express app a bit (https://dev.to/trannguyenhung011086/first-try-with-express-2a3p), I started to diving more in NodeJS events method to make a simple and manageable email sending scheduler.

Below is how I develop my code from using the native NodeJS eventEmitter to Agenda JS methods.

First of all is the mailer service containing methods:

send active email when user register

// mailer.js

const config = require('../config');
const mailer = require('@sendgrid/mail');

mailer.setApiKey(config.sendgrid);

module.exports = {
    sendActiveEmail: async ({ username, email, url }) => {
        try {
            const msg = {
                to: email,
                from: 'learn-svelte@example.com',
                subject: 'Your active link',
                text: `Hi ${username}, please click the following url to activate your account: ${url}.\nThank you!`,
                html: `<p>Hi <strong>${username}</strong>,</p> 
                <p>Please click <a href=${url}>the following url</a> to activate your account.</p>
                <p>Thank you!</p>`,
            };
            const send = await mailer.send(msg);
            console.log('Active email is sent to ' + email);
            return send;
        } catch (err) {
            console.error('Cannot send email!\n', err);
        }
    },
};

Then come the user service which is used for the Express user routes. Here I define 'register' event for the eventEmitter method.

// userService.js

module.exports = {
...
createUser: async ({ username, email, password }) => {
        const payload = { username, email, password };
        await validateRegister(payload);

        ...

        payload.password = await bcrypt.hash(password, 10);
        let newUser = await UserModel.create(payload);

        const send = myEmitter.emit('register', {
             username,
             email,
             url: newUser.activeLink,
         });
         if (!send) myEmitter.emit('error');

        ...

        return newUser;
    },
}
,

Then I call mailer service in event subcribers.

// event.js

const EventEmitter = require('events');
const myEmitter = new EventEmitter();

const mailer = require('../services/mailer');

myEmitter.on('register', async ({ username, email, url }) => {
    await mailer.sendActiveEmail({ username, email, url });
});


module.exports = myEmitter;

The above code work quite well for the case of sending active email right after user registers. But what about if I want to send a welcome email in 1 minutes after they click the active link?

First I think of using setTimeout method but it is not an easy way to scale later with more scheduled jobs. That was when I found out about Agenda JS which is a lightweight solution for such purpose.

You can check how to install Agenda as instructed at https://github.com/agenda/agenda. Please note, it requires to connect to Mongo database to work.

Then I started modifying my code a bit as below.

// agenda.js

const Agenda = require('agenda');
const config = require('../config');

const mailer = require('../services/mailer');

const agenda = new Agenda({
    db: { address: config.database, collection: 'agendaJobs' },
});

agenda
    .on('ready', () => console.log('Agenda started!'))
    .on('error', () => console.log('Agenda connection error!'));

agenda.define('register', async job => {
    const { username, email, url } = job.attrs.data;
    await mailer.sendActiveEmail({ username, email, url });
});

agenda.define('welcome', async job => {
    const { username, email } = job.attrs.data;
    await mailer.sendWelcomeEmail({ username, email });
});

agenda.start();

module.exports = agenda;

Similar to the native event subscriber method, here I define jobs for Agenda to consume. Then in another file, I define methods for Agenda to publish.

// scheduler.js

const agenda = require('./agenda');

module.exports = {
    scheduleActiveEmail: async ({ username, email, url }) => {
        await agenda.schedule('in 1 second', 'register', {
            username,
            email,
            url,
        });
    },

    scheduleResendActiveEmail: async ({ username, email, url }) => {
        await agenda.schedule('in 1 second', 'resend', {
            username,
            email,
            url,
        });
    },

    scheduleWelcomeEmail: async ({ username, email }) => {
        await agenda.schedule('in 30 seconds', 'welcome', { username, email });
    },
};

Next, I use the new Agenda publisher methods in user service.

// userService.js

module.exports = {
    createUser: async ({ username, email, password }) => {
        const payload = { username, email, password };
        await validateRegister(payload);

        ...

        payload.password = await bcrypt.hash(password, 10);
        let newUser = await UserModel.create(payload);

        await scheduler.scheduleActiveEmail({
            username,
            email,
            url: newUser.activeLink,
        });

        return newUser;
    },

    activeUser: async ({ userId, uuid }) => {
        const user = await getUserById(userId);

        ...

        await scheduler.scheduleWelcomeEmail({
            username: user.username,
            email: user.email,
        });

        user.active = true;
        user.welcome = true;
        await user.save();

        return { userId: user._id, active: user.active };
    },
};

Finally, after the jobs are processed, I can query of job data stored in AgendaJobs collection on Mongo database.

In summary, Agenda JS is a suitable solution for simple management of scheduling jobs in NodeJS rather than using the native setTimeout or traditional cron jobs.

It even provides a dashboard solution which I will continue to learn more and write about it soon :)

First try with Express

trannguyenhung011086 — Wed, 27 Nov 2019 13:16:39 +0000

While working as automation QA using NodeJS, I always have big interest in how to build up a web application and recently I have decided to give it a try with the de-facto Express framework.

Starting with the basic guide from Mozilla (https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs), I find it very coherent and easy to follow. But there is a trouble that the guide is using the callback style which is both convenient and troublesome to use in my opinion.

// Display list of all Books.
exports.book_list = function(req, res, next) {

  Book.find({}, 'title author')
    .populate('author')
    .exec(function (err, list_books) {
      if (err) { return next(err); }
      //Successful, so render
      res.render('book_list', { title: "'Book List', book_list: list_books });"
    });

};

Besides, I am using Promise and async-await style for my automation test suites at RealStake, so I try converting the guide to the latter style for easier controlling the code flow.

module.exports = {
    getList: async (req, res, next) => {
        try {
            const bookInstanceList = await BookInstance.find({})
                .populate('book')
                .exec();
            res.render('bookInstanceList', {
                title: "'Book Instance List',"
                bookInstanceList,
            });
        } catch (err) {
            next(err);
        }
    },
}

And the experience is quite amazing because I can achieve both the basic understanding of how Express works and understand more about callback using.

After finishing the guide, I try to relate to the current project at my company and plan to add more features like JWT authentication, Redis storage, etc. to explore more on building a real application. Then I make up my own Trello board (https://trello.com/b/K0mY1Jpo/learn-express) to define and monitor what I have been doing and what I will implement.

By making use of the Trello board, it is easier for me to control my learning flow. It also makes me feel more organized :D

So far I have been able to implement more routes to my small local library app such as:

JWT authentication for log in
blacklist JWT token for log out
register new account and send activation email via Sendgrid
...

And for deployment, I already have some experience with using Zeit for my automation workflow with Asana (https://dev.to/trannguyenhung011086/how-i-sync-github-pull-request-status-to-asana-task-flow-nl2).

So at first, I deployed my app as a normal Express server.

const config = require('./common/config');
const mongo = require('./loaders/mongo');
const redis = require('./loaders/redis');
const app = require('./loaders/express');

app.listen(config.port, () =>
    console.log(`App listening on port ${config.port}`),
);

Then I found an article about using Zeit for serveless functions (https://medium.com/@bmikkelsen22/designing-a-serverless-express-js-api-using-zeit-now-6e52aa962235) and tried to change my code a bit.

It is quite surprising that such task does not take so much effort because Zeit already see an Express app as a function by using @now/node option for deployment. I just need to map routes to those functions.

// now.json
{
    "version": 2,
    "name": "learn-express",
    "builds": [{ "src": "/api/*.js", "use": "@now/node" }],
    "routes": [
        { "src": "/catalog", "dest": "/api/catalog.js" },
        { "src": "/catalog(/.*)", "dest": "/api/catalog.js" },
        { "src": "/user(/.*)", "dest": "/api/user.js" },
        { "src": "(/.*)", "dest": "/api/home.js" }
    ],
}

// api/catalog.js
const app = require('./setup');
const catalogRouter = require('../routes/catalogRoute');
const authMiddleware = require('../middlewares/authMiddleware');

app.use(
    '/catalog',
    authMiddleware.validateToken,
    authMiddleware.refreshToken,
    catalogRouter,
);
app.use('*', (req, res) => {
    res.status(404).send('Page not found!');
});

module.exports = app;

The whole code can be found at https://github.com/trannguyenhung011086/learn-express.

To sum up, by learning to build an application with Express, I gain more insight in development works and feel really hooked with pushing my Node JS skills as far as possible. And I will definitely continue exploring more in the path of building web application.

How I sync Github pull request status to Asana task flow

trannguyenhung011086 — Sun, 24 Nov 2019 03:30:46 +0000

Currently my team at RealStake is using Asana to manage the task flow. It works quite smooth but there is one issue that we need to move tasks manually for every pull request from Github to sync things up.

In order to solve that bottleneck, I think of using Github webhook to trigger events to a micro server hosted on Zeit then calling to Asana API.

For this simple task, bringing up an Express server may be too much, so I search around and find the inspiration from https://codewithhugo.com/simple-zeit-micro-improve-node/ then decide to give Zeit's Micro a try.

Goals

pull request created - > add comment to asana -> move task to "in review" section
pull request merged - > add comment to asana - > move task to "on dev/stg/prod" section

Stack

use Zeit Now platform for deployment (https://zeit.co)
use Zeit Micro for server (https://github.com/zeit/micro)
use Asana library (https://github.com/Asana/node-asana)

To make this work, first I suggest a rule for my team to include Asana task ID into pull request, e.g. pull request name - ref#task_id so I can get the ID correctly.

function match(toMatch) {
    let result = toMatch.match(/#(ref)?([0-9]{16})|([0-9]{16})/g);
    if (result) {
        return result.map(item => item.replace('#', '').replace('ref', ''));
    }
}

Then I define simple steps to extract Asana IDs from pull request data and push to a unique array.

// github.js
 getAsanaIds: async data => {
        let ids = [];

        // check title
        const title = data['pull_request']['title'];
        const matchTitle = match(title);
        if (matchTitle) matchTitle.forEach(item => ids.push(item));

        // check body
        const body = data['pull_request']['body'];
        const matchBody = match(body);
        if (matchBody) matchBody.forEach(item => ids.push(item));

        // check commits
        const commits = await getCommits(data);
        for (const commit of commits) {
            const matchCommit = await match(commit['commit']['message']);
            if (matchCommit) matchCommit.forEach(item => ids.push(item));
        }

        // check comments and review comments
        const comments = (await getComments(data)).concat(
            await getReviewComments(data),
        );
        for (const comment of comments) {
            const matchComment = await match(comment['body']);
            if (matchComment) matchComment.forEach(item => ids.push(item));
        }

        if (ids.length === 0) throw Error('No Asana task ID found!');

        const uniqueIds = [...new Set(ids)];
        return uniqueIds;
    },

// asana.js
getAsanaTask: async asanaId => {
        const task = await client.tasks.findById(asanaId);
        if (!task) throw Error('Failed to find Asana task with id: ' + asanaId);
        return task;
    },

To move Asana tasks to correct collumn, I need to map their names with pull request status.

// github.js
getAsanaSectionId: (asanaSections, data) => {
        let section;

        if (data.merged === false && data.state === 'open') {
            if (data.base === 'develop') section = 'in review';
            if (data.base === 'release') section = 'staging ready';
            if (data.base === 'master') section = 'production ready';
        }

        if (data.merged === true && data.state == 'closed') {
            if (data.base === 'develop') section = 'on test';
            if (data.base === 'release') section = 'on staging';
            if (data.base === 'master') section = 'done';
        }

        for (const item of Object.keys(asanaSections)) {
            if (item.toLowerCase().includes(section)) {
                return asanaSections[item];
            }
        }
    }

// asana.js
addAsanaTask: async ({ asanaId, projectId, sectionId }) => {
        const data = {
            project: projectId,
            section: sectionId,
        };
        const result = await client.tasks.addProject(asanaId, data);

        if (Object.keys(result).length != 0) {
            throw Error("Failed to change Asana task's section!");
        }
    },

Finally, after moving tasks, I need to add comment to Asana task to update necessary info. for the team members.

// github.js
getPullRequestData: async data => {
        let commit_urls = [];
        const commits = await getCommits(data);
        for (const commit of commits) {
            const item = ` ${commit['html_url']} - ${commit['commit']['message']} - ${commit['committer']['login']}`;
            commit_urls.push(item);
        }

        return {
            title: "data['pull_request']['title'],"
            body: data['pull_request']['body'],
            url: data['pull_request']['html_url'],
            state: data['pull_request']['state'],
            user: {
                login: data['pull_request']['user']['login'],
            },
            head: data['pull_request']['head']['ref'],
            base: data['pull_request']['base']['ref'],
            merged: data['pull_request']['merged'],
            commits: commit_urls,
        };
    }

// asana.js
addComment: async (asanaId, githubData) => {
        const comment = {
            text: `Pull Request ${githubData.url} from ${githubData.user.login}
                Title: ${githubData.title} - Body: ${githubData.body}
                From: ${githubData.head} - To: ${githubData.base} - State: ${githubData.state} - Merged: ${githubData.merged}
                Commits: ${githubData.commits}`,
        };

        const story = await client.tasks.addComment(asanaId, comment);

        if (!story)
            throw Error(
                'Failed to add comment to Asana task with id: ' + asanaId,
            );
    },

Besides, we are using Slack for communicating so it is useful to notify pull request status via defined channels.

async function notify(githubData) {
    const text = `Pull Request ${githubData.url} from ${githubData.user.login}
        Title: ${githubData.title} - Body: ${githubData.body}
        From: ${githubData.head} - To: ${githubData.base}
        State: ${githubData.state} - Merged: ${githubData.merged}`;

    try {
        const channel =
            githubData.base === 'master'
                ? '#pull-requests-master'
                : '#pull-requests';
        const res = await bot.chat.meMessage({
            token,
            channel,
            text,
        });
        if (res.ok) console.log('Notified Slack successfully');
    } catch (e) {
        console.log('Failed to notify Slack', e);
    }
}

And the result is as expected.

The whole code can be found at https://github.com/trannguyenhung011086/sync-asana-github-zeit-micro

From building up this simple workflow, I develop more insight in development tasks like trying to predict more future use cases.

Specifically, at first I only thought of using the exact format ref#task_id but we often forget the format so I must make the regexp pattern more flexible. Another example is where I extract the ids from github pull request. It can be in the title, body, or calling further API call to commit details, comment details, etc.

This is my first time implementing such a task for improving our workflow and the experience is so satisfying :)