DEV Community: Raphael Araújo

Integrating Google Firebase Firestore with ChatGPT API. Saving money

Raphael Araújo — Fri, 14 Apr 2023 11:55:46 +0000

As commented in my previous post, I developed a services architecture to save a little when consuming the OpenAI API and the gpt-3.5-turbo model.

Now I’m going to show you some of the code that I inserted in Firebase functions that is called every time a new question is inserted in Firestore.

This function will send the question data registered by the user to a service on Render.com where the API of the ChatGPT model will be consumed.

It’s worth remembering why I didn’t do everything on the Firebase Cloud Function side:

Cloud Function charges based on how long your function runs, as well as the number of invocations and provisioned resources. As the ChatGPT API can be slow to respond depending on the complexity of your query, you could end up paying a lot for the time your function is waiting for the API response.

At the end of the process, the answer to the question will be updated in Firestore based on the data received from the ChatGPT API.

We can highlight some important snippets of the previous code:

Lines 13 and 14: These are custom methods that communicate with Pinecone and the OpenAI API. I suggest looking for more information at https://python.langchain.com/en/latest/use_cases/question_answering.html
Line 60: In the previous lines, the code is responsible for searching the database of questions already asked by users and finding the most similar question. Based on the most similar question ever asked before, line 60 is responsible for checking whether the similarity is so close (95%) that the answer from the previous question can be used to answer the new question. As I commented in my previous post, this comparison would not do very well to different questions such as: How much does 1 kg of your product cost?’ and ‘How much does 1g of your product cost?’.
Line 71: This part of the code solved my problem with the OpenAI API delay. Some may wonder why I haven’t used something related to background processing queues. But as I mentioned in the previous post, my goal, for now, is to look for cheaper alternatives. Hiring a Redis bank and a full-time worker is not my current plan. But changing that is definitely one of my future plans.

Documents that can help you in the development:

References:

Integrando a API do ChatGPT com o Google Firebase Firestore

Raphael Araújo — Thu, 06 Apr 2023 16:05:00 +0000

Como comentado no meu post anterior, eu desenvolvi uma arquitetura de serviços para tentar economizar um pouco na hora de consumir a API da OpenAI e o modelo gpt-3.5-turbo.

Agora vou mostrar um pouco do código que inseri no Firebase functions que é chamado toda vez uma nova perguntar é inserida no Firestore.

Essa função irá enviar os dados da pergunta cadastrada pelo usuário, para um serviço no Render.com onde a API do modelo do ChatGPT será consumida.

Vale relembrar o motivo de eu não ter feito tudo do lado do Firebase Cloud Function:

O Cloud Function cobra de acordo com o tempo de execução da sua função, além do número de invocações e dos recursos provisionados. Como a API do ChatGPT pode demorar para responder, dependendo da complexidade da sua consulta, você pode acabar pagando muito pelo tempo que a sua função fica aguardando a resposta da API.

Ao fim do processo, a pergunta terá sua resposta atualizada no Firestore com base nos dados recibidos da API do ChatGPT.

Podemos destacar alguns trechos importantes do código anterior:

Linhas 13 e 14: São métodos customizados que fazem a comunicação com o Pinecone e a API da OpenAI. Sugiro buscar mais informações em https://python.langchain.com/en/latest/use_cases/question_answering.html
Linha 60: Nas linhas anteriores, o código é responsável por buscar na base de dados de perguntas já realizadas pelos usuários e encontrar a pergunta mais parecida. Com base na pergunta mais parecida já feita anteriormente, a linha 60 é responsável por verificar se a similaridade é tão próxima (95%) que a resposta da pergunta anterior pode ser utilizada para responder a nova pergunta. Como comentei no meu post anterior, esse comparativo não se daria muito bem para diferenciar perguntas como: Quanto custa 1kg do seu produto?’ e ‘Quanto custa 1g do seu produto?’.
Linha 71: Essa parte do código foi responsável por sanar meu problema com o delay da API da OpenAI. Alguns podem estar se perguntando o porquê de eu não ter utilizado algo relacionado a filas de processamento em background. Mas como comentei no post anterior, meu objetivo por agora é buscar alternativas mais baratas. Contratar um banco Redis e um worker em periodo integral, não está nos meu planos por agora. Mas, mudar isso, com certeza está em um dos meu planos futuros.

Documentações que podem lhe ajudar no desenvolvimento:

Referências:

Trying to Maintain a Workable Budget Creating a Chatbot Using GPT and Vector Database

Raphael Araújo — Wed, 05 Apr 2023 10:47:03 +0000

In this post, I’ll show you how to use a vector database to lower GPT token costs in a Q&A application. The vector database I chose was Pinecone, which allows you to store and query high-dimensional vectors in an efficient and scalable way. The idea is to turn the questions and answers into vectors using a pre-trained natural language model, such as the text-embedding-ada-002 model, and then use Pinecone to find the most similar answers to users’ questions.

Vector databases can provide better text query results than SQL databases because they use a mathematical representation of the data called a vector, which allows you to measure the similarity between documents and queries using operations such as distance or angle. SQL databases, on the other hand, use a structured query language (SQL) that can be limited to searching for exact or fuzzy matches of words or phrases using the CONTAINS command. Also, SQL databases can require more resources and time to index and search large amounts of text data than vector databases.

I used Firebase Firestore, a NoSQL database in the cloud that offers real-time sync and offline support to manage the questions. Each question is stored in a document with a unique identifier and a field for the answer. Firestore also lets you create functions in the cloud that are triggered by events in the database, such as creating, updating, or deleting documents. I used a Cloud Function to fire an event whenever a new question is registered in Firestore. This function is responsible for sending the question to the ChatGPT API, a service that uses the GPT-3.5-turbo model to generate conversational responses. The ChatGPT returns the answer in text format, which is then converted to a vector using the same natural language model that was used for the questions. This vector is sent to Pinecone, which stores the vector for future new queries.

One advantage of using Pinecone is that it allows you to do similarity queries using the question and answer vectors. So when a user asks a question, I don’t need to send the question to ChatGPT and spend GPT tokens. I can simply convert the query into an array and send it to Pinecone, which returns the identifiers of the most similar arrays. I can adopt an existing answer as the question’s answer based on the similarity score between the new question and questions that have been asked before. This significantly reduces GPT token costs, as I only need to use ChatGPT to generate answers to new or significantly different questions than existing ones.

It can be quite a challenge to find the exact answer to the user’s question based on previous questions. Example: ‘How much does 1kg of your product cost?’ or ‘How much does 1g of your product cost?’. Possibly the score of the vectorized text of the two questions will have a score of almost 1.0. This can be a problem depending on your interest. I haven’t come up with anything concrete to solve this problem, but I believe there are ways around this by defining differentiations between some words such as ‘kilogram’ and ‘gram’.

To compare the cost between Pinecone queries and the GPT-3.5-turbo model , I used the following values:

The price of Pinecone is $0.096 / hour.
ChatGPT price is $0.002 / 1K gpt-3.5-turbo tokens.

Tokens are common strings of characters found in text. GPT processes text using tokens and understands the statistical relationships between them. Tokens can include spaces and even subwords. The maximum number of tokens that GPT can take as input depends on the model and tokenizer used. For example, the text-embedding-ada-002 template uses the cl100k_base tokenizer and can receive up to 8191 tokens.

I even considered using Cloud Function as the hosting platform for my ChatGPT query code, but it would bring me high financial costs. That’s because Cloud Function charges based on your function’s runtime and the number of invocations and provisioned resources. As the ChatGPT API can be slow to respond depending on the complexity of your query, you could end up paying a lot for the time your function is waiting for the API response.

One way I found to reduce costs with ChatGPT is to use a web service on Render.com, where it is possible to create an application in Flask where I managed to use threads so as not to wait for the GPT API to respond before responding to the Cloud Function. Render.com is a platform that lets you host web applications in a simple and inexpensive way, with plans starting at $7 per month. The idea is to create an intermediate layer between the Cloud Function and the ChatGPT, which receives the question from the Cloud Function and sends an immediate response saying the response is being generated. Then, the Flask application creates a thread to send the question to the ChatGPT API and once it gets the answer, it updates the question in Firestore.

I had thought of describing the project's source code in this post. But I think that for now, it is worth reflecting on the applied infrastructure. Later I will create new posts to detail more about the NodeJS project inserted in Google Cloud Function and the project in Python (Flask) hosted on Render.com.

The web application for inserting and querying questions in Firestore is still being developed. I intend to publish it soon as well.

Tentando não ficar pobre antes de ficar rico criando uma Startup de serviços de inteligência artificial

Raphael Araújo — Tue, 04 Apr 2023 20:17:18 +0000

Neste post, vou mostrar como utilizar um banco de dados vetorial para diminuir os custos com tokens do GPT em uma aplicação de perguntas e respostas. O banco de dados vetorial que eu escolhi foi o Pinecone, que permite armazenar e consultar vetores de alta dimensão de forma eficiente e escalável. A ideia é transformar as perguntas e as respostas em vetores usando um modelo de linguagem natural pré-treinado, como o modelo text-embedding-ada-002, e depois usar o Pinecone para encontrar as respostas mais similares às perguntas dos usuários.

Bancos de dados vetoriais podem oferecer melhores resultados de consultas de texto do que os bancos de dados SQL porque eles usam uma representação matemática dos dados chamada vetor, que permite medir a similaridade entre documentos e consultas usando operações como distância ou ângulo. Os bancos de dados SQL, por outro lado, usam uma linguagem de consulta estruturada (SQL) que pode ser limitada para pesquisar correspondências exatas ou difusas de palavras ou frases usando o comando CONTAINS. Além disso, os bancos de dados SQL podem exigir mais recursos e tempo para indexar e pesquisar grandes quantidades de dados de texto do que os bancos de dados vetoriais.

Para gerenciar as perguntas, eu usei o Firebase Firestore, um banco de dados NoSQL na nuvem que oferece sincronização em tempo real e suporte offline. Cada pergunta é armazenada em um documento com um identificador único e um campo para a resposta. O Firestore também permite criar funções na nuvem que são acionadas por eventos no banco de dados, como a criação, atualização ou exclusão de documentos. Eu usei uma Cloud Function para disparar um evento sempre que uma nova pergunta for cadastrada no Firestore. Essa função é responsável por enviar a pergunta para a API do ChatGPT, um serviço que usa o modelo GPT-3.5-turbo para gerar respostas conversacionais. O ChatGPT retorna a resposta em formato de texto, que é então convertida em um vetor usando o mesmo modelo de linguagem natural que foi usado para as perguntas. Esse vetor é enviado para o Pinecone, que armazena o vetor para futuras novas perguntas.

Uma vantagem de usar o Pinecone é que ele permite fazer consultas por similaridade usando os vetores das perguntas e das respostas. Assim, quando um usuário faz uma pergunta, eu não preciso enviar a pergunta para o ChatGPT e gastar tokens do GPT. Eu posso simplesmente converter a pergunta em um vetor e enviar para o Pinecone, que me retorna os identificadores dos vetores mais similares. Eu posso adotar uma resposta existente como a resposta da pergunta baseado no score de similaridade entre a nova pergunta e as perguntas que já foram feitas anteriormente. Isso reduz significativamente os custos com tokens do GPT, já que eu só preciso usar o ChatGPT para gerar respostas para perguntas novas ou muito diferentes das existentes.

Pode ser um grande desafio encontrar a resposta exata para a pergunta do usuário com base nas perguntas realizadas anteriormente. Exemplo: ‘Quanto custa 1kg do seu produto?’ ou ‘Quanto custa 1g do seu produto?’. Possivelmente o score do texto vetorizado das duas perguntas terá um score quase de 1.0. Isso pode ser um problema dependendo do seu interesse. Não cheguei a pensar em algo concreto para resolver esse problema, mas acredito que haja formas de contornar isso definindo diferenciações entre algumas palavras tais como ‘kilograma’ e ‘grama’.

Para fazer um comparativo entre o custo entre as consultas do Pinecone e o modelo GPT-3.5-turbo, eu usei os seguintes valores:

O preço do Pinecone é de $0.096 / hora.
O preço do ChatGPT é de $0.002 / 1K tokens do gpt-3.5-turbo.

Tokens são sequências comuns de caracteres encontrados no texto. O GPT processa o texto usando tokens e entende as relações estatísticas entre eles. Os tokens podem incluir espaços e até sub-palavras. O número máximo de tokens que o GPT pode receber como entrada depende do modelo e do tokenizador usados. Por exemplo, o modelo text-embedding-ada-002 usa o tokenizador cl100k_base e pode receber até 8191 tokens.

Cheguei a considerar o uso do Cloud Function como plataforma de hospedagem do meu código de consulta ao ChatGPT, mas me traria altos custos financeiros. Isso porque o Cloud Function cobra de acordo com o tempo de execução da sua função, além do número de invocações e dos recursos provisionados. Como a API do ChatGPT pode demorar para responder, dependendo da complexidade da sua consulta, você pode acabar pagando muito pelo tempo que a sua função fica aguardando a resposta da API.

Uma forma que encontrei de reduzir os custos com o ChatGPT é usar um web service no Render.com, onde é possível criar uma aplicação em Flask onde eu consegui utilizar threads para não aguardar a API do GPT responder antes de já responder ao Cloud Function. O Render.com é uma plataforma que permite hospedar aplicações web de forma simples e barata, com planos a partir de $7 por mês. A ideia é criar uma camada intermediária entre o Cloud Function e o ChatGPT, que recebe a pergunta do Cloud Function e envia uma resposta imediata dizendo que a resposta está sendo gerada. Em seguida, a aplicação em Flask cria uma thread para enviar a pergunta para a API do ChatGPT e assim que obtiver a resposta, atualize a pergunta no Firestore.

Havia pensado em descrever o código fonte do projeto nesse post. Mas acho que por hora, vale a reflexão sobre a infra aplicada. Posteriormente estarei estarei criando novos post para detalhar mais sobre o projeto em NodeJS inserido no Google Cloud Function e o projeto em Python (Flask) hospedado no Render.com.

A aplicação web responsável por inserir e consultar as perguntas no Firestore ainda está em fase de desenvolvimento. Pretendo publicá-la em breve também.

Consuming a Rails API with a NextJs client

Raphael Araújo — Mon, 28 Nov 2022 13:08:20 +0000

tl;dr; Previously here, I wrote about how to write a modern web application using Rails as a full-stack framework. Just writing HTML and Vanilla Javascript. Now we will go ahead creating a React client App using NextJs.

The existing API has the following endpoints:

GET /api/kit/products(.:format)
POST /api/kit/products(.:format)
GET /api/kit/products/:id(.:format)
PATCH /api/kit/products/:id(.:format)
PUT /api/kit/products/:id(.:format)
DELETE /api/kit/products/:id(.:format)

In the new project, we have the same screen as before:

We have a lot of customized components:

src/components/
├── Form
│   ├── Actions.tsx
│   ├── Input.tsx
│   └── index.tsx
├── Layout
│   ├── Page.tsx
│   ├── Sidebar.tsx
│   └── index.tsx
├── Loading.tsx
├── LoadingOverlay.tsx
├── Notification.tsx
├── Products
│   ├── Form.tsx
│   └── Sidebar.tsx
└── SearchList
    ├── Form.tsx
    ├── ListItem.tsx
    └── index.tsx

But our pages are simple and short. For example, take a look at the products page code:

Highlights

NextJs is just a choice, not a requirement

The API made using Ruby on Rails is completely independent of the Next JS client application developed with NextJs. You could use any RESTfull client application to consume the existing API.
In my project, I am using the NextJs project as a subfolder of my Rails repository, but you could put it anywhere.

Why Next JS

I already worked with React Router and React Navigation, but when I knew the Next/Router and all related features, as the Next/Link, I loved it. We can use partial load and caches. Get more info here.

Conventions

You can create your own convention for your own projects. But, in my opinion, it is beneficial to use a convention that is popular and validated in production by many other developers. Like Ruby on Rails, NextJs gives you a directory structure, core resources (link, routes, image, etc.) and rich documentation.

SSR

After to create a few projects using SPA, it doesn’t seem to me a good choice for big projects. So, for now, I am using SSR with NextJs. The main use of SSR is to improve the SEO, but like this approach to offer the a better UX.

React Query is the link between the Rails and NextJs

Working together with Axios (my code), it is a great option to consume the REST API (and GraphQL). You have the access to: isLoading, isError, data, error, and others. It is a very easy way to load data and rescue errors.

React Context

I don’t like https://react-redux.js.org/, it brings a complexity to the project that I don’t think soo is a good thing. But we can use the React Context and React Reducer to offer a store and events to manager states of the application. You can see the it on the project here https://github.com/raphox/rails-7-fullstack/tree/nextjs/frontend/src/contexts/products.
In my code, I am using a context to share the state between different components on the same screen. The Form component is able to update the product item in the sidebar list.
The React Query also uses a context and we can exchange states between them. After updating a product, we can trigger changes to the product list.

React components with namespace

It is something that I learned recently. I used it in my project to offer a way to override children of some components and prevent to set many properties throth the parent. Like in the following code:

In the previous code, we have the namespace SidebarPrimitive with nested Root, Header, and List components. I am using the Root component to wrap the content and I am passing the props to the respected child.

Tailwind

There are controversies related to it, but trust me, create a project using it and make sure that you don’t like or love it.

Dependencies:

https://tailwindcss.com/ “Tailwind CSS works by scanning all of your HTML files, JavaScript components, and any other templates for class names, generating the corresponding styles and then writing them to a static CSS file.”
https://www.radix-ui.com/docs/primitives/utilities/slot “Merges its props onto its immediate child.”
https://tanstack.com/query/ “Powerful asynchronous state management for TS/JS, React, Solid, Vue and Svelte”
https://axios-http.com/ “Axios is a simple promise based HTTP client for the browser and node.js. Axios provides a simple to use library in a small package with a very extensible interface.”
https://github.com/lukeed/clsx “A tiny (228B) utility for constructing className strings conditionally.”
https://lodash.com/ “A modern JavaScript utility library delivering modularity, performance & extras.”
https://react-hook-form.com/ “Performant, flexible and extensible forms with easy-to-use validation”
https://github.com/jquense/yup “Yup is a schema builder for runtime value parsing and validation. Define a schema, transform a value to match, assert the shape of an existing value, or both.”

The project

rails-7-fullstack/frontend at nextjs · raphox/rails-7-fullstack

External references:

Rails 7 + Hotwire (Turbo + Stimulus) = Modern web applications

Raphael Araújo — Thu, 24 Nov 2022 12:20:29 +0000

tl;dr; Let’s understand how we can migrate a scaffold generated by the command ‘bin/rails g scaffold Kit::Product name:string’, to an improved and performative screen. Inserting partial load and DOM manipulation writing a very little code in VanillaJS (“almost” without to write Javascript).

Scaffold (Rails 7)

Getting Started with Rails (https://guides.rubyonrails.org/getting_started.html#creating-a-new-rails-project-installing-rails)
Generating the scaffold (https://guides.rubyonrails.org/command_line.html#bin-rails-generate):
Using just one command you will generate all of these files: Model, Migration, Controllers, Routes, Tests, Helpers, and JSON.

    $ **bin/rails generate scaffold HighScore game:string score:integer**
        invoke  active_record
        create    db/migrate/20190416145729_create_high_scores.rb
        create    app/models/high_score.rb
        invoke    test_unit
        create      test/models/high_score_test.rb
        create      test/fixtures/high_scores.yml
        invoke  resource_route
         route    resources :high_scores
        invoke  scaffold_controller
        create    app/controllers/high_scores_controller.rb
        invoke    erb
        create      app/views/high_scores
        create      app/views/high_scores/index.html.erb
        create      app/views/high_scores/edit.html.erb
        create      app/views/high_scores/show.html.erb
        create      app/views/high_scores/new.html.erb
        create      app/views/high_scores/_form.html.erb
        invoke    test_unit
        create      test/controllers/high_scores_controller_test.rb
        create      test/system/high_scores_test.rb
        invoke    helper
        create      app/helpers/high_scores_helper.rb
        invoke      test_unit
        invoke    jbuilder
        create      app/views/high_scores/index.json.jbuilder
        create      app/views/high_scores/show.json.jbuilder
        create      app/views/high_scores/_high_score.json.jbuilder

After installing Rails, create your project, and generate your scaffold. Let's try to improve the default interface.

I am using the Tailwind to insert styles on the interfaces and the ViewComponent for “creating reusable, testable & encapsulated view components, built to integrate seamlessly with Ruby on Rails”.

You could see the resulting code at https://github.com/raphox/rails-7-fullstack.

Turbo (heart of Hotwire)

Hotwire is an alternative approach to building modern web applications without using much JavaScript by sending HTML instead of JSON over the wire.

You can see the unique application’s screen in the following image. Yes, this project has just one screen. Let’s try to be simple to focus on the most important things.

In this screen, we have two dynamic areas:

List of products
Form to create and edit the selected product

I created a PR so you can see the exact part of the code that was changed or inserted to use the Hotwire on your project. You can see this PR on https://github.com/raphox/rails-7-fullstack/pull/1.

The unique screen on the system has a few triggers to call actions or make requests.

Plus button to access the ‘kit/products/new’ page
Search input field
Products links to access the product’s edit page
Buttons to update, remove or create products based on the form’s fields data

Using the Turbo, we can override the click event of the plus button to request the HTML of a new product form from the backend and put it in the ‘product_form’ area.

You don’t need to write Javascript to override the events on your elements, just insert a new attribute ‘turbo_frame’ on your HTML tag. Like here:

We can use the same approach of the plus button in the link of products and prevent reloading the entire screen HTML and assets.

Now we need to use a little Javascript to insert the event ‘change’ to request a filtered list of products from the backend.

In my code, I insert the ‘turbo_frame’ attribute into the form to change the submit action making the request using Turbo. In the next topic, we can see how to use the Stimulus and Javascript controllers to change the event ‘change’ of the input.

Now we will need to submit the form and refresh the list of products. But we can do this better. We know that the updated list has changed on the created/updated product. So, we need to update the list with this product, instead refresh and query all the list of products from the database. To do this we need to use a lot of the Javascript code or a few actions from the Stimulus.

You can see about the Javascript controller in the next topic.

Stimulus (modest JavaScript framework)

You won’t see any difference. But the code savings and ease of integration will help you a lot when inserting new features.

The objective here is to insert an event on the search input form to make a request when the user changes the input’s value. When it occurs, the system will make a request to get all products filtered by the ‘name’ column. The result will be inserted into the list after it.

To implement the Stimulus in our current project, we need to follow the following steps:

1. Insert the attribute ‘data-controller=”search-list”’ to the wrapper with the form and the list of the products (app/components/search_list_component.html.erb line 1).
— This attribute is responsible to relate the following code to the DIV element. Get more info here https://stimulus.hotwired.dev/.
— You can use the command ‘./bin/rails generate stimulus controllerName’ to generate the Javascript file with the class definition and import it in the ‘app/javascript/controllers/index.js’ automatically.

2. Insert the attribute ‘data-search_list_target=”inputSearch”’ to the form search field (app/components/search_list_form_component.html.erb line 2).

3. Insert the action (click->search-list#selectItem) to activate the currently selected product (app/components/search_list_item_component.rb lines 20).

4. Insert the attribute ‘search_list_target=”item”’ to each item of the product list (app/components/search_list_item_component.rb lines 21).

I am using https://viewcomponent.org/ to define the view components instead to use a view or partial. Pay attention to this part https://viewcomponent.org/#performance.

I really liked having this structure in my project:
rails-7-fullstack/_list.html.erb at 11a479dafa5c84a1e769721d57792f39914a1204 ·…

This approach is very similar to the React components or other Javascript Frameworks.

I created a PR where you can see the exact part of the code I changed or inserted to use the Stimulus on my project. You can see this PR on https://github.com/raphox/rails-7-fullstack/pull/2.