Introduction
I was pleased after having contributed to various repositories during Hacktoberfest, but as soon as Hacktoberfest, I had this new profound excitement to contribute to more Open-Sourced Projects. I had contributed to a lot of projects that had a tech stack that included both backend and fronted, but this time around I wanted to contribute to an AI-based project, particularly something related to RAG (Retrieval Augmented Generation) as I wanted to delve into it
ORAssistant
While looking for a lot of repos based on RAG, I stumbled across a perfect open-sourced RAG tool, ORAssistant, which is again, a chatbot that responds to common questions or inquiries for a larger project.
The architecture of this tool is quite complex, I am still trying to figure out how the main query architecture operates, but this is the exciting part, learning as I contribute.
Issue
For my first issue, I picked up one of the issues where the task was to automate the feedback loop, in Leyman terms, what this entails is that an RAG app usually depends on feedback from the user to further fine-tune the response, the task was to get this feedback from the user and store it in a database and feed it back to the model itself
The architecture would look something like this
Currently, the system stores the feedback in Google Sheets, which again is not an optimized approach
This issue in itself would take around 4-5 PRs to get solved, but for the focus of this blog, I will restrict it to the first PR I have made.
First Pull Request
For the first pull request, as evident from the discussions from the issue, my task was to first get the database design set up and running. In doing so, I encountered a lot of issues
Issues faced
- During the setup, the documentation to get the
GOOGLE_SERVICE_KEYwasn't straightforward, so I had to double-check it with the maintainer and tweak a lot of settings in my personal Google account to get it up and running, the maintainers were helpful during the entire process - There were some inconsistencies in the backend which led to the frontend not operating correctly, but one of the good things about this project is since the backend is changing dynamically, they have a mock backend so that while the backend is being developed, the fronted doesn't suffer.
Main solution
The solution that I had proposed for this PR, revolved around the discussion of choosing the right database, after an elaborate discussion with the maintainer, we decided that it was best to use MongoDB for the project, thinking about scalability and flexibility in the fields due to the schema-less nature of MongoDB.
After making the initial design, I opened a PR, which was related to having the initial design setup for the frontend
One of the issues faced during the process of getting this merged was that it wasn't passing a test in the CI pipeline, which wasn't related to an error in my code, but because some repository secrets were not being propagated to the fork of the repository that I was working on, so the maintainer had to give me write access to the repo to get my PR merged
Further Contributions
This PR would now act as the base for further PRs that would eventually solve the entire issue. To be honest, this is one of the best projects I have worked on in a while, it is taking me about 6-7 PRs to just solve one issue, this shows how intricate and developed the project is.
I am really enjoying how my Open-Source Journey is shaping out to be.
Top comments (1)
Hey folks, came across this post and thought it might be helpful for you! Check out this blog on evaluating RAG performance: metrics and benchmarks - Rag Evaluation Metrics