DEV Community

Chat with your CSV: Visualize Your Data with Langchain and Streamlit

Ngonidzashe Nzenze on May 17, 2023

Large language models (LLMs) have become increasingly powerful and capable. These models can be used for a variety of tasks, including generating t...

Read full post

Gavin S • May 19 '23

You might consider json.loads() instead of eval. Otherwise you might let the model out 😉

Ngonidzashe Nzenze • May 19 '23

Oh thanks for the feedback!🙂 I'll definitely consider using json.loads() instead of eval().

Raghavendra Samant • May 22 '23

Nice article Ngonidzashe !
Just encountered a small issue following this : tabulate , need to install too.

Ngonidzashe Nzenze • May 23 '23

Thanks for reading my article! I'm glad you found it helpful.

You're right, I forgot to add the installation instructions for the tabulate package. You can install it with the following command: pip install tabulate

Once you have installed the tabulate package, you should be able to follow the rest of the instructions in the article without any problems.

Raghavendra Samant • May 23 '23

Right but running into openAI credit limit issues 0 of 18$ . Do you have paid account or does the trail account tokens suffice ?

Ngonidzashe Nzenze • May 24 '23

The tokens provided for your trial account are enough initially, but it appears that you have exhausted them. It would be advisable to think about upgrading to a paid account.

Femi Akinyemi • May 22 '23

Nice and well Written! Well done 👍🏾

Ngonidzashe Nzenze • May 22 '23

Thank you for reading, I'm glad you liked it.

mingjun1120 • May 21 '23

I was thinking of doing something similar to your work. Instead of uploading a CSV file, I want to upload a PDF file. Do you have any idea how to that?

Ngonidzashe Nzenze • May 22 '23

You could make use of the UnstructuredPDFLoader and the load_qa_chain as follows:

from langchain.document_loaders import UnstructuredPDFLoader
from langchain.llms import OpenAI
from langchain.chains.question_answering import load_qa_chain

API_KEY = 'api-key'

loader = UnstructuredPDFLoader("your_document.pdf")
data = loader.load()

chain = load_qa_chain(
    OpenAI(temperature=0.9, openai_api_key=API_KEY), chain_type="stuff"
)

# model response
response = chain.run(input_documents=data, question="<Input your query here>")

You can get more information here

talmoscovitz • May 20 '23

Does the CSV have a size limitation?
Very nice work!

Ngonidzashe Nzenze • May 22 '23

I'm glad you liked the article.

Although the file upload size limit in Streamlit is 200MB, the documentation for create_pandas_dataframe_agent does not explicitly state any size limit. However, it is important to note that larger dataframes will consume more memory.

pdkang • May 20 '23

how to load the csv from a URL address? I couldn't figure it out. If you have some ideas how to handle that, that will be great!

Ngonidzashe Nzenze • May 22 '23

You can load a CSV from a URL just like you would normally load a CSV in pandas:

import pandas as pd

url = "https://example.com/data.csv"
df = pd.read_csv(url)

Hope that helps🙂

Ngonidzashe Nzenze • May 22 '23

Glad you liked it!

Lilin Wang • May 19 '23

Nice tutorial! Wondering if this is doable with the current Javascript support in LangChain? The Javascript support in LangChain is definitely limited

Ngonidzashe Nzenze • May 22 '23

Thank you, I'm happy you liked the tutorial!

While I'm not sure about the full capabilities of the current Javascript support in LangChain, there should be a way to make it happen. It may be helpful to explore LangChain's documentation for more insights.

JmuneraCLQ • Jul 28

Hi Ngonidzashe

Good Article and Excelent app for test chat with csv

I have an error when running the application in streamlit
InvalidRequestError: The model text-davinci-003 has been deprecated

What should I fix and where?

Thank you very much

varun surana • Jul 29

HI, can someone help in replacing text-davinci-003 with other model as there is token limitation also which is causing issue

lokesh • Sep 26 '23

langchain.schema.output_parser.OutputParserException: Could not parse LLM output: Since the observation is not a valid tool, I will use the python_repl_ast tool to extract the required columns from the dataframe.
I am facing this error can anyone help

Jean Bertin • Jul 20 '23 • Edited

hi ! Thanks a lot for the nice tutorial ! :)
I would like to know if it possible to adapt the agent.py script in order to use an "Azure Open AI API Key" instead of "Open AI API Key".
The documentation concerning the basic switch is there but I don´t know precisely how to integrate it on your app :
learn.microsoft.com/en-us/azure/ai...

Thanks a lot for any help

subhadipbhatta • Jul 29 '23

Very nice article and very well described. However there is glitch in my case. When I am trying to analyze, I get the following error: openai.error.APIError: HTTP code 401 from API (Unknown api key). Trust me this is my personal API key and works perfectly elsewhere.
Also please suggest the implementation with azure open AI

openai.api_type = "azure"

openai.api_version = "...."

openai.api_key = "...."

openai.api_base = "....."

Much appreciated

Pritish Chugh • Aug 1 '23

While running this prompt, the agent is getting stuck in a loop. Can you see why is it happening.

Prompt: Create a bar graph for the first 5 rows in the data with Ordernumber on x-axis and quantity ordered on y-axis

Entering new AgentExecutor chain...
Thought: To create a bar graph, we need to specify the columns for the x-axis and y-axis. We also need to limit the data to the first 5 rows.

Action: Create a bar graph using the specified columns and limited data.
Action Input: df.head(5), "ORDERNUMBER", "QUANTITYORDERED"
Observation: Create a bar graph using the specified columns and limited data. is not a valid tool, try another one.
Thought:To create a bar graph, we can use the matplotlib library in Python. Let's import it and try again.

Action: Import the matplotlib library.
Action Input: import matplotlib.pyplot as plt
Observation: Import the matplotlib library. is not a valid tool, try another one.
Thought:To create a bar graph, we can use the matplotlib library in Python. Let's import it and try again.

Mbulelo Lomo • Aug 7 '23

Hi nice article. I have two errors please advise how can I resolve them:

ICreativeKid • Jun 12 '23

Awesome article!

I was thinking of doing something similar for YouTube videos. Any thoughts?

Dil Ahdan • May 24 '23

Thanks for the article. How would you approach this if you didn’t want to use streamlit, and instead wanted a Flask app that gives API for a frontend ?

Devanshu-17 • Jul 11 '23

Hey, @ngonidzashe

As @dilrajahdan asked above, is it possible to run it via flask?

Ed1123 • May 19 '23

Nice article.

Is the code hosted in GitHub? I'd like to fork it. ^^

Ngonidzashe Nzenze • May 19 '23

Glad you liked it. The code is available here

anumber8 • Jul 4 '23

bravo @ngonidzashe this is such a very good article on the AI, I enjoy it and looking forward for more and dirtying my fingers exploring more of it 👏👏👏

phoenix_magicianGirl シェファード • Jul 8 '23

Interesting application of gpt, nice article @ngonidzashe

1gamedevprod • Jul 9 '23

It's a shame we can't create custom tools that the prebuilt agent can use as input argument (need to create an agent from scratch I think)

vinitmodigit • Jul 10 '23

Hi
Nice article very well explained.
If I have a CSV file with 16000 rows and 20 columns, would there be any limitations on using this file?

Raheem Ridwan Lekan • Aug 3 '23

Thanks for the insightful article. I want to know how to use flask as the endpoint instead of the streamlit.

Hamsa • Aug 19 '23

Is there a way to load an XML file and extract data from it?

MG007 • Sep 25 '23

How can I integrate this feature with bubble.io

Gaurav Sharma • Oct 10 '23

in csv file all datatype of columns should be object or it can take any datatype int,float

rsk • Feb 11

I face this error.
What can be the issue? can I pls get a help on this.

Sid Keesara • May 7 • Edited

Im getting the following error

Can someone help me with it

Sunil Kulkarni • Jun 17

Nice Article.
I have use-case where pdf has rules and data is present in csv. How do I combine both pdf and csv and have a chat ?

abhishekkumarjjha • May 25 '23

Nice work. But I having issues with tabulate. installed it already and I can confirm it but it still says "No module named 'tabulate''. Can't we bypass the use of tabulate?