❔ About
🤔 Sometimes you face lack of data or data quality issues that prevent you from producing insights.
💡 Whatif you could call AI to the rescue to fix/enhance some data
I first started some Prompt engineering on chatGPT:
☝️ Notice
Notice that guessing gender on firstnames can seem useless or a bit dumb (or nerdy). Yes,but...
- 🗺️ This work relies on openAI... which acts as a universal language firstname parser
- 💡 This work is just an illustration of how prompt engineering and OpenAPI'API can help review/fix any kind of data quality issues... and makes a concrete illustration on how you may enrich your enterprise data pipeline
🎯 Target
The purpose of this article is to see how openai's API can help on a very specific testable dataset.
📝 Kaggle Notebook
This short notebook I will:
- 📥 Download data
- 🐼 Load data in
pandas - 🦾 Call
openai's API to guess firstname's gender - ⚖️ Compare guessed vs. real data
🍿 Demo
🗃️ Input Dataset
I have used the top-10-prenoms-a-noumea-depuis-1860 open dataset from data.gouv.nc:
Top 10 des Prénoms à Nouméa depuis 1860 — Open Data NC
Ce jeu de données présente la liste des dix prénoms les plus donnés à Nouméa, depuis 1860, d'après le registre de l'état civil. Fréquence de mise à jour : Annuelle
data.gouv.nc
🤖 The text-davinci-003 model
I have used text-davinci-003 from GPT-3.5 models as they can:
"understand and generate natural language or code."
📊 Results 👏
☝️ Notice
Notice that I have put the guessed value in a dedicated structure... so we can easily flag it as AI generated when reporting its metadatas:
MIT Sloan School of Management@mitsloan
Ethical AI isn’t just good for humans — it’s good for business. mitsloan.co/3FKsj0619:10 PM - 23 Mar 2023
💰 Gains
- 📈 Data quality
- 💡 Better decisions & opportunities
- 💸 Puts the cost of the lack of data quality in evidence (API calls are not free)
- 🧠 Create more intelligence
👨🔬 Further optimizations
- Benchmark models to spend as less money as possible while getting the best results as possible
🔭 News & perpsectives
OpenAI@openaiChatGPT and Whisper are now available through our API (plus developer policy updates). We ❤️ developers: openai.com/blog/introduci…18:04 PM - 01 Mar 2023
VentureBeat@venturebeatOpenAI today announced its support of new third-party plugins for ChatGPT, and it already has Twitter buzzing about the company's potential platform play. venturebeat.com/ai/openai-turn…18:29 PM - 23 Mar 2023
TechCrunch@techcrunchGet all the latest details on OpenAI connecting ChatGPT to the internet here 👇 techcrunch.com/2023/03/23/ope…18:58 PM - 23 Mar 2023
Top comments (9)
Making the most of AI: The latest lessons from MIT Sloan Management Review | MIT Sloan
Knowing how to evaluate AI tools, manage data effectively, and share data strategically will help leaders see the results from their AI investments.