Continuing from: “🚀 My First Step Towards AI/ML Model Integration | Inspire Sphere”
Back then, the ML/AI model I deployed to get categories of the quotes written by users was trained on data using the MultinomialNB algorithm from naive_bayes. The accuracy of my model was not more than 17%, mainly due to several incompatibilities and the absence of supportive parameters in the TFIDF Vectorizer, as well as an inappropriate use of MultinomialNB.
This algorithm works well only when the features (words) in the document are independent of each other, as it calculates probabilities by taking the product of individual probabilities. It's a good fit where text can be classified based on specific independent flagged words, like in detecting spam or fraud in an email.
However, my previous model/system lacked important TfidfVectorizer parameters like:
- stop_words
- ngram_range
- max_df
- min_df
These parameters help in converting the text into a cleaner and more structured numeric form, making it more ready for classification.
What’s New Now?
So, the next algorithm to take the place of MultinomialNB is Logistic Regression from linear_model. Unlike the former, this doesn’t assume that features are independent — it considers relationships between word appearances in the document.
The results were notably better. When my first model categorized this quote on Inspire Sphere as:
Category: Love “A wolf saw me when I was alone… I was that wolf”
Then the improved model categorized it as:
Category: Humour
That was quite a significant and meaningful difference.
Why This Upgrade Mattered
After upgrading my model, I realized that the real value of an AI model also depends on the data processing and how it interacts with real users.
I made the model predict the category of quotes written by users on Inspire Sphere to auto-fill the title input field. This made the platform look more professional and supportive — helping users feel guided and saving time.
Top comments (0)