In 2022, I worked on a data science project for a retailer. The project was to predict cashflows through better demand forecasting and inventory management. This project followed the common pitfalls of all data science and AI projects and made me rethink our strategy and tools.
I’ll tell you all about the mistakes I made ( and will never do again) and what tool I used to save money and time.
Common pitfalls in AI & Data projects
Over the years, I’ve worked on many AI and dastascience projects, delivering substantial ROI through algorithms and AI models. Despite the AI hype, many non-software companies struggle with successful AI strategies, often limited to standard data projects and few impactful AI deployments, resulting in uneven AI adoption.
Here are the main causes:
- Siloed Teams: There’s often a big disconnect between the data scientists and the end-users. There are valid reasons for having different roles and the need for specialization. However, it's important to recognize that in real projects, this leads to a significant gap between data scientists and end-users. Each group tends to use different technology stacks; for instance, data scientists usually work with Python, while IT developers might use JavaScript, Java, Scala, and other languages. This influences teamwork in taking more time and making teamwork tricky.
These are the various groups involved in a typical AI/ DS project:
- Getting acceptance from the end-users / business users: If end-users aren't part of the development process, they might not use the software once it’s up and running.
What I needed:
1- Go All-In on Python: It's easy to learn and works well with other tech. It is at the heart of the AI stack and ideal for integrating with other environments. We considered libraries like Streamlit, which is excellent for prototype quick applications, but we quickly felt the limits of this library for performant multi-user applications.
2- Better Interaction with end-users: It is critical to ensure the software works well for users and track how happy they are with it.
The solution:
Taipy came from the need for a easy tool, like Python, and strong enough for big projects. It handles loads of data fast, can be tailored to specific business needs, and connects data scientists with business users. It also makes decision-making smarter with features that let users play with different scenarios.
Now let's go into detail:
- Answer to the siloed teams
The obvious answers would go towards these points:
- Standardize on a single programming language.
- Provide an easy-to-learn and use programming experience for all skill levels.
- Python is ideal for AI, with many user-friendly libraries, though they often face performance and customization issues.
For instance, libraries like Plotly Dash offer full-code solutions, while Streamlit or Gradio are easier but lack performance and flexibility. Python developers shouldn't have to choose between productivity and performance.
We created Taipy to combine ease of development with high performance and customization.
- Answer to bringing back the end-user to the center of the project
Addressing the two key points is crucial:
Smooth end-user interaction with backend algorithms.
For smooth interaction, end-users need control over algorithm variables through the GUI, the ability to run algorithms with different parameters, and the option to compare results and track KPI performance over time. Taipy addresses this with the 'scenario' concept, storing all data elements and enabling users to track runs, revisit past scenarios, and analyze results.Easy tracking of business-user satisfaction.
For tracking satisfaction, Taipy's scenario function bridges the gap between end-users and data scientists by providing access to all runs and allowing end-users to tag and share scenarios with data scientists. This feature enhances software acceptance beyond basic testing and drift detection.
Taipy - Build Python Data & AI web applications
Support open-source and give Taipy a star: https://github.com/Avaiga/taipy
What about the results?
Implementing Taipy transformed our approach to managing cash flow and demand forecasting. The tool improved our processing capabilities by leaps and bounds and gained quick acceptance from end-users thanks to its intuitive design and relevance to their daily tasks.
We went from using 4 full-time developers to 1.5. The initial team was eclectic, with specialties ranging from Javascript and Java and Python.
The use of Taipy enabled the data scientist using Python to create a full-blown application ready for use by the end-users. This facilitated communication and reduced the siloed team process common to all AI projects. A gain of time and money is crucial to the success of any project.
For the concrete results, Taipy reduced the overall costs by a factor 10!
| Project Phase | Budget | IT Staff | Duration |
|---|---|---|---|
| Initial Setup | $600K | 4 | 8 months |
| With Taipy | $60K | 1.5 | 2 months |
Achievements:
4x Faster Projects: We sped up everything from start to finish and spent less on keeping things running.
10x Cheaper: Most of the tech work was done by Python developers, reducing the need for help from other departments and making project management a breeze.
Check out these applications made with Taipy: https://docs.taipy.io/en/latest/gallery/
Conclusion
Hope you enjoyed a little context on the creation of Taipy and what we to achieve with it.
Top comments (2)
"4x faster and 10x cheaper" - A dream for any company, haha!
That's why we believe in it :)