DEV Community

Cover image for From Data Gurus to Deployment Heroes: Exploring the Diverse Experts Behind the Scenes
Bala Madhusoodhanan
Bala Madhusoodhanan

Posted on

From Data Gurus to Deployment Heroes: Exploring the Diverse Experts Behind the Scenes

Intro:
Machine learning projects are complex endeavors that require a diverse team of professionals, each with their unique expertise and skills. From the data gurus who meticulously analyze and prepare the datasets to the deployment heroes who ensure the models run smoothly in production, every individual plays a vital role in the success of the project. In this blog, we will delve into the different personas involved at various stages of machine learning projects and highlight the significance of their contributions. Let's explore the essential team behind machine learning and how their collective efforts shape the future of AI-driven solutions

Lets explore a bit more on the key themes each persona would contribute to the various stages in a machine learning project

Inspiration / Problem Identification:

Questions to be answered Personas Involved
•Identify the usecase / decision making for which AI approach needs to be done •Quantify the business value •Define the acceptance criteria / performance index and validated against loss •Validate the use case against the ethical principles Product Owner / Product Manager & Data Analyst
•Identify the availability of data scope •Do we have the necessary Hardware / platform available •Formulate ML problem Data Analyst, ML Engineers & Data Engineers
•Define the AI system architecture •Enabling platform for the AI system •Data Governance & Security •Integration •Validate the design for Security , Risk, DevOps Architecture Working Group (Various Stake Holders from Platform Architect to Compliance and Risk SME)

EDA and Data Engineering:

Questions to be answered Personas Involved
Data Sourcing:

1. Identify the data objects and data source

2. Feature engineering Approach

3. Data privacy and data masking
Product Owner, Data Analyst & Data Engineer
Data Engineering:

1) Data profiling

2) Data Cleansing

3) Split Data for Training and Test
Data Engineer
EDA (Exploratory data analysis):

1) Data Insight

2) Detect outliers and anomalies

3) Validate the hypothesis of the business case

4) Leverage the EDA for Feature Engineering
Data Analyst

Feature Engineering & ML Models:

Questions to be answered Personas Involved
Tool Identification:

1) Decision on packages / library need
2) UX design
3) Validate the architecture and identify any new Platform / capability needed
Data Scientist & ML Engineer
Model Design:

1) Cross validation data split
2) Leverage training dataset to find patterns
3) Evaluate performance on the rest of your training data
Data Scientist & ML Engineer
Model Evaluation:

1) Evaluate the model with the test data set
2) Fine tune the model based on performance
3) Review the explainability of the model
Data Scientist & Product Owner

Operationalise:

Questions to be answered Personas Involved
Build for Prod:
1) MLOps Setup
2) Environment setup and CI/CD
3) Logging, alerting
4) DR
Data Engineer & ML Engineer
Launch:
1) Evaluation
Product Owner & Data Scientist
Maintain:
1) Monitor
2) Maintenance of plan
ML Engineer

Closing Remark:
The world of machine learning thrives on collaboration and the collective brilliance of diverse individuals. Each persona involved brings their specialized knowledge and experience to the table, working towards a common goal of building intelligent and impactful solutions. As we conclude this exploration of the essential team behind machine learning, it's important to recognize and appreciate the efforts of every individual involved. By fostering collaboration, respecting expertise, and leveraging the strengths of each persona, we can unlock the true potential of machine learning and pave the way for innovative advancements. Together, let's continue to empower the team, embrace diversity, and drive remarkable achievements in the world of AI.

Reference:
1) Deliverables with ML Project

2) Maximizing the Value of Your Machine Learning

3) Tech Junction - Data Professionals

Top comments (0)