Machine Learning for Web devs

When I started my career as a Software Engineer, I felt comfortable in the realm of web development. Whenever I heard about someone working in AI or ML, I couldn't help but admire them. I assumed these people possessed a deep understanding of mathematics and statistics that enabled them to do such impressive work. However, with the rise of ChatGPT and its incredible capabilities, I decided to dig deeper into the topic. What I discovered was eye-opening: you don't need to be a genius to work with ML or AI!

Like any discipline, there are multiple sub-domains within ML and AI that require different skill sets. As an engineer and developer, I approach this field from the perspective of how to integrate it into modern web applications. I recently passed the Azure AI-900 certification, and I realized that my fellow developers could benefit from this knowledge too. So, let's explore what ML is all about!

What is Machine Learning?

At its core, Machine Learning (ML) is a subset of artificial intelligence that focuses on creating algorithms and models that can learn from data. Unlike traditional programming where we explicitly define rules, ML algorithms learn patterns and relationships from data to make predictions or decisions.

To put it in mathematical terms, ML is about learning a function f(x) = y, where:

• x represents the input data (features)
• y represents the output (predictions or labels)
• f is the learned function or model

The key difference from traditional programming is that the function f is not explicitly coded but learned from data during a process called training.

Key Terminology

Before we dive deeper, let's familiarize ourselves with some essential terms:

1. Features: These are the observed attributes or input data. For example, if we're predicting house prices, features might include the size of the house, its location, and the number of bedrooms.

2. Labels: These are the outcomes that the model tries to predict. In our house price example, the label would be the actual price of the house.

Types of Machine Learning

There are several types of machine learning, but we'll focus on the two main categories:

1. Supervised Learning

   • Uses training data that includes both feature values and known label values
   • Format: [X1, X2, X3], Y
   • Used when you have historical data with known outcomes
   • Example: Predicting house prices based on features like location, size, etc.

2. Unsupervised Learning

   • Uses training data that includes only feature values, without labels
   • Format: [X1, X2, X3]
   • Used to find patterns or relationships in data without predefined outcomes
   • Example: Grouping similar customers based on demographics and behavior

The Training Process

Now that we understand the basics, let's look at how ML models are actually trained

1. Data Collection: Gather a large dataset relevant to your problem.

2. Data Preprocessing: Clean and prepare your data, handling missing values and outliers.

3. Split the Data: Divide your dataset into:

   • Training set: Used to teach the model (typically 70-80% of the data)
   • Validation set: Used to tune the model and prevent overfitting (typically 10-15%)
   • Test set: Used to evaluate the final model performance (typically 10-15%)

4. Choose an Algorithm: Select an ML algorithm appropriate for your problem. The beauty is that these algorithms are often interchangeable, allowing you to experiment with different approaches.

5. Train the Model: Feed the training data into your chosen algorithm, allowing it to learn the patterns in the data.

6. Validate and Tune: Use the validation set to fine-tune your model's parameters and prevent overfitting.

7. Test and Evaluate: Finally, use the test set to assess how well your model generalizes to new, unseen data.

Want to Learn More?

I'm excited to announce that I'm creating a free course on Azure AI-900! If you're interested in diving deeper into AI and ML, particularly within the Azure ecosystem, this course is perfect for you.

Sign up here to get notified when the course launches!

Thanks for reading, and stay tuned for the next post where we'll dive into computer vision capabilities. The world of AI and ML is vast and exciting, and I can't wait to explore it further with you!

Comments

[migduroli]

I recommend to have a look at flama, an open-source project which is specifically thought for the productionalisation of ML models via ML APIs. To have a look at an actual example of an entire ML pipeline run with flama, you can check this post, which I think contains all the relevant information.

[Vortico]

Hey, great post! We really enjoyed it. You might be interested in knowing how to productionalise ML models in ridiculously easy way. If so, please have a look at flama for Python. We introduced some time ago an introductory post here Introducing Flama for Robust ML APIs. As you'll see, all the steps you describe in this post can be found in our post, and eventually end up with a model running locally and ready to receive requests to make predictions. If you have any doubts, or you'd like to learn more about it and how it works in more detail, don't hesitate to give us a shout. And if you like it, please gift us a star ⭐ here.

[vikas nautiyal]

Clean & descriptive for beginners to get an overview. Hope to see a more such articles.

[Capgo]

Cool, thanks!