“If you can apply practical machine learning skills to industry projects, companies aren’t going to require you to understand ML theory inside and out.” Allen Lu, CTO & Co-founder of AdaptiLab, recently ML engineer at Google and Microsoft
“Machine learning is something every developer at every company needs to know about, right now." Kevin Scott, CTO, Microsoft.
Here’s what will be covered today:
- Why machine learning
- Misconceptions about machine learning
- Practical vs Theory
- Skills a software engineer needs to become a ML engineer
- How software engineers can get started in their ML career
Why machine learning?
Artificial Intelligence (AI) and Machine Learning (ML) are two of the hottest fields in technology right now, with 71% of GeekWire Cloud Summit tech leaders saying it’s the most important technology over the next two to three years. From healthcare and agriculture to manufacturing, energy, and retail, many companies across industries are leveraging these technologies to get ahead. In fact, the Future of Jobs survey found that 73% of companies are likely to adopt machine learning by 2022. So, if you’re in search of the most in-demand and most-exciting career domains, gearing up yourself with machine learning skills is a good move now.
More career opportunities
Virtually every industry will be looking to apply machine learning in their domain, ultimately opening up a world of opportunities for engineers to develop impactful machine learning applications. Machine learning is also very closely related to data science, meaning that if you pursue a career in machine learning then you’ll also have the necessary skills to become a data scientist.
Employers love when you shows signs of proficiency in both because you have the foundation to analyze data, create meaning from it, and later make use of that information to train a machine learning model to predict results.
Jobs on the rise
According to job site Indeed, machine learning positions have seen massive growth over the last year, up 344% from 2018. Employers are on the lookout, but the talent lacks.
Lack of ML talent
This comes down to the lack of quantity and quality of candidates.
Quantity: The sourcing of candidates is incredibly difficult because the field and its applications are so nascent. Finding someone with the right experience to do a particular job is like finding a needle in a haystack.
Quality: It’s no surprise that if you have the skills to become a machine learning engineer that you’ll be picked up pretty quickly by a major company. With big-name companies swooping most of the talent, it stalls many other companies' initiatives to build their data teams and usher themselves into the data-driven era.
There is a huge opportunity for software engineers to close this gap and learn the skills to become a machine learning engineer. In fact, many software engineers already have the foundation required to learn machine learning, they just don’t know it yet. Here’s how software engineers can get their start in machine learning.
Misconceptions about machine learning
Machine learning is being used to solve a myriad of difficult problems across many industries and verticals. However, there is a common misconception among the general population that ML and AI are incredibly difficult and advanced fields, requiring robust experience in advanced statistics and beyond.
However, there are two major misconceptions that steer individuals away from machine learning:
You don’t need to learn the incredibly complex theory behind all the ML models. In the industry, 90% of projects don’t require you to build complex models. Those projects are actually data engineering, data analysis, and analyzing insights from models rather than the process of actually building models.
Companies only want to hire machine learning engineers that have academic training in machine learning. Because of the vast opportunity machine learning brings to the table, companies are going to be looking to hire engineers that have experience in software and data engineering. Meaning that as long as you can apply machine learning concepts to projects, companies aren’t going to require you to understand ML theory inside and out. Ultimately, developers who can apply those fundamentals to new applications and spaces are going to win in the end.
Machine learning in the tech industry is pretty different from how people perceive it. Most people view it as an impenetrable field, where only a select few with years of research experience can hope to enter. However, with the rise of popularity in the field and new frameworks to make coding models a piece of cake, there is more opportunity than ever to work on machine learning in industry. With the necessary background, most people could probably do most industry-level machine learning tasks with just a few months of appropriate training.
Learn the practical skills OR learn the theory? Which should you choose to kickstart your career faster?
The trend is abundantly clear: a majority of computer science students want to become engineers and work in the industry, not pursue research or theoretical work in graduate school or beyond.
Therein lies the problem; most university and MOOC courses focus on theory rather than practical applications. Students are expected to have strong math foundations and are often tested on proofs or concepts rather than their ability to code models and apply them to real-world datasets. Even coding assignments tend to reduce practical coding value by providing curated datasets, narrowing the scope of the problem of implementing certain functions or failing to consider model tuning and performance.
Considering data preprocessing, feature engineering, and efficient model deployment make up a majority of a machine learning engineer’s time, this is a painful oversight when it comes to preparing students for industry machine learning.
Consequently, software engineers lack the opportunity to easily gain valuable machine learning skills needed to thrive in the data-driven era. A lot of developers already have the foundation required to learn machine learning, but advanced academic programs and courses don’t focus on the practical skills required for the majority of machine learning projects in the industry.
Practical concepts software engineers need to become a machine learning engineer
Instead of theoretical topics and formulas, what you really should focus on are the main concepts in machine learning. An ML engineer’s job is largely centered around building models for specific tasks using frameworks like TensorFlow. You don’t need to know the calculus behind backpropagation and gradient descent; you can just call an API function. Instead, you need to know things like which models to use for certain tasks, how they might improve training speed or model accuracy, and above all else, how to code machine learning models.
Developers should know how to utilize industry-standard python frameworks — NumPy, pandas, scikit-learn, and Tensorflow to manipulate and process data and train models.
- Data manipulation with NumPy
- Data analysis with Pandas
- Data preprocessing / modeling / clustering with scikit-learn
- Deep learning with TensorFlow and Keras
Even some of the most complicated and innovative ML models could be built relatively easily using these frameworks. You can get started here.
Additional concepts to explore:
- Learn how to code machine learning models for tasks like classification or regression.
- Identify problems with certain machine learning approaches and how’d you fix them.
Along with focusing on general machine learning concepts, you’ll want to identify what industry you want to work in, what field you want to explore (i.e. face recognition, speech, etc.) and then really understand the job’s domain. For example, if you want to be a self-driving car engineer, you’ll definitely need to focus on image recognition and convolutional neural networks. On the other hand, if you’re looking at a text analysis position, you’ll want to know concepts related to natural language processing.
How software developers can get their start in ML
The good news is that if you’re a software developer, then you probably already have the foundation to pick it up, and faster than you may think. Here are a few steps developers should take if they want to get into machine learning:
Have a firm grasp on data science
As a developer, if you have an interest in expanding your skills into machine learning, a large part of your role will require a foundation in data science. You should be able to analyze data, create meaning from it, and later make use of that information to train a machine learning model to predict results.
Understand the frameworks available to you and what they’re used for
Software developers should become comfortable with the frameworks that are available to them like NumPy, scikit-learn, Pandas, and TensorFlow. You should also be aware that some of these frameworks are open source, so you can leverage community-driven projects created by major tech giants (Google, Facebook, etc.).
Find an industry or focus you enjoy
What interests you? Find out what you want your focus to be (i.e. image/speech recognition, etc.) and then find an industry that you really enjoy. For example, if your focus is on image recognition and convolutional neural networks, then you may find that you’d like to work with self-driving cars.
Practical skills / concepts
Machine learning theory is important, but if you really want to kickstart your journey to become a machine learning engineer, then it’s best to focus on the practical concepts and skills that you’ll be using on a daily basis.
To get hands-on experience with the frameworks and concepts mentioned throughout this post, then head on over to “Machine Learning for Software Engineers". This course was created by AdaptiLab co-founders Allen Lu and James Wu who have extensive machine learning experience at companies like Google and Microsoft and know what it takes to get started down this career path.
Happy learning!
Further readings
Course Track: Become a Machine Learning Engineer
Course: Grokking Data Science
Article: Machine learning 101 & data science: Tips from an industry expert
Article: How and why to become a machine learning engineer
Article: My experience working with ML at Google and Microsoft
Top comments (5)
Hi Amanda,
Do you feel that R is a big player in M.L.? What percent of the companies that you know work with Python? What percent use R? What ML frameworks do you use the most?
R frequently makes the list for popular ML languages, though Python is usually #1, since it was one of the first languages to support ML through frameworks and libraries. About 57% of ML developers use Python, and R is at about 31%.
While R is popular, it's not necessarily prioritized and is used alongside other languages. It's still popular because it's able to handle functional and statistical algorithms well, and it can interface to Tensorflow.
Popular Python frameworks are Scikit, NumPy Tensorflow, and Pybrain. For R, there are some useful ML packages, like RODBC, Class, and Tm.
So do many companies advertise for R? Or is it just academia that is really hot on R. R seems to have a rrrabid following but I don't see many ads for it. Do you agree?
I am not certain if they advertise for R, but I do know that Facebook uses R for data analysis. Twitter uses it for data visualization. AirBnb has an internal package called Rbnb for data science. Microsoft Azure uses R for data visualization, and John Deere uses R for forecasting projects. I agree that it is not commonly advertised, and many of the sites fail to list R in their preferred programming languages.
Very interesting, thanks for your input.