by Manpreet Singh (Director, Software Engineering) and Himanshu Mittal (Lead Software Engineer)
Last year, we started work on a new product dealing with temporal data. The stakeholders included two team members with a focus on pure research and machine learning algorithm evaluation. After a few sprints, we realized that we needed to take a step back and look at the bigger picture before diving into work on the machine learning aspect of the product. We needed to examine:
- Data pipelines availability
- The infrastructure and processing engine
- Storage
- The platforms available
- API capabilities open to us
- The user interface
Not to mention, we needed to address some more fundamental questions: are the requirements accurate? How generic or specific do we want to make the service?
In short, we realized that, while the main driving force for this project was machine learning, there was a lot to do before we could start on the data science piece! We needed more team members with software engineering skills to build out the ecosystem and scale it to meet customer needs. Our project wasn’t going to require an entire team focused solely on machine learning.
Through this project and others, we’ve learned that software engineering and machine learning can’t be treated entirely separately unless you are in pure research and your only goal is to publish new findings or patent ideas. Just like NoSQL systems are also sometimes called "Not only SQL“ when they support SQL-like query languages or sit alongside SQL databases (source), we call our team system NoML, for Not Only Machine Learning.
There are many catchphrases in the world of artificial intelligence (AI). There’s machine learning, deep learning, and data science. And it’s easy to get hung up on job titles: “I am a data scientist” or “I’m an AI Researcher.” With the wealth of sources for learning available these days—whether through college courses, online platforms, or conferences—many people are becoming armchair experts in AI. This can lead to a mindset of, “Serving models in production is not my problem. I’m a scientist.”
At an enterprise company like Salesforce, this type of thinking isn’t helpful! It’s our responsibility to make the models we build applicable to the real world, keeping reliability, scalability, and utility to our customers top of mind.
The next time you’re starting a project with a machine learning focus, keep our “NoML” learnings in mind and see how you can branch out of pure machine learning by making an effort to be a well-rounded technician of the services you build.
Top comments (0)