Machine learning projects should not be managed as software development projects. Although both use code, ML projects are prone to a higher rate of change throughout their lifetime. This happens because of their close connection to the environment.
In crude terms, one might think about a software development project as building a finished product such as a plane or a car. On the other hand, the ML project is similar to operating a farm. You need to have all the necessary tools to manage a farm, but you also need to respond to the changes in the environment such as weather, nutrients in the soil, possibility of being invaded by parasites or your crop getting sick etc.
A farm needs much more attention since it's embedded into its surrounding environment much closer than the car into the road. In the same way, a farm is connected to its surrounding; ML models are connected to their data. If models are not managed or taken care of, then they would not yield good predictions. Similarly, a farm would not produce a good crop if its environment is neglected.
Instead of using software development processes, it's better to focus on the software development principles of version control, testing, code review and design principles. Then we should re-arrange them into workflows that would better fit with the nature of ML projects.
Top comments (0)