DEV Community

Discussion on: I am a polyglot developer who does web front/back end, data science, native iOS, and a bit of ethical hacking, Ask Me Anything!

Collapse
 
mightyking7 profile image
Buitrago

Have you ever had a job in Data Science ? If so, what was it like ? What are some pros/cons vs Software Engineering.

Collapse
 
rcdilorenzo profile image
Christian Di Lorenzo

I have not although I will be transitioning into that capacity within the next several months (starting already). I'm planning to bring data science to the company I work for now. I've already started on a homegrown product recommendation system for example.

Differences

Since we are in the consulting business, one of the most important things is building these systems in what I call a "value-driven algorithm development." It's easy from my master's work to want to quickly jump into the most complicated algorithm. However, there are many times that you can actually provide quite a bit of value with a super-simple example. Start with something manual (or manually assigned tags/category/groups) and then build up from there.

The other portion that's really different is how you think about data collection. In this case, we actually need to track relevant information for performing later analysis. This is a bit of departure from traditional software development since we aren't specifically creating data structures to directly fulfill a customer feature. Instead, it requires a more holistic approach and think through future scenarios since you can never go back and use data you never collected.

Pros and Cons

Obviously, I'm a little biased towards data science since I have been working on a masters. I personally feel that data scientists would in an ideal scenario have a good background in software engineering. However, let's consider them separately.

Software Engineer (no data science)

Software guys can build almost any product from scratch (assuming enough experience, budget, and business goals). These engineers can break down a complicated problem into the relevant architecture and pull in others to get things done. When it comes to statistics or this machine learning algorithms, the software engineer can still have good resources (e.g. fast.ai) but won't be able to implement these "smart features" that require a different type of thinking

Data Scientist (no formal experience in software)

The data scientist can work across a broad range of industries from software products to research labs. They can write horribly procedural code that does things that most software engineers would have glazed eyes over. However, they do know decent optimization techniques where large data sets are concerned. They typically have a good foundation in statistics and are closer to the executive level of business and customers. Viewing the world through data-tinted glasses is their gig.

The "unicorn"

As some call it, a data science unicorn is someone who combines business expertise, data science, and computer science in a single individual. This is my personal goal as it allows building the entire data science pipeline from raw data to a scaled ML algorithm that integrates with the product. I've tried to keep this in mind during my school work such as this project: github.com/rcdilorenzo/abfs.

Conclusion

I'm sure that's probably a lot more than what you were asking. Hopefully, some of that was valuable. In short, I love everything in-and-around software engineering, but I also want to push the envelope and use data to help people accomplish everyday tasks.

In fact, as a Christian, doing data science and realizing how much of this applied mathematics is human-driven and confined to a specific application gives me a new appreciation for even the smallest biological systems God has designed. My jaw drops at the sheer complexity and natural intelligence even in a rat. Yes--these systems seem "intelligent" and "smart," but they truly don't compare.

On the flip side, creating a system that indirectly makes decisions is really neat. It's freeing to know that we are taking good stewardship of the tools we have been given. Go forth and create!