Data Scientists|Engineers: What are the Frameworks you use the most at your job?

strikingloo profile image Luciano Strika ・1 min read

I've seen a lot of statistics about programmers, but not specifically about Data Scientists or Engineers.
Because of that, I'd like to propose this survey:

  1. Do you identify as a Data Scientist or Data Engineer?
  2. What are the languages and frameworks you use the most at your job? Say, the top 5 or 6.

As a bonus:

  1. What are the Frameworks you wish you were using instead?

I will write back with some visuals or analysis if this gets enough traction.
Of course all the data will be public here, so you can do your own too.


markdown guide

As a Machine Learning Engineer (I'm responsible for creating and deploying models) I use mostly Python with Flask and Tensorflow on Google Cloud Platform. This workflow has had some success in the past few projects that I've participated.


Do you feel GCP is more seamlessly integrated to TensorFlow than PyTorch? Or is it mostly agnostic? (I'm asking from a pretty ignorant standpoint)


I'm not really experienced with PyTorch in production, but on my experience, Tensorflow feels natural on GCP. They have all kinds of services and support for it. For what I have seen, they have started to work on a framework agnostic machine learning hub. I've not experimented with PyTorch yet, but feels like I'll be doing it in a couple of months.


I am finishing by Masters in Bioinformatics and my department basically relies on R/Rstudio and Python and Docker. Quite a few people in the field also use Bash scripting quite a bit too, with a very little Perl too.


Awesome! It's interesting to see the perspective from someone who doesn't come from a computers/software background.
I also didn't think Bash was that widespread.

  1. A bit of both, although coming from a software eng background I tend to do more data eng than science.
  2. Python, Spark with PySpark, Databricks

Cool, that's a very data engineering like profile. What's Databricks used for?