DEV Community

Alexander Chichigin
Alexander Chichigin

Posted on • Edited on

1 2

What's wrong with computational notebooks?

Maybe we still don't have a definitive answer, but at least we have an observational study. The link points to a summary page that have a link to the paper. But I'll summarize the summary below. 😄

Main pain-points

After observing 5 Data Scientists at work and interviewing 15 more the authors identified 9 major pain points with computational notebooks. But I'll list only a subset:

  1. Setup of the notebook, libraries and data sets.
  2. Exploration and visualization.
  3. Writing code.
  4. Version management.
  5. Sharing and collaboration.
  6. Reproduction and reuse.
  7. Production deployment.

If you're a versed Data Scientist you're probably already either giggling or facepalming. Because the list above basically enumerates all the steps in your usual workflow and we have issues with every single one of them!

I now have a question "what's right with computational notebooks?" 😄

And yet the notebooks are indispensable for Data Analysis and present huge improvement upon traditional purely code-centric development. Still we have a lot to borrow from "the old ways" in particular advanced IDE features and integration with Version Control and Continuous Integration Systems.

The paper authors say there's a huge demand for new advanced tools integrating both IDE features and Notebook features. And I have an impression JetBrains are evaluating such an opportunity and maybe even already designing or developing a Data Science tailored IDE with Notebook features and support for R language and major libraries. 😊

UPDATE. Apparently there's already an R language plugin for JetBrains IDEs that JetBrains now officially support and improve. And it indeed brings some Notebook features into a full-featured IDE experience. Yet still I think they might go for a separate Data Science-oriented IDE product.

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up

Top comments (0)

Billboard image

Try REST API Generation for Snowflake

DevOps for Private APIs. Automate the building, securing, and documenting of internal/private REST APIs with built-in enterprise security on bare-metal, VMs, or containers.

  • Auto-generated live APIs mapped from Snowflake database schema
  • Interactive Swagger API documentation
  • Scripting engine to customize your API
  • Built-in role-based access control

Learn more

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay