We are happy to present the launch of our new product we've been working on for over a year.
For 14 years, we've been developing components for data visualization and data analytics and noticed a need for a component that can work fast with enormous datasets. We gathered all our experience and expertise to create new technology that easily handles this task. The new approach responds immediately and smoothly to any user interaction, regardless of file size.
We decided to show our technology in the form of a grid, as it's a straightforward but effective instrument to work with data. We also have ambitious plans to develop this technology further and go beyond the grid functionality, but today we want to tell you about DataTableDev.
DataTableDev - is a grid library for instant big data processing🚀
Diving into the topic deeply, we challenged ourselves with a more complex task - to save users' time.
So our goal was to get end-users to access the data instantly despite the volumes, without time spent on reloading data & redrawing the grid each time the user interacts with it.
How did we get there? 🎉
Considering core stages when working with big data - data loading, preprocessing, and then its visualization on the web - we are developing a unique technology that optimizes all these steps and helps not to overlap them.
As stated in multiple studies on how the human brain perceives information when it comes to web-based applications, there are three primary time limits within which the app should respond to user input:
- 0.1 seconds - people regard it as an immediate action😊
- > 1 second - users can spot an intermediary😐
- > 10 seconds - people start losing attention😔
So our grid should load, show & interact with gigabytes of data in less than 0.1 seconds.
Well, we did it. And that’s how.🎉👇
When the screen is updated and repainted by the browser, everything is encapsulated in a frame.
The frame rate target is 60 frames per second to be perceived as responsive. Plus, the frames need to be equal in length to ensure a steady frame rate. It helps steer clear of jaggy and rigid animations.
We managed to specify the structure of frames and different behaviors in different situations, define each task's execution time and sequence, and use requestIdleCallback() and requestAnimationFrame() to optimize free time in frames.
But in fact, behind all this work is even greater efforts that we have invested in the server part of the technology. Thoughtful and optimized work with data transfer and operations strongly supports the approach developed on the client side and makes it possible to use it on other software as well.
We are still working on our product, but you already can try a demo showing 11 million rows from a 1,6 GB file with primary grid features that also work instantly. As we developed this product for developers, we are open to your feedback or suggestions that will help us to improve our product further.
If you want to stay updated with our latest news, subscribe to our early-bird list.
Give it a try, and let us know what you think.
Top comments (0)