DEV Community

Florian Bernard
Florian Bernard

Posted on

2

Kotlin DataFrame ❤️ Arrow

Kotlin DataFrame v0.14 comes with improvements for reading Apache Arrow format, especially loading a DataFrame from any ArrowReader.
This improvement can be used to easily load results from analytical databases (such as DuckDB, ClickHouse) directly into Kotlin DataFrame.

Here are two examples of integrations that allow for smooth data import into Kotlin DataFrames using Apache Arrow.

DuckDB

DuckDB is an Analytics database that can be embedded for use in a Kotlin notebook. DuckDB facilitates reading query results as an Arrow Stream, enabling straightforward loading into a Kotlin DataFrame.

Here is a basic notebook which uses DuckDB for querying data from a remote parquet file and then importing the results into a Kotlin dataFrame using Arrow Stream

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

ClickHouse

ClickHouse is a high-performance, column-oriented SQL database management system (DBMS) designed for online analytical processing (OLAP).
ClickHouse allows using Arrow Stream as an output format.

The next notebook uses the ClickHouse client for querying data in Arrow stream format and loads it in a kotlin dataFrame.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Conclusion

Loading Arrow data into Kotlin DataFrame is both straightforward and efficient, harnessing the full power of Kotlin for data analysis. This integration not only simplifies the process but also enhances performance, making Kotlin DataFrame a powerful tool for handling and analyzing large datasets.

Sentry mobile image

Mobile Vitals: A first step to Faster Apps

Slow startup times, UI hangs, and frozen frames frustrate users—but they’re also fixable. Mobile Vitals help you measure and understand these performance issues so you can optimize your app’s speed and responsiveness. Learn how to use them to reduce friction and improve user experience.

Read the guide

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs