<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Florian Bernard</title>
    <description>The latest articles on DEV Community by Florian Bernard (@fb64).</description>
    <link>https://dev.to/fb64</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F341147%2F133608cc-440d-47dc-9738-add7a162ed7e.png</url>
      <title>DEV Community: Florian Bernard</title>
      <link>https://dev.to/fb64</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/fb64"/>
    <language>en</language>
    <item>
      <title>Java + Rust with FFM</title>
      <dc:creator>Florian Bernard</dc:creator>
      <pubDate>Fri, 19 Dec 2025 17:17:47 +0000</pubDate>
      <link>https://dev.to/fb64/java-rust-with-ffm-c6k</link>
      <guid>https://dev.to/fb64/java-rust-with-ffm-c6k</guid>
      <description>&lt;p&gt;The Java &lt;strong&gt;F&lt;/strong&gt;oreign &lt;strong&gt;F&lt;/strong&gt;unction and &lt;strong&gt;M&lt;/strong&gt;emory API simplifies the secure execution of native code and memory outside the JVM's management. This article will demonstrate how to effortlessly invoke a Rust native function and facilitate memory sharing between Java and Rust.&lt;/p&gt;

&lt;h2&gt;
  
  
  Requirements
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;☕ Java 25 SDK: &lt;a href="https://adoptium.net/" rel="noopener noreferrer"&gt;https://adoptium.net/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;🦀 Rust : &lt;a href="https://rust-lang.org/tools/install/" rel="noopener noreferrer"&gt;https://rust-lang.org/tools/install/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;⏱️ 10-15 minutes&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  FFM API
&lt;/h2&gt;

&lt;p&gt;The Java &lt;a href="https://docs.oracle.com/en/java/javase/25/core/foreign-function-and-memory-api.html" rel="noopener noreferrer"&gt;Foreign Function &amp;amp; Memory API&lt;/a&gt; (FFM API), introduced as part of Project Panama, is a modern Java API designed to enable seamless interoperability between Java and native code. It allows Java applications to efficiently call native functions (written in languages like C, C++, or Rust) and safely access native memory, without the overhead and complexity of the traditional &lt;a href="https://docs.oracle.com/en/java/javase/25/docs/specs/jni/intro.html" rel="noopener noreferrer"&gt;Java Native Interface&lt;/a&gt; (JNI).&lt;/p&gt;

&lt;p&gt;This API comes with two main components:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Foreign Function Interface&lt;/strong&gt; (FFI) API: This allows Java code to invoke native functions directly, using a more intuitive and type-safe approach than JNI&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Memory Access&lt;/strong&gt; API: This provides controlled access to off-heap memory, enabling Java programs to interact with native memory layouts (e.g., structs, unions, and pointers) in a structured and safe manner.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  👋🦀 Hello Rust
&lt;/h2&gt;

&lt;p&gt;The FFM Api rely on the native linker to dynamically load and bind native library ensuring that function calls are resolved efficiently. &lt;br&gt;
To do so we will need to build a dynamic library in rust.&lt;/p&gt;

&lt;p&gt;Let's create a new rust project:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cargo init &lt;span class="nt"&gt;--lib&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Configure the project to make a dynamic library (&lt;code&gt;Cargo.toml&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="nn"&gt;[package]&lt;/span&gt;
&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"rust_lib"&lt;/span&gt;
&lt;span class="py"&gt;version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"0.1.0"&lt;/span&gt;
&lt;span class="py"&gt;edition&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"2024"&lt;/span&gt;

&lt;span class="nn"&gt;[lib]&lt;/span&gt;
&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"rust_lib"&lt;/span&gt;
&lt;span class="py"&gt;crate-type&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"cdylib"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

&lt;span class="nn"&gt;[dependencies]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then we need to write a C-compatible native function in order to be call with the Java FFM api (&lt;code&gt;src/lib.rs&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="nd"&gt;#[unsafe(no_mangle)]&lt;/span&gt; &lt;span class="c1"&gt;//preserve the function name&lt;/span&gt;
&lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;extern&lt;/span&gt; &lt;span class="s"&gt;"C"&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;hello_rust&lt;/span&gt;&lt;span class="p"&gt;(){&lt;/span&gt;
    &lt;span class="nd"&gt;println!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"🦀👋 Hello from native rust lib 🦀"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Build the library:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cargo build &lt;span class="nt"&gt;--release&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The library is built in the &lt;code&gt;./target/release&lt;/code&gt; folder with a different extension according to the operating system (Linux, MacOS, Windows) &lt;code&gt;librust_lib.so|dylib|dll&lt;/code&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  ☕ Java App
&lt;/h2&gt;

&lt;p&gt;Let's create a basic main to call ou native Rust function:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight java"&gt;&lt;code&gt;&lt;span class="kn"&gt;package&lt;/span&gt; &lt;span class="nn"&gt;fr.flob.ffm&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt;

&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;java.lang.foreign.FunctionDescriptor&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;java.lang.foreign.Linker&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;java.lang.foreign.SymbolLookup&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;java.lang.invoke.MethodHandle&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;public&lt;/span&gt; &lt;span class="kd"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;App&lt;/span&gt; &lt;span class="o"&gt;{&lt;/span&gt;

    &lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="kt"&gt;void&lt;/span&gt; &lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="o"&gt;()&lt;/span&gt; &lt;span class="kd"&gt;throws&lt;/span&gt; &lt;span class="nc"&gt;Throwable&lt;/span&gt; &lt;span class="o"&gt;{&lt;/span&gt;
        &lt;span class="c1"&gt;// Load rust dynamic library&lt;/span&gt;
        &lt;span class="nc"&gt;System&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;loadLibrary&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"rust_lib"&lt;/span&gt;&lt;span class="o"&gt;);&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;linker&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Linker&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;nativeLinker&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;lookup&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;SymbolLookup&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;loaderLookup&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;helloFunction&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;lookup&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;find&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"hello_rust"&lt;/span&gt;&lt;span class="o"&gt;).&lt;/span&gt;&lt;span class="na"&gt;orElseThrow&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;

        &lt;span class="c1"&gt;// Function does not take any arguments and returns nothing&lt;/span&gt;
        &lt;span class="nc"&gt;FunctionDescriptor&lt;/span&gt; &lt;span class="n"&gt;helloDescriptor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FunctionDescriptor&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;ofVoid&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;

        &lt;span class="c1"&gt;// Create the method handle to call the rust function&lt;/span&gt;
        &lt;span class="nc"&gt;MethodHandle&lt;/span&gt; &lt;span class="n"&gt;helloHandle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;linker&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;downcallHandle&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;helloFunction&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;helloDescriptor&lt;/span&gt;
        &lt;span class="o"&gt;);&lt;/span&gt;

        &lt;span class="c1"&gt;// Invoke the native function&lt;/span&gt;
        &lt;span class="n"&gt;helloHandle&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;invoke&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;
    &lt;span class="o"&gt;}&lt;/span&gt;

&lt;span class="o"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Once compiled with your favorite tool (maven, Gradle, Ant, javac ....) run the App main method:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;java &lt;span class="nt"&gt;-Djava&lt;/span&gt;.library.path&lt;span class="o"&gt;=&lt;/span&gt;&amp;lt;RUST_PROJECT_PATH&amp;gt;/target/release &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--enable-native-access&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;ALL-UNNAMED &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;-cp&lt;/span&gt; &amp;lt;CLASS_PATH&amp;gt; &lt;span class="se"&gt;\&lt;/span&gt;
fr.flob.ffm.App
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Tada 🎉&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;🦀👋 Hello from native rust lib 🦀
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  💾 Shared memory
&lt;/h2&gt;

&lt;p&gt;Now we know how to call our native Rust library, we can leverage on the Memory Access API to share some data between Java and Rust&lt;/p&gt;

&lt;p&gt;Let's add a rust method that takes a pointer to a memory address and the number of elements:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="nd"&gt;#[unsafe(no_mangle)]&lt;/span&gt;
&lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;extern&lt;/span&gt; &lt;span class="s"&gt;"C"&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;read_data_rust&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;java_ptr&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="nb"&gt;i32&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;i32&lt;/span&gt;&lt;span class="p"&gt;){&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;java_ptr&lt;/span&gt;&lt;span class="nf"&gt;.is_null&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;||&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;slice&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;unsafe&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nn"&gt;std&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;from_raw_parts&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;java_ptr&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nb"&gt;usize&lt;/span&gt;&lt;span class="p"&gt;)};&lt;/span&gt;
    &lt;span class="nd"&gt;println!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"🦀[rust] data received from Java 🔎 📦"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;slice&lt;/span&gt;&lt;span class="nf"&gt;.iter&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="nf"&gt;.enumerate&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nd"&gt;println!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\t&lt;/span&gt;&lt;span class="s"&gt;[{}] = {}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Update the Java main method to prepare Memory and call this new native function:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight java"&gt;&lt;code&gt;&lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="kt"&gt;void&lt;/span&gt; &lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="o"&gt;()&lt;/span&gt; &lt;span class="kd"&gt;throws&lt;/span&gt; &lt;span class="nc"&gt;Throwable&lt;/span&gt; &lt;span class="o"&gt;{&lt;/span&gt;
        &lt;span class="c1"&gt;// Load rust dynamic library&lt;/span&gt;
        &lt;span class="nc"&gt;System&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;loadLibrary&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"rust_lib"&lt;/span&gt;&lt;span class="o"&gt;);&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;linker&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Linker&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;nativeLinker&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;lookup&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;SymbolLookup&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;loaderLookup&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;

        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;dataSize&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt;

        &lt;span class="c1"&gt;// Get rust function address&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;rustDataFunction&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;lookup&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;find&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"read_data_rust"&lt;/span&gt;&lt;span class="o"&gt;).&lt;/span&gt;&lt;span class="na"&gt;orElseThrow&lt;/span&gt;&lt;span class="o"&gt;();&lt;/span&gt;
        &lt;span class="c1"&gt;// Function takes two arguments and returns nothing&lt;/span&gt;
        &lt;span class="nc"&gt;FunctionDescriptor&lt;/span&gt; &lt;span class="n"&gt;rustDataDescriptor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FunctionDescriptor&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;ofVoid&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;
                &lt;span class="nc"&gt;ValueLayout&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;ADDRESS&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt;  &lt;span class="c1"&gt;// pointer to shared memory&lt;/span&gt;
                &lt;span class="nc"&gt;ValueLayout&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;JAVA_INT&lt;/span&gt; &lt;span class="c1"&gt;// size of elements&lt;/span&gt;
        &lt;span class="o"&gt;);&lt;/span&gt;

        &lt;span class="c1"&gt;// Create the method handle to call the rust function&lt;/span&gt;
        &lt;span class="nc"&gt;MethodHandle&lt;/span&gt; &lt;span class="n"&gt;rustDataHandle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;linker&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;downcallHandle&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;rustDataFunction&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;rustDataDescriptor&lt;/span&gt;
        &lt;span class="o"&gt;);&lt;/span&gt;

        &lt;span class="c1"&gt;// Allocate native memory that both Java and Rust can access&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Arena&lt;/span&gt; &lt;span class="n"&gt;arena&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Arena&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;ofConfined&lt;/span&gt;&lt;span class="o"&gt;())&lt;/span&gt; &lt;span class="o"&gt;{&lt;/span&gt;
            &lt;span class="c1"&gt;// Allocate memory X integers&lt;/span&gt;
            &lt;span class="nc"&gt;MemorySegment&lt;/span&gt; &lt;span class="n"&gt;sharedMemory&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;arena&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;allocate&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;ValueLayout&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;JAVA_INT&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt; &lt;span class="n"&gt;dataSize&lt;/span&gt;&lt;span class="o"&gt;);&lt;/span&gt;
            &lt;span class="c1"&gt;// Fill the memory from Java&lt;/span&gt;
            &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;dataSize&lt;/span&gt;&lt;span class="o"&gt;;&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="o"&gt;++)&lt;/span&gt; &lt;span class="o"&gt;{&lt;/span&gt;
                &lt;span class="n"&gt;sharedMemory&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;setAtIndex&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;ValueLayout&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;JAVA_INT&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="o"&gt;);&lt;/span&gt;
            &lt;span class="o"&gt;}&lt;/span&gt;
            &lt;span class="c1"&gt;// Pass the memory address to Rust code&lt;/span&gt;
            &lt;span class="n"&gt;rustDataHandle&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="na"&gt;invoke&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sharedMemory&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt; &lt;span class="n"&gt;dataSize&lt;/span&gt;&lt;span class="o"&gt;);&lt;/span&gt;
        &lt;span class="o"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Compile both Rust and Java project and run Java app again&lt;/p&gt;

&lt;p&gt;Tada (again) 🎉&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;🦀[rust] data received from Java 🔎 📦
    &lt;span class="o"&gt;[&lt;/span&gt;0] &lt;span class="o"&gt;=&lt;/span&gt; 0
    &lt;span class="o"&gt;[&lt;/span&gt;1] &lt;span class="o"&gt;=&lt;/span&gt; 10
    &lt;span class="o"&gt;[&lt;/span&gt;2] &lt;span class="o"&gt;=&lt;/span&gt; 20
    &lt;span class="o"&gt;[&lt;/span&gt;3] &lt;span class="o"&gt;=&lt;/span&gt; 30
    &lt;span class="o"&gt;[&lt;/span&gt;4] &lt;span class="o"&gt;=&lt;/span&gt; 40
    &lt;span class="o"&gt;[&lt;/span&gt;5] &lt;span class="o"&gt;=&lt;/span&gt; 50
    &lt;span class="o"&gt;[&lt;/span&gt;6] &lt;span class="o"&gt;=&lt;/span&gt; 60
    &lt;span class="o"&gt;[&lt;/span&gt;7] &lt;span class="o"&gt;=&lt;/span&gt; 70
    &lt;span class="o"&gt;[&lt;/span&gt;8] &lt;span class="o"&gt;=&lt;/span&gt; 80
    &lt;span class="o"&gt;[&lt;/span&gt;9] &lt;span class="o"&gt;=&lt;/span&gt; 90
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Java FFM Api combined with Rust provided a really easy way to call native function and shared memory efficiently without any complex over head like with JNI (its predecessor).&lt;/p&gt;

&lt;p&gt;Full source code and more examples are available on GitHub: &lt;a href="https://github.com/fb64/java-ffm-rust" rel="noopener noreferrer"&gt;https://github.com/fb64/java-ffm-rust&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Resources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://docs.oracle.com/en/java/javase/25/core/foreign-function-and-memory-api.html" rel="noopener noreferrer"&gt;https://docs.oracle.com/en/java/javase/25/core/foreign-function-and-memory-api.html&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://doc.rust-lang.org/reference/linkage.html#r-link.dylib" rel="noopener noreferrer"&gt;https://doc.rust-lang.org/reference/linkage.html#r-link.dylib&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.oracle.com/en/java/javase/25/docs/specs/jni/index.html" rel="noopener noreferrer"&gt;https://docs.oracle.com/en/java/javase/25/docs/specs/jni/index.html&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/fb64/java-ffm-rust" rel="noopener noreferrer"&gt;https://github.com/fb64/java-ffm-rust&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>java</category>
      <category>rust</category>
      <category>programming</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>DuckDB + Iceberg: The ultimate synergy</title>
      <dc:creator>Florian Bernard</dc:creator>
      <pubDate>Sun, 21 Sep 2025 12:54:07 +0000</pubDate>
      <link>https://dev.to/fb64/duckdb-iceberg-the-ultimate-synergy-1o5e</link>
      <guid>https://dev.to/fb64/duckdb-iceberg-the-ultimate-synergy-1o5e</guid>
      <description>&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://iceberg.apache.org/" rel="noopener noreferrer"&gt;Apache Iceberg&lt;/a&gt; and &lt;a href="https://duckdb.org/" rel="noopener noreferrer"&gt;DuckDB&lt;/a&gt; have established themselves as key players in data architecture landscape. With DuckDB 1.4's native support for Iceberg writes, combined with &lt;a href="https://polaris.apache.org/" rel="noopener noreferrer"&gt;Apache Polaris&lt;/a&gt; and &lt;a href="https://www.min.io/" rel="noopener noreferrer"&gt;MinIO&lt;/a&gt;, this promising stack offers efficiency, scalability, and flexibility.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftz716ze2dryhmbjo2223.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftz716ze2dryhmbjo2223.png" alt="Architecture diagram that show DuckDB + Polaris + MinIO &amp;gt;" width="800" height="1804"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Requirements
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.oracle.com/ae/java/technologies/downloads/#java21" rel="noopener noreferrer"&gt;Java&lt;/a&gt; &amp;gt;= 21&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://podman.io/" rel="noopener noreferrer"&gt;Podman&lt;/a&gt; or &lt;a href="https://www.docker.com/" rel="noopener noreferrer"&gt;Docker&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://duckdb.org/" rel="noopener noreferrer"&gt;DuckDB&lt;/a&gt; &amp;gt;= 1.4&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Setup Polaris + Minio
&lt;/h2&gt;

&lt;p&gt;🪞 Clone Apache Polaris repository&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/apache/polaris.git
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;🏗️ Build Polaris Docker image&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;cd &lt;/span&gt;polaris
./gradlew :polaris-server:assemble &lt;span class="nt"&gt;-Dquarkus&lt;/span&gt;.container-image.build&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;true&lt;/span&gt; 
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;▶️ Start Polaris + MinIO&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;podman compose &lt;span class="nt"&gt;-f&lt;/span&gt; getting-started/minio/docker-compose.yml up
&lt;span class="c"&gt;# you can replace podman with docker&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This will create a MinIO Bucket: &lt;code&gt;bucket123&lt;/code&gt; and a Polaris catalog: &lt;code&gt;quickstart_catalog&lt;/code&gt; &lt;/p&gt;

&lt;p&gt;MinIO user=&lt;strong&gt;minio_root&lt;/strong&gt; password=&lt;strong&gt;m1n1opwd&lt;/strong&gt;&lt;br&gt;
Polaris user=&lt;strong&gt;root&lt;/strong&gt; password=&lt;strong&gt;s3cr3t&lt;/strong&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Let's use DuckDB with Iceberg
&lt;/h2&gt;

&lt;p&gt;🦆 Install DuckDB&lt;br&gt;
&lt;code&gt;curl https://install.duckdb.org | sh&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;▶️ Start DuckDB client:&lt;br&gt;
&lt;code&gt;duckdb&lt;/code&gt; or &lt;code&gt;duckdb -ui&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;🧊 Install and load Iceberg extension&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;INSTALL&lt;/span&gt; &lt;span class="n"&gt;httpfs&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="n"&gt;INSTALL&lt;/span&gt; &lt;span class="n"&gt;iceberg&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;LOAD&lt;/span&gt; &lt;span class="n"&gt;httpfs&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;--https://github.com/duckdb/duckdb-iceberg/issues/483&lt;/span&gt;
&lt;span class="k"&gt;LOAD&lt;/span&gt; &lt;span class="n"&gt;iceberg&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;🔒 Create a secret to connect to Apache Polaris&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="n"&gt;SECRET&lt;/span&gt; &lt;span class="n"&gt;polaris_secret&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="k"&gt;TYPE&lt;/span&gt; &lt;span class="n"&gt;iceberg&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;CLIENT_ID&lt;/span&gt; &lt;span class="s1"&gt;'root'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;CLIENT_SECRET&lt;/span&gt; &lt;span class="s1"&gt;'s3cr3t'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;ENDPOINT&lt;/span&gt; &lt;span class="s1"&gt;'http://localhost:8181/api/catalog'&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;🔗 Attach the Polaris Catalog&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;ATTACH&lt;/span&gt; &lt;span class="s1"&gt;'quickstart_catalog'&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;polaris_catalog&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="k"&gt;TYPE&lt;/span&gt; &lt;span class="n"&gt;iceberg&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;ENDPOINT&lt;/span&gt; &lt;span class="s1"&gt;'http://localhost:8181/api/catalog'&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;🆕 Create a new Schema (namespace in Polaris)&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;create&lt;/span&gt; &lt;span class="k"&gt;schema&lt;/span&gt; &lt;span class="n"&gt;polaris_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;duckdb&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;🚖 Create a new Iceberg table&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;create&lt;/span&gt; &lt;span class="k"&gt;table&lt;/span&gt; &lt;span class="n"&gt;polaris_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;duckdb&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;taxi&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; 
&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;read_parquet&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
&lt;span class="s1"&gt;'https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2025-01.parquet'&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;📊 Query Iceberg table&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;polaris_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;duckdb&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;taxi&lt;/span&gt; &lt;span class="k"&gt;limit&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5e0w4042ec7hfv10qzng.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5e0w4042ec7hfv10qzng.png" alt="DuckDB query result table" width="800" height="417"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;📁 Explore created files on MinIO:&lt;br&gt;
Open &lt;a href="http://localhost:9001" rel="noopener noreferrer"&gt;http://localhost:9001&lt;/a&gt; click on the bucket &lt;code&gt;bucket123&lt;/code&gt; and explore the content of &lt;code&gt;duckdb/taxi&lt;/code&gt; folders:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;data&lt;/strong&gt; folder contains Iceberg parquet files&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;metadata&lt;/strong&gt; folder contains Iceberg metadata files&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fkzsqcmdadpws1zussg1o.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fkzsqcmdadpws1zussg1o.png" alt="MinIO file explorer" width="800" height="311"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;In conclusion, combining Apache Polaris, MinIO and DuckDB (with Iceberg support) offers a powerful, open-source solution for data platform architecture. This stack helps avoid vendor lock-in and ensures high performance, providing the scalability and efficiency required for modern data needs.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://polaris.apache.org/releases/1.1.0/getting-started/minio/" rel="noopener noreferrer"&gt;https://polaris.apache.org/releases/1.1.0/getting-started/minio/&lt;/a&gt;&lt;br&gt;
&lt;a href="https://duckdb.org/2025/09/16/announcing-duckdb-140.html" rel="noopener noreferrer"&gt;https://duckdb.org/2025/09/16/announcing-duckdb-140.html&lt;/a&gt;&lt;br&gt;
&lt;a href="https://duckdb.org/docs/stable/core_extensions/iceberg/iceberg_rest_catalogs" rel="noopener noreferrer"&gt;https://duckdb.org/docs/stable/core_extensions/iceberg/iceberg_rest_catalogs&lt;/a&gt;&lt;br&gt;
&lt;a href="https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page" rel="noopener noreferrer"&gt;https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page&lt;/a&gt;&lt;/p&gt;

</description>
      <category>apacheiceberg</category>
      <category>duckdb</category>
      <category>apachepolaris</category>
      <category>database</category>
    </item>
    <item>
      <title>Kotlin DataFrame ❤️ Arrow</title>
      <dc:creator>Florian Bernard</dc:creator>
      <pubDate>Thu, 10 Oct 2024 21:23:43 +0000</pubDate>
      <link>https://dev.to/fb64/kotlin-dataframe-arrow-54kh</link>
      <guid>https://dev.to/fb64/kotlin-dataframe-arrow-54kh</guid>
      <description>&lt;p&gt;&lt;a href="https://github.com/Kotlin/dataframe/releases/tag/v0.14.1" rel="noopener noreferrer"&gt;Kotlin DataFrame v0.14&lt;/a&gt; comes with improvements for reading &lt;a href="https://arrow.apache.org/" rel="noopener noreferrer"&gt;Apache Arrow&lt;/a&gt; format, especially loading a DataFrame from any &lt;a href="https://arrow.apache.org/docs/java/reference/org/apache/arrow/vector/ipc/ArrowReader.html" rel="noopener noreferrer"&gt;ArrowReader&lt;/a&gt;. &lt;br&gt;
This improvement can be used to easily load results from analytical databases (such as DuckDB, ClickHouse) directly into Kotlin DataFrame.&lt;/p&gt;

&lt;p&gt;Here are two examples of integrations that allow for smooth data import into Kotlin DataFrames using Apache Arrow.&lt;/p&gt;
&lt;h2&gt;
  
  
  DuckDB
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://duckdb.org/" rel="noopener noreferrer"&gt;DuckDB&lt;/a&gt; is an Analytics database that can be embedded for use in a Kotlin notebook. DuckDB facilitates reading query &lt;a href="https://duckdb.org/docs/api/java.html#arrow-methods" rel="noopener noreferrer"&gt;results as an Arrow Stream&lt;/a&gt;, enabling straightforward loading into a Kotlin DataFrame.&lt;/p&gt;

&lt;p&gt;Here is a basic notebook which uses DuckDB for querying data from a remote parquet file and then importing the results into a Kotlin dataFrame using Arrow Stream&lt;/p&gt;


&lt;div class="ltag_gist-liquid-tag"&gt;
  
&lt;/div&gt;


&lt;h2&gt;
  
  
  ClickHouse
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://clickhouse.com/" rel="noopener noreferrer"&gt;ClickHouse&lt;/a&gt; is a high-performance, column-oriented SQL database management system (DBMS) designed for online analytical processing (OLAP). &lt;br&gt;
ClickHouse allows using Arrow Stream as an &lt;a href="https://clickhouse.com/docs/en/interfaces/formats#data-format-arrow-stream" rel="noopener noreferrer"&gt;output format&lt;/a&gt;. &lt;/p&gt;

&lt;p&gt;The next notebook uses the ClickHouse client for querying data in Arrow stream format and loads it in a kotlin dataFrame. &lt;/p&gt;


&lt;div class="ltag_gist-liquid-tag"&gt;
  
&lt;/div&gt;


&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Loading Arrow data into Kotlin DataFrame is both straightforward and efficient, harnessing the full power of Kotlin for data analysis. This integration not only simplifies the process but also enhances performance, making Kotlin DataFrame a powerful tool for handling and analyzing large datasets.&lt;/p&gt;

</description>
      <category>kotlin</category>
      <category>arrow</category>
      <category>duckdb</category>
      <category>clickhouse</category>
    </item>
  </channel>
</rss>
