DEV Community

Markus
Markus

Posted on • Originally published at the-main-thread.com on

SFTP Meets PostgreSQL: Building a Smart File Upload System with Quarkus Dev Services

#java #nlp #ai #textanalysis

hero image

Most file-handling tutorials stop at REST endpoints and a local filesystem. That’s fine for hello-world demos, but not for real systems.

In the enterprise, file transfers usually happen over secure protocols like SFTP, and metadata about those files, who sent what, when, and how big, must be tracked reliably for auditing, integration, or downstream processing. Think of use cases like invoice ingestion, batch job triggers, regulatory compliance, or secure document exchange.

In this hands-on guide, you’ll build a Quarkus-based file handling system that mirrors those real-world requirements. Files will be securely uploaded to an SFTP server, and every upload will store structured metadata (like filename, size, and timestamp) in a PostgreSQL database using Hibernate Panache. All of it runs in dev mode with zero manual container setup, thanks to Quarkus Dev Services and Compose Dev Services.

You get the speed of Quarkus, the structure of a proper data model, and a reproducible development environment—all essential ingredients for robust, production-ready file workflows.

What You’ll Build

By the end of this tutorial, you’ll have a running system with:

  • An SFTP server spun up via Docker Compose.

  • A PostgreSQL database managed by Quarkus Dev Services.

  • A FileMetadata entity stored with Hibernate Panache.

  • A REST API to upload files, download them, and query metadata.

🔧 Prerequisites

Make sure you have the basics installed:

  • Java 17+

  • Maven 3.8.1+

  • Podman + (Podman Compose or Docker Compose)

  • Your favorite IDE (VS Code or IntelliJ IDEA)

Create the Project

Start by generating a new Quarkus application with all the right extensions:

quarkus create app com.example:quarkus-sftp-compose \
  --extension='rest-jackson,hibernate-orm-panache,jdbc-postgresql' \
  --no-code
cd quarkus-sftp-compose
Enter fullscreen mode Exit fullscreen mode

This gives you a clean slate with REST, Panache ORM, PostgreSQL JDBC, and Docker image support. If you want to start with the ready-built project, clone it from my Github repository.

Add the JSch Dependency

We’ll use the mwiede/jsch fork to handle SFTP:

<dependency>
  <groupId>com.github.mwiede</groupId>
  <artifactId>jsch</artifactId>
  <version>2.27.2</version>
</dependency>
Enter fullscreen mode Exit fullscreen mode

Define compose-devservices.yml

Create a compose-devservices.yml file at the root of the project. It will define an SFTP server:

services:
  sftp:
    image: atmoz/sftp:latest
    container_name: my-sftp-dev
    volumes:
      - ./target/sftp_data:/home/testuser/upload
    ports:
      - "2222:22"
    command: testuser:testpass:::upload
    labels:
      io.quarkus.devservices.compose.wait_for.logs: .*Server listening on :: port 22.*
Enter fullscreen mode Exit fullscreen mode

Quarkus will detect and launch this for you during quarkus dev.

Configure application.properties

Now wire everything up by editing src/main/resources/application.properties:

# SFTP Configuration
sftp.host=localhost
sftp.port=2222
sftp.user=testuser
sftp.password=testpass
sftp.remote.directory=/upload

# PostgreSQL
quarkus.datasource.db-kind=postgresql

# Hibernate ORM
quarkus.hibernate-orm.database.generation=drop-and-create
Enter fullscreen mode Exit fullscreen mode

This setup ensures Quarkus will automatically launch PostgreSQL and SFTP containers before starting your app.

Create the FileMetadata Entity

Let’s create a simple entity to track uploaded files. Create src/main/java/com/example/FileMetadata.java:

package com.example;

import java.time.LocalDateTime;

import io.quarkus.hibernate.orm.panache.PanacheEntity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;

@Entity
public class FileMetadata extends PanacheEntity {

    @Column(unique = true, nullable = false)
    public String fileName;

    public long fileSize;

    public LocalDateTime uploadTimestamp;
}
Enter fullscreen mode Exit fullscreen mode

Thanks to Panache, you get a default id field and easy CRUD access methods out of the box.

Build the SFTP Service

Now create src/main/java/com/example/SftpService.java. This service will:

  • Upload files to the SFTP container.

  • Save metadata to the database.

  • Handle file download.

package com.example;

import java.io.InputStream;
import java.time.LocalDateTime;

import org.eclipse.microprofile.config.inject.ConfigProperty;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.SftpException;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.transaction.Transactional;

@ApplicationScoped
public class SftpService {

    @ConfigProperty(name = "sftp.host")
    String host;
    // ... (other @ConfigProperty fields for port, user, password, remoteDirectory)
    @ConfigProperty(name = "sftp.port")
    int port;

    @ConfigProperty(name = "sftp.user")
    String user;

    @ConfigProperty(name = "sftp.password")
    String password;

    @ConfigProperty(name = "sftp.remote.directory")
    String remoteDirectory;

    @Transactional // This is crucial for database operations
    public FileMetadata uploadFile(String fileName, long fileSize, InputStream inputStream)
            throws JSchException, SftpException {
        // 1. Upload the file to SFTP
        ChannelSftp channelSftp = createSftpChannel();
        try {
            channelSftp.connect();
            String remoteFilePath = remoteDirectory + "/" + fileName;
            channelSftp.put(inputStream, remoteFilePath);
        } finally {
            disconnectChannel(channelSftp);
        }

        // 2. Persist metadata to the database
        FileMetadata meta = new FileMetadata();
        meta.fileName = fileName;
        meta.fileSize = fileSize;
        meta.uploadTimestamp = LocalDateTime.now();
        meta.persist(); // Panache makes saving simple!

        return meta;
    }

    public InputStream downloadFile(String fileName) throws JSchException, SftpException {
        // This method remains the same as before
        ChannelSftp channelSftp = createSftpChannel();
        channelSftp.connect();
        String remoteFilePath = remoteDirectory + "/" + fileName;
        return new SftpInputStream(channelSftp.get(remoteFilePath), channelSftp);
    }

    // The private helper methods (createSftpChannel, disconnectChannel,
    // SftpInputStream)
    // remain the same. Copy them from the previous tutorial.
    private ChannelSftp createSftpChannel() throws JSchException {
        JSch jsch = new JSch();
        Session session = jsch.getSession(user, host, port);
        session.setPassword(password);
        java.util.Properties config = new java.util.Properties();
        config.put("StrictHostKeyChecking", "no");
        session.setConfig(config);
        session.connect();
        return (ChannelSftp) session.openChannel("sftp");
    }

    private void disconnectChannel(ChannelSftp channel) {
        if (channel != null) {
            if (channel.isConnected()) {
                channel.disconnect();
            }
            try {
                if (channel.getSession() != null && channel.getSession().isConnected()) {
                    channel.getSession().disconnect();
                }
            } catch (JSchException e) {
                // Ignore
            }
        }
    }

    private class SftpInputStream extends InputStream {
        private final InputStream sftpStream;
        private final ChannelSftp channelToDisconnect;

        public SftpInputStream(InputStream sftpStream, ChannelSftp channel) {
            this.sftpStream = sftpStream;
            this.channelToDisconnect = channel;
        }

        @Override
        public int read() throws java.io.IOException {
            return sftpStream.read();
        }

        @Override
        public void close() throws java.io.IOException {
            sftpStream.close();
            disconnectChannel(channelToDisconnect);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Build the REST API

Create src/main/java/com/example/SftpResource.java. This exposes our three endpoints:

package com.example;

import java.io.InputStream;

import jakarta.inject.Inject;
import jakarta.ws.rs.Consumes;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.HeaderParam;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;

@Path("/files")
public class SftpResource {

    @Inject
    SftpService sftpService;

    @POST
    @Path("/upload/{fileName}")
    @Consumes(MediaType.APPLICATION_OCTET_STREAM)
    @Produces(MediaType.APPLICATION_JSON)
    public Response uploadFile(@PathParam("fileName") String fileName, @HeaderParam("Content-Length") long fileSize,
            InputStream fileInputStream) {
        try {
            FileMetadata meta = sftpService.uploadFile(fileName, fileSize, fileInputStream);
            return Response.status(Response.Status.CREATED).entity(meta).build();
        } catch (Exception e) {
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
                    .entity("Failed to upload file: " + e.getMessage()).build();
        }
    }

    @GET
    @Path("/meta/{fileName}")
    @Produces(MediaType.APPLICATION_JSON)
    public Response getMetadata(@PathParam("fileName") String fileName) {
        return FileMetadata.find("fileName", fileName)
                .firstResultOptional()
                .map(meta -> Response.ok(meta).build())
                .orElse(Response.status(Response.Status.NOT_FOUND).build());
    }

    @GET
    @Path("/download/{fileName}")
    @Produces(MediaType.APPLICATION_OCTET_STREAM)
    public Response downloadFile(@PathParam("fileName") String fileName) {
        // This method remains the same as before
        try {
            InputStream inputStream = sftpService.downloadFile(fileName);
            return Response.ok(inputStream).header("Content-Disposition", "attachment; filename=\"" + fileName + "\"")
                    .build();
        } catch (Exception e) {
            if (e instanceof com.jcraft.jsch.SftpException
                    && ((com.jcraft.jsch.SftpException) e).id == com.jcraft.jsch.ChannelSftp.SSH_FX_NO_SUCH_FILE) {
                return Response.status(Response.Status.NOT_FOUND).entity("File not found: " + fileName).build();
            }
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
                    .entity("Failed to download file: " + e.getMessage()).build();
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Run and Test It All 🧪

Start the app:

quarkus dev
Enter fullscreen mode Exit fullscreen mode

Watch it pull and start your SFTP and PostgreSQL services automatically.

Upload a File 

echo "Hello from Dev Services!" > my-file.txt

curl -X POST -H "Content-Type: application/octet-stream" \
     --data-binary "@my-file.txt" \
     http://localhost:8080/files/upload/my-file.txt
Enter fullscreen mode Exit fullscreen mode

Check Metadata 

curl http://localhost:8080/files/meta/my-file.txt
Enter fullscreen mode Exit fullscreen mode

You should see a JSON response with file metadata.

Download the File 

curl http://localhost:8080/files/download/my-file.txt -o downloaded.txt
cat downloaded.txt
Enter fullscreen mode Exit fullscreen mode

You can also check in the container if the file actually exists in the SFTP server:

podman exec my-sftp-dev ls /home/testuser/upload
Enter fullscreen mode Exit fullscreen mode

What You’ve Learned

You’ve now combined:

  • SFTP file transfers

  • PostgreSQL persistence via Panache

  • Quarkus Dev Services with Compose Dev Services

All in a way that mimics a real production scenario, while remaining developer-friendly.

Want to go further? You could:

  • Add authentication for upload/download.

  • Track version history per file.

  • Integrate with OpenTelemetry for observability.

  • Build a UI using Qute or integrate with React.

But for now, you’ve got a full-stack, smart file handling service built in under an hour. Well played.

Top comments (0)