DEV Community

Mallikarjun H T
Mallikarjun H T

Posted on

3

Multipart download From S3 in Java

   public void multipartDownload(String accessKey, String secretKey, String region, String bucketName, String sourceFileName, String destinationFileName) throws Exception {

    try {
        // Create an S3 client using the provided access key, secret key, and region
        S3Client s3Client = getAWSS3Client(accessKey, secretKey, region);

        // List objects in the bucket with the specified prefix
        ListObjectsV2Response listObjectsV2Response = s3Client.listObjectsV2(b -> b.bucket(bucketName).prefix(sourceFileName));
        List<S3Object> objects = listObjectsV2Response.contents();
        Long size = objects.get(0).size();
        log.info("size of the file is {}", size);

        // Check if the file size is 0 and throw an exception if it is
        if (size == 0) {
            throw new FileNotFoundException("File is empty");
        }

        long partSize = 1024 * 1024 * 10; // Set the part size to 10 MB
        long contentLength = size;

        FileOutputStream fos = new FileOutputStream(destinationFileName);

        // Download the file in parts
        for (long start = 0; start < contentLength; start += partSize) {
            long end = Math.min(start + partSize - 1, contentLength - 1);
            GetObjectRequest rangeGetObjectRequest = GetObjectRequest.builder()
                    .bucket(bucketName)
                    .key(sourceFileName)
                    .range("bytes=" + start + "-" + end)
                    .build();

            ResponseInputStream<GetObjectResponse> responseInputStream = s3Client.getObject(rangeGetObjectRequest);

            byte[] buffer = new byte[1024 * 1024 * 10];
            int bytesRead;
            while ((bytesRead = responseInputStream.read(buffer)) != -1) {
                fos.write(buffer, 0, bytesRead);
            }
            responseInputStream.close();
        }
        fos.close();
    } catch (Exception e) {
        log.error("Exception Occurred while downloading files for key {} ", e.getMessage());
        throw new RuntimeException(e);
    }
}

Enter fullscreen mode Exit fullscreen mode

Image of Datadog

The Essential Toolkit for Front-end Developers

Take a user-centric approach to front-end monitoring that evolves alongside increasingly complex frameworks and single-page applications.

Get The Kit

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more