DEV Community

Asanka Boteju
Asanka Boteju

Posted on

1 1 1

In-place Serverless Querying AWS S3 Data

We often have a need to directly query unstructured data stored in S3 Buckets in various data formats such as CSV, JSON, AVRO, ORC, PARQUET for ad-hoc querying or may be as a part of building a comprehensive data solution.

Below are some AWS Serverless services that you can use to directly query your S3 data.

1. Amazon Athena
Suitable for ad-hoc data discovery and SQL querying. In this service you are charged based on the amount of data scanned.

Image description

2. Amazon Redshift Spectrum
Suitable if you have to use more complex queries and also if you need to support a large user base.

Image description

Redshift spectrum is recommended due to below reasons.

  • Uses Redshift Data warehouse SQL syntax which can spans Redshift Tables and S3 Data Lakes.

  • Provides sophisticated query optimization.

  • Distributes queries across multiple nodes for parallel processing.

  • Can be used with already existing BI tools.

Thank you for your time...

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay