DEV Community

Aki profile picture

Aki

12× AWS‑Certified • AWS Community Builder (Data) Engineer specializing in cloud and data platforms, focusing on serverless ETL and modern lakehouse architectures.

Location Japan Joined Joined on  Personal website https://zenn.dev/penginpenguin github website twitter website
Comparing the Permission Models of the Iceberg REST APIs Provided by Glue Data Catalog and S3 Tables
datapenguin profile

Comparing the Permission Models of the Iceberg REST APIs Provided by Glue Data Catalog and S3 Tables

#aws #dataengineering
5
Comments 2
9 min read

Want to connect with Aki?

Create an account to connect with Aki. You can also sign in below to proceed if you already have an account.

Create Account
Already have an account? Sign in
Connecting Amazon S3 Tables with Snowflake
datapenguin profile

Connecting Amazon S3 Tables with Snowflake

#aws #snowflake #dataengineering
3
Comments
11 min read
Verifying How IAM and Lake Formation Behave for the Glue REST Catalog and S3 Tables
datapenguin profile

Verifying How IAM and Lake Formation Behave for the Glue REST Catalog and S3 Tables

#aws #dataengineering
5
Comments 3
8 min read
Hitting the Iceberg REST Catalog Directly: Understanding the Differences Between Glue Data Catalog and S3 Tables
datapenguin profile

Hitting the Iceberg REST Catalog Directly: Understanding the Differences Between Glue Data Catalog and S3 Tables

#aws #iceberg #dataengineering
9
Comments
17 min read
Does Amazon S3 Tables Replace AWS Glue Data Catalog? Understanding Their Relationship
datapenguin profile

Does Amazon S3 Tables Replace AWS Glue Data Catalog? Understanding Their Relationship

#aws #iceberg #dataengineering
4
Comments
8 min read
Track Apache Iceberg Schema Changes in AWS Glue Data Catalog with aws glue get-table-versions
datapenguin profile

Track Apache Iceberg Schema Changes in AWS Glue Data Catalog with aws glue get-table-versions

#aws #dataengineering #iceberg
4
Comments
6 min read
Organizing How to Use AWS Lake Formation
datapenguin profile

Organizing How to Use AWS Lake Formation

#aws #dataengineering
5
Comments
9 min read
Rethinking Lakehouse Architecture Through Data Ownership: AWS vs. Snowflake
datapenguin profile

Rethinking Lakehouse Architecture Through Data Ownership: AWS vs. Snowflake

#aws #snowflake #iceberg #dataengineering
5
Comments
9 min read
Exploring Snowpark While Comparing It with Apache Spark
datapenguin profile
Aki

Exploring Snowpark While Comparing It with Apache Spark

#dataengineering #snowflake
4
Comments
8 min read
Organizing How to Use AWS Glue Workflow
datapenguin profile

Organizing How to Use AWS Glue Workflow

#aws #dataengineering
6
Comments
9 min read
Organizing the Use Cases of AWS Step Functions and Glue Workflow for ETL Processing with AWS Glue Jobs
datapenguin profile

Organizing the Use Cases of AWS Step Functions and Glue Workflow for ETL Processing with AWS Glue Jobs

#aws #dataengineering
4
Comments
7 min read
Differences Between Snowflake Editions and Secure Connectivity with AWS
datapenguin profile

Differences Between Snowflake Editions and Secure Connectivity with AWS

#aws #snowflake #dataengineering
3
Comments
8 min read
What Is Apache Polaris? Why Open Data Catalogs Matter and How to Use Them with AWS
datapenguin profile

What Is Apache Polaris? Why Open Data Catalogs Matter and How to Use Them with AWS

#aws #snowflake #dataengineering #iceberg
8
Comments
11 min read
Lightweight ETL on AWS Lambda Using DuckDB and Snowflake Connector
datapenguin profile

Lightweight ETL on AWS Lambda Using DuckDB and Snowflake Connector

#aws #snowflake #dataengineering
6
Comments
6 min read
Securely Implementing Snowflake AWS Lambda Integration with Key Pair Authentication + Secrets Manager
datapenguin profile

Securely Implementing Snowflake AWS Lambda Integration with Key Pair Authentication + Secrets Manager

#aws #snowflake #dataengineering
4
Comments
5 min read
AWS Snowflake Lakehouse: 2 Practical Apache Iceberg Integration Patterns
datapenguin profile

AWS Snowflake Lakehouse: 2 Practical Apache Iceberg Integration Patterns

#aws #snowflake #dataengineering #apacheiceberg
5
Comments
7 min read
Is AWS Glue Data Catalog Sufficient as a Data Catalog? Organizing Its Design, Limitations, and Complementary Strategies
datapenguin profile

Is AWS Glue Data Catalog Sufficient as a Data Catalog? Organizing Its Design, Limitations, and Complementary Strategies

#aws #dataengineering
7
Comments
10 min read
Boosting Lightweight ETL on AWS Lambda & Glue Python Shell with DuckDB and Apache Arrow Dataset
datapenguin profile

Boosting Lightweight ETL on AWS Lambda & Glue Python Shell with DuckDB and Apache Arrow Dataset

#aws #duckdb #chdb #dataengineering
6
Comments
9 min read
Does Increasing AWS Lambda Memory to 10GB Really Make It Faster? (AWS Lambda chDB/DuckDB PyIceberg Benchmark)
datapenguin profile

Does Increasing AWS Lambda Memory to 10GB Really Make It Faster? (AWS Lambda chDB/DuckDB PyIceberg Benchmark)

#aws #duckdb #chdb #iceberg
5
Comments
7 min read
S3 Triggers: How to Launch Glue Python Shell via AWS Lambda
datapenguin profile

S3 Triggers: How to Launch Glue Python Shell via AWS Lambda

#aws #dataengineering #etl #lambda
4
Comments
8 min read
Exploring the Potential of AWS Glue Python Shell as a Long-Running Batch Execution Environment
datapenguin profile

Exploring the Potential of AWS Glue Python Shell as a Long-Running Batch Execution Environment

#aws #dataengineering
4
Comments
7 min read
AWS Lambda and AWS Glue Python Shell in the Context of Lightweight ETL
datapenguin profile

AWS Lambda and AWS Glue Python Shell in the Context of Lightweight ETL

#aws #dataengineering #etl
3
Comments
7 min read
Lightweight ETL with AWS Glue Python Shell, chDB, and PyIceberg (Compared with DuckDB)
datapenguin profile

Lightweight ETL with AWS Glue Python Shell, chDB, and PyIceberg (Compared with DuckDB)

#aws #iceberg #duckdb #chdb
7
Comments
8 min read
Lightweight ETL with AWS Lambda, chDB, and PyIceberg (Compared with DuckDB)
datapenguin profile

Lightweight ETL with AWS Lambda, chDB, and PyIceberg (Compared with DuckDB)

#aws #iceberg #chdb #duckdb
11
Comments
5 min read
Looking at AWS Inter-Region Latency Through “Distance”
datapenguin profile

Looking at AWS Inter-Region Latency Through “Distance”

#aws
4
Comments
9 min read
Building a Data Platform on AWS: Essential Design Considerations for Power BI
datapenguin profile

Building a Data Platform on AWS: Essential Design Considerations for Power BI

#aws #dataengineering #data
5
Comments
5 min read
Optimizing Data Processing on AWS with Data Compaction
datapenguin profile

Optimizing Data Processing on AWS with Data Compaction

#aws #dataengineering #iceberg #lambda
4
Comments
7 min read
Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS
datapenguin profile

Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS

#aws #dataengineering #lambda #sqs
3
Comments
6 min read
Interoperating Open Table Formats on AWS Using Apache XTable (Delta Iceberg)
datapenguin profile

Interoperating Open Table Formats on AWS Using Apache XTable (Delta Iceberg)

#aws #iceberg #deltalake #dataengineering
4
Comments
4 min read
Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg
datapenguin profile

Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

#aws #dataengineering #duckdb #iceberg
6
Comments 4
4 min read
Lightweight ETL with AWS Lambda, DuckDB, and delta-rs
datapenguin profile

Lightweight ETL with AWS Lambda, DuckDB, and delta-rs

#aws #dataengineering #delta
4
Comments
5 min read
Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg
datapenguin profile

Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

#aws #duckdb #iceberg #dataengineering
5
Comments
4 min read
Build a Lightweight Serverless ETL Pipeline to Iceberg Tables with AWS Lambda Athena
datapenguin profile

Build a Lightweight Serverless ETL Pipeline to Iceberg Tables with AWS Lambda Athena

#aws #iceberg #dataengineering
2
Comments
4 min read
PyIceberg on AWS Lambda: Comparing GlueCatalog and REST Catalog Access Methods
datapenguin profile

PyIceberg on AWS Lambda: Comparing GlueCatalog and REST Catalog Access Methods

#aws #iceberg #dataengineering
2
Comments
3 min read
Lightweight ETL with AWS Glue Python Shell, DuckDB, and PyIceberg
datapenguin profile

Lightweight ETL with AWS Glue Python Shell, DuckDB, and PyIceberg

#aws #iceberg #dataengineering #duckdb
5
Comments 1
7 min read
AWS IP Range Lookup Tool for Quick Security Checks
datapenguin profile

AWS IP Range Lookup Tool for Quick Security Checks

#aws
2
Comments
5 min read
Building a CLI Tool to Visualize AWS Iceberg Table Snapshot History
datapenguin profile

Building a CLI Tool to Visualize AWS Iceberg Table Snapshot History

#aws #iceberg #opensource
6
Comments
7 min read
loading...