DEV Community: Abhishek @ Ace The Cloud

Exploring Supabase's Postgres-powered features for GoLang applications

Abhishek @ Ace The Cloud — Fri, 24 Mar 2023 16:20:38 +0000

Introduction
Supabase is a cloud-based platform that provides developers with various features that are essential in building and maintaining web and mobile applications. One of the most powerful features of Supabase is its Postgres-powered database. In this article, we will explore how to use Supabase's Postgres-powered features for GoLang applications.

Connecting to Supabase Postgres Database
To start using Supabase's Postgres-powered database, you first need to connect to it. Supabase provides a client library for GoLang that allows you to connect to the database and perform various operations. To get started, you need to install the client library.

go get github.com/supabase/supabase-go

After installing the client library, you can create a Supabase client by passing in your Supabase credentials.

import (
    "context"
    "fmt"

    "github.com/supabase/supabase-go"
)

func main() {
    sb, err := supabase.NewClient(context.Background(), "<SUPABASE_URL>", "<SUPABASE_KEY>")
    if err != nil {
        fmt.Println("Failed to create Supabase client:", err)
        return
    }
}

Once you have created the Supabase client, you can start using it to interact with the database.

Querying Data
The next step is to query data from the database. You can do this using the Supabase client's QueryBuilder. The QueryBuilder allows you to construct SQL queries and execute them against the database.

res, err := sb.From("todos").Select("*").Execute(context.Background())
if err != nil {
    fmt.Println("Failed to query todos:", err)
    return
}

fmt.Println("Todos:", res.Data)

The above code queries all the rows from the "todos" table and prints them to the console. You can also filter the results by adding a "Where" clause to the query.

res, err := sb.From("todos").Select("*").Eq("completed", true).Execute(context.Background())
if err != nil {
    fmt.Println("Failed to query todos:", err)
    return
}

fmt.Println("Completed todos:", res.Data)

The above code queries all the completed todos from the "todos" table and prints them to the console.

Inserting Data
To insert data into the database, you can use the Supabase client's InsertBuilder. The InsertBuilder allows you to construct SQL insert statements and execute them against the database.

res, err := sb.From("todos").Insert(map[string]interface{}{
    "title": "Buy milk",
    "completed": false,
}).Execute(context.Background())
if err != nil {
    fmt.Println("Failed to insert todo:", err)
    return
}

fmt.Println("Inserted todo:", res.Data)

The above code inserts a new todo into the "todos" table and prints the inserted row to the console.

Updating Data
To update data in the database, you can use the Supabase client's UpdateBuilder. The UpdateBuilder allows you to construct SQL update statements and execute them against the database.

res, err := sb.From("todos").Update(map[string]interface{}{
    "completed": true,
}).Eq("title", "Buy milk").Execute(context.Background())
if err != nil {
    fmt.Println("Failed to update todo:", err)
    return
}

fmt.Println("Updated todo:", res.Data)

The above code updates the "completed" field of the todo with title "Buy milk" to true and prints the updated row to the console.

Deleting Data
To delete data from the database, you can use the Supabase client's DeleteBuilder. The DeleteBuilder allows you to construct SQL delete statements and execute them against the database.

res, err := sb.From("todos").Delete().Eq("title", "Buy milk").Execute(context.Background())
if err != nil {
    fmt.Println("Failed to delete todo:", err)
    return
}

fmt.Println("Deleted todo:", res.Data)

The above code deletes the todo with title "Buy milk" from the "todos" table and prints the deleted row to the console.

Conclusion
In this article, we explored how to use Supabase's Postgres-powered features for GoLang applications. We learned how to connect to the database, query data, insert data, update data, and delete data. Supabase's Postgres-powered database provides a robust and scalable solution for building and maintaining web and mobile applications. By leveraging Supabase's client library for GoLang, you can easily interact with the database and build powerful applications with ease.

Exploring 10 Shades of Apache Kafka: A Deep Dive into the World of Streaming Data

Abhishek @ Ace The Cloud — Wed, 18 Jan 2023 06:59:26 +0000

Apache Kafka is a powerful, open-source streaming platform that is widely used for data integration, real-time data processing, and data analytics. Its ability to handle large volumes of data and its highly scalable, fault-tolerant architecture make it a popular choice among organizations of all sizes.

In this blog post, we will explore 10 different shades of Apache Kafka, delving into its various features and capabilities. We’ll start with a basic overview of Kafka’s architecture, then move on to more advanced topics such as data replication, partitioning, and security. We’ll also provide code examples to illustrate each concept.

1. Kafka Architecture

At its core, Kafka is a distributed, fault-tolerant, publish-subscribe messaging system. It is composed of three main components:

Producers: Applications that produce data and send it to Kafka topics.
Topics: Logical channels where data is stored and distributed to consumers.
Consumers: Applications that consume data from Kafka topics.
Here is a simple example of how to create a Kafka topic and send data to it using the Kafka Producer API:


import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);
ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key", "value");
producer. Send(record);
producer. Close();

2. Data Replication

Kafka replicates data across multiple brokers to ensure high availability and fault tolerance. Each topic is divided into multiple partitions, and each partition is replicated across a configurable number of replicas. This means that if one broker goes down, the data can still be accessed from one of the replicas.

Here is an example of how to configure data replication for a topic using the Kafka Admin API:

import org.apache.kafka.clients.admin.AdminClient;
import org.apache.kafka.clients.admin.NewTopic;

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");

AdminClient adminClient = AdminClient.create(props);
NewTopic newTopic = new NewTopic("my-topic", 3, (short) 2);
adminClient.createTopics(Collections.singleton(newTopic));
adminClient.close();

3. Partitioning

Kafka uses a partitioning strategy to distribute data across multiple brokers. The default partitioning strategy is based on the key of the record, but custom partitioning strategies can also be implemented. This allows for more granular control over how data is distributed and can help to achieve better performance and scalability.

Here is an example of how to implement a custom partitioning strategy using the Kafka Producer API:

import org.apache.kafka.clients.producer.Partitioner;
import org.apache.kafka.common.Cluster;

public class CustomPartitioner implements Partitioner {

    @Override
    public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
        // Implement custom logic to determine which partition to send the record to
        // For example, based on the value of the key
        return key.hashCode() % cluster.partitionCountForTopic(topic);
    }

    // Other required methods
}

4. Security

Kafka provides several security features to protect data and ensure that only authorized users can access it. These include:

Authentication: Users must provide valid credentials to access Kafka.
Authorization: Users are only allowed to access specific topics and perform specific operations.
Encryption: Data is encrypted in transit and at rest.
Here is an example of how to configure SSL encryption for a Kafka cluster:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("security.protocol", "SSL");
props.put("ssl.truststore.location", "/path/to/truststore.jks");
props.put("ssl.truststore.password", "password");
props.put("ssl.keystore.location", "/path/to/keystore.jks");
props.put("ssl.keystore.password", "password");
KafkaProducer<String, String> producer = new KafkaProducer<>(props);

5. Stream Processing

Kafka Streams is a client library for building stream processing applications on top of Kafka. It allows developers to easily create, process, and analyze streams of data in real-time. It provides a high-level API for defining stream processing logic, as well as a powerful set of operations for data transformation and aggregation.

Here is an example of how to use Kafka Streams to filter and transform data from a source topic and write the results to a target topic:

import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.kstream.KStream;

Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "my-stream-processing-app");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");

StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> sourceStream = builder.stream("source-topic");
KStream<String, String> filteredStream = sourceStream.filter((key, value) -> value.contains("important"));
filteredStream.to("target-topic");

KafkaStreams streams = new KafkaStreams(builder.build(), props);
streams. Start();

6. Connectors

Kafka Connect is a framework for building and running reusable data import and export connectors. It allows developers to easily connect Kafka to other systems, such as databases, file systems, and message queues. Connectors can be deployed as standalone processes or as part of a distributed worker cluster.

Here is an example of how to use the Kafka Connect JDBC Connector to import data from a MySQL database and write it to a Kafka topic:

import org.apache.kafka.connect.source.SourceRecord;
import org.apache.kafka.connect.source.SourceTask;

import java.util.List;
import java.util.Map;

public class MySQLSourceTask extends SourceTask {

    @Override
    public String version() {
        return "1.0";
    }

    @Override
    public void start(Map<String, String> props) {
        // Create JDBC connection and configure it using the properties provided
    }

    @Override
    public List<SourceRecord> poll() throws InterruptedException {
        // Execute SQL query and create a list of SourceRecord objects representing the query results
        return results;
    }

    @Override
    public void stop() {
        // Close JDBC connection
    }
}

7. Monitoring

Kafka provides several built-in tools for monitoring and managing the performance of a Kafka cluster. Some of these include:

Kafka Manager: A web-based tool for managing and monitoring Kafka clusters.
Kafka Monitoring: A set of scripts and Grafana dashboards for monitoring Kafka performance metrics.
Kafka Tool: A command-line tool for managing and monitoring Kafka topics and clusters.

8. Compaction

Kafka provides a feature called log compaction which allows for more efficient storage of log data by only retaining the latest version of each key. This can be useful in situations where the data being stored is periodically updated, such as in a database table where updates are made to the same primary key. Compaction can be enabled on a per-topic basis.

Here is an example of how to enable compaction on a topic using the Kafka Admin API:

import org.apache.kafka.clients.admin.AdminClient;
import org.apache.kafka.clients.admin.NewTopic;
import java.util.Collections;

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");

AdminClient adminClient = AdminClient.create(props);
Map<String, String> configs = new HashMap<>();
configs.put("cleanup.policy", "compact");
adminClient.alterTopicConfigs("my-topic", configs);
adminClient.close();

9. Consumer Groups

Kafka allows for multiple consumers to read from a topic simultaneously through the use of consumer groups. Each consumer in a group is assigned a subset of the partitions to read from, and if one consumer goes down, the other consumers will take over the workload. This allows for horizontal scaling of consumer applications.

Here is an example of how to create a consumer group using the Kafka Consumer API:

import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.consumer.ConsumerRecords;

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "my-consumer-group");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList("my-topic"));
ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(1000));

10. Event Sourcing

Kafka can also be used to implement event sourcing, a pattern in which the state of an application is stored as a series of events. Events are written to a topic, and the current state of the application can be reconstructed by replaying the events. This can be useful in situations where it’s necessary to maintain a complete history of all changes to the data.

Here is an example of how to implement event sourcing using Kafka:

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);

// Write events to the "events" topic
producer.send(new ProducerRecord<>("events", "user1", "user created"));
producer.send(new ProducerRecord<>("events", "user1", "user updated"));
producer.send(new ProducerRecord<>("events", "user1", "user deleted"));

// Read events from the "events" topic and rebuild the current state of the application
List<String> events = new ArrayList<>();
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList("events"));
ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, String> record : records) {
events.add(record.value());
}

// Output the current state of the application
System.out.println("Current state: " + events.get(events. Size() - 1));

These are just a few more examples of the many features and capabilities of Apache Kafka. As you can see, Kafka can be used in various ways, depending on the needs of your application, and it’s important to choose the right features to implement based on your use case.