Clojure core.async: A Question & Answer Journey on Channels

Introduction

core.async is Clojure's library for writing concurrent code without dealing with locks, callbacks, or complex threading.

Write asynchronous code that looks synchronous. Your code reads top-to-bottom, even though operations happen concurrently.

Components

• Channels: queues that connect producers and consumers. They hold data temporarily and coordinate when to send and receive values

• Operations: Non-blocking and blocking put and take operations for sending and receiving data

• Go block: Place where you write sequential-looking async code

This post focuses exclusively on Channels. We'll explore Operations (>!, <!, >!!, <!!) and Go blocks in the next posts.

Channels

What exactly is a channel?

A channel is much more than a simple queue—it's a coordination mechanism that manages communication between independent processes

Internal Structure:

Every channel maintains four key components:

Buffer: Optional fixed-size queue holding values in transit. Can be:

• nil (unbuffered - rendezvous semantics)
• Fixed buffer (bounded queue)
• Sliding buffer (drops oldest when full)
• Dropping buffer (drops newest when full)

Take Queue: List of consumers waiting to receive values (when buffer is empty)

Put Queue: List of producers waiting to send values (when buffer is full)

Closed Flag + Lock: Thread-safe state management for the channel lifecycle

How data flow in the channel internal structure?

When you put a value on a channel, the implementation checks if any consumers are waiting. If so, the value is handed directly to a waiting consumer. If not, the value goes into the buffer (if there's space), or the producer joins the put queue to wait.

(require '[clojure.core.async :as a :refer [<! >! <!! >!! go go-loop chan close!]])

;; Unbuffered channel - producers and consumers must rendezvous
(def unbuffered-ch (chan))

;; Fixed buffer - can hold up to 10 values before blocking producers
(def buffered-ch (chan 10))

;; Sliding buffer - drops oldest values when full
(def sliding-ch (chan (a/sliding-buffer 5)))

;; Dropping buffer - drops newest values when full
(def dropping-ch (chan (a/dropping-buffer 5)))

;; Demonstration of buffer behavior
(let [ch (chan 2)]  ; Buffer size 2
  (go
    (>! ch 1)  ; Goes into buffer immediately
    (>! ch 2)  ; Goes into buffer immediately
    (println "Put two values")
    (>! ch 3)  ; Blocks here - buffer full, no consumer
    (println "Put third value"))

  (Thread/sleep 100)  ; Let producer run
  (go
    (println "First value:" (<! ch))   ; 1
    (println "Second value:" (<! ch))  ; 2
    (println "Third value:" (<! ch)))) ; 3 - now producer can complete

;; Output:
;; Put two values
;; First value: 1
;; Second value: 2
;; Put third value
;; Third value: 3

What is diff between buffered vs unbuffered channels?

Buffered channel = A mailbox where you can drop letters (items stored until someone picks them up).

Unbuffered channel = Handing something directly to someone (requires both people present at the same time).

Unbuffered channels have no storage capacity. They cannot hold any items at all.
When you do (chan) with no buffer size, you create a channel with a buffer size of 0. This means:

• A producer trying to put a value must wait until a consumer is ready to take it
• A consumer trying to take a value must wait until a producer is ready to give it
• The value passes directly from producer to consumer in a "handshake" (No values are ever stored in the channel itself)

Important Note:
Both buffered and unbuffered channels work with all four operations: >!, <!, >!!, <!!
The difference is in when blocking/parking happens:

• Buffered: Only blocks/parks when buffer is full (for puts) or empty (for takes)
• Unbuffered: Always blocks/parks until handoff completes

What are use cases for buffered and unbuffered channels?

Buffered Channels Use case

I'm putting this in the queue for you to get to it when you can.

1. When producers generate data in bursts but consumers process steadily (e.g., user clicking buttons rapidly, but backend processes one request at a time).
2. When you want to queue up work without blocking the producer (e.g., logging system that accepts messages quickly while writing to disk happens in background)
3. Email Queue: Marketing system generates 10,000 emails instantly when campaign launches, but email server can only send 100 per minute. Buffer stores the emails while they're sent gradually.
4. Restaurant Order System: Kitchen receives orders in bursts during lunch rush (10 orders in 2 minutes, then nothing for 5 minutes). Buffer holds the orders so they're not lost, and kitchen processes them steadily.

Unbuffered Channels Use case

When you need confirmation that data was received before continuing (e.g., a worker acknowledging it accepted a task)

1. Worker Task Assignment: A manager hands work to a team member and waits until they confirm "I've got it" before walking away. You don't want to assume they took the task—you need to see them accept it.
2. Database Transaction Coordination: Service A completes a database write and must wait for Service B to acknowledge it read the data before A can proceed. Both must be synchronized at that exact moment.
3. Checkout Process: Customer submits payment, and the system must wait for payment gateway to confirm receipt before showing "Order Placed." Can't just fire-and-forget.
4. Handshake Protocols: Two systems establishing a connection—client sends "hello," waits for server to respond "hello back" before proceeding. Each step requires mutual acknowledgment.

What happens when a channel closes?

Closing a channel signals "no more values will ever be sent." This is the standard way to communicate completion in core.async pipelines.

Semantics:

• Closed channels immediately return nil to all consumers
• Attempts to put on a closed channel return false (or throw in blocking operations)
• You can still drain values that were buffered before the close
• Closing is idempotent—closing an already-closed channel is safe

(def ch (chan 5))

;; Producer sends values then closes
(go
  (>! ch 1)
  (>! ch 2)
  (>! ch 3)
  (close! ch)
  (println "Channel closed")

  ;; Attempting to put on closed channel
  (let [result (>! ch 4)]
    (println "Put after close returned:" result)))  ; false

;; Consumer loops until channel closes
(go-loop []
  (if-some [val (<! ch)]
    (do
      (println "Received:" val)
      (recur))
    (println "Channel closed, exiting")))

;; Output:
;; Received: 1
;; Received: 2
;; Received: 3
;; Channel closed
;; Put after close returned: false
;; Channel closed, exiting

Critical pattern: Use if-some or when-some to distinguish between:

• nil because channel closed
• nil as an actual value (though sending nil is discouraged)

;; Proper close detection
(go-loop []
  (when-some [val (<! ch)]  ; Continues only if val is non-nil
    (process val)
    (recur))
  ;; Loop exits when channel returns nil (closed)
  (println "Cleanup and shutdown"))

Key Takeaways

1. Channels are coordination mechanisms, not just queues. They manage the flow of data between independent processes.

2. Choose your buffer type wisely:

   • Unbuffered: When you need rendezvous/handshake semantics
   • Fixed buffer: For backpressure and controlled queuing
   • Sliding buffer: When newest data matters most (like sensor readings)
   • Dropping buffer: When you can afford to lose data under load

3. Always close channels when production is complete. This is how consumers know to stop waiting.

4. Use if-some/when-some to properly detect channel closure versus nil values.

5. Buffered channels decouple producers and consumers in time, while unbuffered channels force synchronization.

What's Next?

Now that you understand how channels work internally and when to use different buffer types, the next post will dive into Operations:

• The difference between parking (>!, <!) and blocking (>!!, <!!) operations
• When to use each operation type
• How go blocks enable efficient concurrent programming
• Composing channels with alts!, mult, pub/sub, and pipelines