DEV Community

Tech Forge
Tech Forge

Posted on

Designing Idempotent APIs for Reliability

#api #webdev #javascript #reliability

What Is Idempotency?

Idempotency is a property of an operation that can be applied multiple times without changing the result beyond the first application. In HTTP terms, an idempotent request is one where the client can safely retry the same request and get the same effect, no matter how many times it is sent.

For example, GET is naturally idempotent: fetching a resource multiple times yields the same data. PUT and DELETE are also idempotent by design: replacing a resource with the same payload or deleting an already deleted resource results in the same final state. POST is not idempotent because it creates a new resource each time.

Why Idempotency Matters

In distributed systems, network failures, timeouts, and retries are common. If a client sends a request and the server processes it but the response is lost, the client will retry. Without idempotency, that retry could cause duplicate orders, double charges, or inconsistent state. Idempotency ensures that retries are safe, making your API more reliable and user-friendly.

Implementing Idempotency Keys

The standard way to make a non-idempotent endpoint (like POST) idempotent is to use an idempotency key. The client generates a unique key (often a UUID) and sends it in a header, typically Idempotency-Key. The server stores the key and the response for a certain period. On retry, the server checks if the key already exists; if so, it returns the stored response without re-executing the operation.

Here is a simple implementation in Node.js using an in-memory store:

const express = require('express');
const crypto = require('crypto');

const app = express();
app.use(express.json());

const idempotencyStore = new Map();

app.post('/orders', (req, res) => {
  const key = req.get('Idempotency-Key');
  if (!key) {
    return res.status(400).json({ error: 'Missing Idempotency-Key' });
  }

  if (idempotencyStore.has(key)) {
    return res.status(200).json(idempotencyStore.get(key));
  }

  // Simulate order creation
  const order = { id: crypto.randomUUID(), ...req.body };
  idempotencyStore.set(key, order);

  res.status(201).json(order);
});

app.listen(3000);
Enter fullscreen mode Exit fullscreen mode

In a real application, you would persist the idempotency store in a database with a unique constraint on the key to avoid race conditions.

Best Practices for Idempotency Keys

  • Client-side generation: The client should generate a unique key for each logical operation. Reuse the same key for retries of that operation.
  • Server-side expiry: Store the key-response pair with an expiry (e.g., 24 hours) to avoid unbounded growth.
  • Response caching: Return the original response on duplicate requests, including the same status code and body.
  • Handle concurrent requests: Use database constraints (e.g., unique index) to prevent two simultaneous requests with the same key from both executing.
  • Include the key in retries: Ensure your HTTP client automatically adds the same key when retrying.

Idempotency in REST Semantics

While GET, PUT, DELETE, and HEAD are idempotent by HTTP spec, POST and PATCH are not. You can also make PATCH idempotent by using a specific patch format (like JSON Patch) or by using an idempotency key. However, it is often simpler to use PUT for full updates and reserve PATCH for partial updates, but still add an idempotency key if needed.

Handling Non-Idempotent Operations

Some operations are inherently non-idempotent, such as "increment counter" or "append to log". For these, you can still use idempotency keys by storing the result and returning it on retries. The key insight is that the server must deduplicate the operation based on the key, not on the data.

Testing Idempotency

Write tests that send the same request twice with the same key and assert that the second response is identical to the first and that only one resource was created. Also test with different keys to ensure separate operations are unaffected.

Conclusion

Idempotency is a simple yet powerful concept that greatly improves API reliability. By implementing idempotency keys on non-idempotent endpoints, you protect your system and your clients from the effects of network retries. It is a small addition that pays off in robustness and user trust.

Top comments (0)