Designing Idempotent APIs

The Network Is Not Reliable

In a perfect world, every HTTP request succeeds exactly once and the client receives the response. In practice, networks time out, servers crash after committing a write but before sending the response, and clients retry requests that already succeeded. Without idempotency, these retries create duplicate records, double charges, or double-sent emails.

Idempotency is the property where sending the same request multiple times produces the same result as sending it once. GET and DELETE are naturally idempotent. POST — which creates or triggers something — is not, unless you design it to be.

This matters more as your system grows. Background job retries, webhook delivery, and mobile apps on unreliable connections all rely on safe retries. For the broader picture of secure, well-designed APIs, Securing REST APIs: Best Practices covers the security dimension that complements this.

The Idempotency Key Pattern

The standard approach: the client generates a unique key for each logical operation and sends it in a request header. The server stores the outcome of the first successful execution and returns that same outcome on any subsequent request with the same key.

POST /api/orders
Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000
Content-Type: application/json

{ "productId": "abc123", "quantity": 2 }

If the client sends this request three times (the first timed out, the second got a network error, the third succeeded), the order is created exactly once.

Implementation in a Next.js Route Handler

The server needs a fast, persistent store for idempotency records. Redis is the natural fit: it has atomic operations and TTL support.

typescript
// lib/idempotency.ts
import { createClient } from 'redis';

const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();

const TTL_SECONDS = 60 * 60 * 24; // 24 hours

type IdempotencyRecord =
  | { status: 'processing' }
  | { status: 'complete'; statusCode: number; body: unknown };

export async function getIdempotencyRecord(
  key: string
): Promise<IdempotencyRecord | null> {
  const raw = await redis.get(`idempotency:${key}`);
  return raw ? JSON.parse(raw) : null;
}

export async function setIdempotencyRecord(
  key: string,
  record: IdempotencyRecord
): Promise<void> {
  await redis.set(`idempotency:${key}`, JSON.stringify(record), {
    EX: TTL_SECONDS,
  });
}

typescript
// app/api/orders/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { getIdempotencyRecord, setIdempotencyRecord } from '@/lib/idempotency';
import { createOrder } from '@/lib/orders';

export async function POST(req: NextRequest) {
  const idempotencyKey = req.headers.get('Idempotency-Key');

  if (!idempotencyKey) {
    return NextResponse.json(
      { error: 'Idempotency-Key header is required' },
      { status: 400 }
    );
  }

  // Check for an existing record
  const existing = await getIdempotencyRecord(idempotencyKey);

  if (existing) {
    if (existing.status === 'processing') {
      // A concurrent request is still running
      return NextResponse.json(
        { error: 'Request is already being processed' },
        { status: 409 }
      );
    }
    // Return the cached response
    return NextResponse.json(existing.body, { status: existing.statusCode });
  }

  // Mark as in-progress before doing any work
  await setIdempotencyRecord(idempotencyKey, { status: 'processing' });

  try {
    const body = await req.json();
    const order = await createOrder(body);

    const responseBody = { orderId: order.id, status: order.status };
    await setIdempotencyRecord(idempotencyKey, {
      status: 'complete',
      statusCode: 201,
      body: responseBody,
    });

    return NextResponse.json(responseBody, { status: 201 });
  } catch (error) {
    // On failure, remove the processing lock so clients can retry
    await setIdempotencyRecord(idempotencyKey, {
      status: 'complete',
      statusCode: 500,
      body: { error: 'Internal server error' },
    });
    return NextResponse.json({ error: 'Internal server error' }, { status: 500 });
  }
}

The Critical Sequencing

The order of operations matters:

1. Check for existing record
2. Write { status: 'processing' } before executing the operation
3. Execute the operation
4. Write the final result

If you check and then execute without the intermediate write, two concurrent requests with the same key can both pass the check and both execute. The processing record is a distributed lock.

Idempotency Key Validation

Always validate the format of the key to prevent abuse:

typescript
const UUID_REGEX =
  /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;

if (!UUID_REGEX.test(idempotencyKey)) {
  return NextResponse.json(
    { error: 'Idempotency-Key must be a valid UUIDv4' },
    { status: 400 }
  );
}

A UUID v4 is a reasonable format to require — it is long enough to avoid collisions, short enough to not stress storage, and widely supported by every client SDK.

Database-Level Idempotency

For some operations, you can enforce idempotency at the database level instead of (or in addition to) the Redis layer. A unique index on the idempotency key column means the second insert fails with a duplicate key error rather than creating a duplicate record:

typescript
// MongoDB
OrderSchema.index({ idempotencyKey: 1 }, { unique: true, sparse: true });

This is a defense-in-depth layer, not a replacement for the Redis check. The database constraint catches races the application layer might miss, but it doesn't give you a cached response to return — for that you still need the Redis record.

For operations that span multiple collections or tables, Database Transactions: When You Actually Need Them covers how to combine idempotency with atomicity.

Client-Side Responsibilities

Idempotency keys must be:

• Generated client-side (not server-side) so the same key survives a retry
• Scoped to a single logical operation — don't reuse a key for a different action
• Generated fresh for each new operation, even if it looks similar to a previous one

A good rule: generate the key when the user initiates an action (button click, form submit) and persist it in memory until the operation confirms success. On retry, reuse the same key. On success, discard it.

When Idempotency Keys Are Not Enough

Idempotency keys solve the "same request, multiple times" problem. They don't solve:

• Out-of-order delivery — two different operations arriving in the wrong order
• Concurrent mutations — two users editing the same resource simultaneously (use optimistic locking or ETags)
• At-least-once consumers — message queues that may deliver the same message to multiple workers (use a deduplication ID in the message itself)

For queue-based architectures, see Background Jobs and Queues in Node.js for how idempotency interacts with job retries.

Summary

Idempotency keys are a small addition to your API surface that make a large difference in reliability. The pattern is straightforward: client-generated UUID, server-side Redis check, write a processing lock before executing, cache the final response for 24 hours. Pair it with a database-level unique constraint for defense in depth. Any operation that creates, charges, or sends something should be idempotent.

See How to Design APIs for Next.js Applications for the broader API design context, and visit /about to learn more about the engineering approach behind this portfolio.