Post

Idempotency Explained: Building APIs That Survive Retries

Posted
Preview Image
By Billy Okeyo 8 min read

Imagine you’re purchasing a product online.

You click the “Pay Now” button.

Nothing happens.

After a few seconds, you assume the request failed, so you click the button again.

And again.

A few minutes later, you discover you’ve been charged three times.

What happened?

From the user’s perspective, the payment seemed to fail. From the server’s perspective, however, it successfully processed every request it received.

This is one of the most common problems in distributed systems, and it’s exactly why idempotency exists.

Whether you’re building payment systems, booking platforms, inventory management software, or any API that changes data, retries are inevitable. Networks fail, clients time out, mobile connections drop, and users double-click buttons.

A well-designed API should survive these retries without creating duplicate side effects.

In this article, we’ll explore what idempotency is, why it matters, and how to implement it in your own APIs.

What Is Idempotency?

In simple terms, an idempotent operation can be performed multiple times without changing the final result beyond the first successful execution.

For example:

1
2
3

Turn on the light.
Turn on the light again.
Turn on the light again.

The light is still ON.

Nothing new happened after the first request.

The final state remains the same.

That’s idempotency.

Now compare it with this:

1
2
3

Deposit $100
Deposit $100
Deposit $100

Your account balance increases by $300.

This operation is not idempotent because every request changes the system.

Why Retries Happen

Many developers assume users only send one request.

Reality is different.

Requests are retried because of:

  • Slow internet connections
  • Gateway timeouts
  • Reverse proxies
  • Mobile network interruptions
  • Browser refreshes
  • Double-clicking buttons
  • Client retry mechanisms
  • Load balancers
  • Microservice communication failures

Imagine this timeline:

1
2
3
4
5
6
7
8
9
10
11
12
13

Client -------- POST /payments --------> API

             Payment succeeds

API -------- 200 OK --------X

(Response never reaches client)

Client waits...

Client retries.

POST /payments again.

The client believes the payment failed.

The server already completed it.

Without idempotency…

The payment happens twice.

HTTP Methods and Idempotency

HTTP itself distinguishes between idempotent and non-idempotent methods.

GET

1

GET /users/10

Read the user.

Call it once.

Call it 100 times.

Nothing changes.

Idempotent

PUT

1
2
3
4

PUT /users/10
{
   "name": "Billy"
}

Replacing the same resource repeatedly produces the same result.

Idempotent

DELETE

1

DELETE /users/10

Delete the user.

Deleting an already deleted user doesn’t delete them twice.

The final state is still:

1

User does not exist.

Idempotent

POST

1

POST /orders

Create a new order.

Call it twice.

You now have two orders.

Not idempotent

This is why POST requests often require additional protection.

Why Payment APIs Use Idempotency Keys

Payment providers like Stripe popularized the use of Idempotency Keys.

The idea is simple.

The client generates a unique identifier.

Example:

1
2

Idempotency-Key:
6ab89d3b-acde-4d71-b20d-483d8d0ef091

Every retry sends the same key.

1
2
3
4

POST /payments

Idempotency-Key:
6ab89d3b-acde-4d71-b20d-483d8d0ef091

When the server receives the request:

  1. Check if this key already exists.
  2. If not, process the payment.
  3. Save both the key and the response.
  4. Return the response.

If the same request arrives again with the same key:

Instead of charging the customer again…

Return the previously stored response.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

Client

POST /payments
Key: ABC123

↓

Server

Charge customer

↓

Store

ABC123 → Payment #456

↓

Return success

---

Retry

POST /payments
Key: ABC123

↓

Lookup

ABC123 exists

↓

Return Payment #456

No second charge.

Implementing Idempotency

A common workflow looks like this.

Step 1

Receive request.

1

POST /orders

Headers

1
2

Idempotency-Key:
XYZ987

Step 2

Search database.

1
2
3

SELECT *
FROM idempotency_keys
WHERE key = 'XYZ987'

Found?

Yes.

Return stored response.

Done.

Step 3

Not found?

Create the resource.

1

Create Order

Step 4

Store:

1
2
3
4
5
6
7

Key

Response

Status Code

Timestamp

Now every retry returns the same response.

Example in Node.js (Express)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

app.post("/payments", async (req, res) => {
    const key = req.header("Idempotency-Key");

    const existing = await Idempotency.findOne({ key });

    if (existing) {
        return res.status(existing.status).json(existing.response);
    }

    const payment = await processPayment(req.body);

    await Idempotency.create({
        key,
        status: 201,
        response: payment,
    });

    return res.status(201).json(payment);
});

Python (FastAPI)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

app = FastAPI()

@app.post("/payments")
async def create_payment(request: Request):
    key = request.headers.get("Idempotency-Key")

    existing = await Idempotency.find_one(key=key)

    if existing:
        return JSONResponse(
            content=existing.response,
            status_code=existing.status,
        )

    body = await request.json()
    payment = await process_payment(body)

    await Idempotency.create(
        key=key,
        status=201,
        response=payment,
    )

    return JSONResponse(content=payment, status_code=201)

C# (ASP.NET Core)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

app.MapPost("/payments", async (
    HttpRequest request,
    IdempotencyStore store,
    PaymentService payments) =>
{
    var key = request.Headers["Idempotency-Key"].ToString();

    var existing = await store.FindAsync(key);
    if (existing is not null)
    {
        return Results.Json(existing.Response, statusCode: existing.Status);
    }

    var body = await request.ReadFromJsonAsync<PaymentRequest>();
    var payment = await payments.ProcessAsync(body!);

    await store.CreateAsync(new IdempotencyRecord(key, 201, payment));

    return Results.Json(payment, statusCode: StatusCodes.Status201Created);
});

Go

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

func createPayment(w http.ResponseWriter, r *http.Request) {
    key := r.Header.Get("Idempotency-Key")

    existing, err := idempotency.FindOne(r.Context(), key)
    if err == nil && existing != nil {
        w.WriteHeader(existing.Status)
        json.NewEncoder(w).Encode(existing.Response)
        return
    }

    var body PaymentRequest
    if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    payment, err := processPayment(r.Context(), body)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    if err := idempotency.Create(r.Context(), IdempotencyRecord{
        Key:      key,
        Status:   http.StatusCreated,
        Response: payment,
    }); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(payment)
}

Laravel

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

Route::post('/payments', function (Request $request) {
    $key = $request->header('Idempotency-Key');

    $existing = Idempotency::where('key', $key)->first();

    if ($existing) {
        return response()->json($existing->response, $existing->status);
    }

    $payment = processPayment($request->all());

    Idempotency::create([
        'key' => $key,
        'status' => 201,
        'response' => $payment,
    ]);

    return response()->json($payment, 201);
});

The logic is surprisingly simple.

The complexity comes from storing and managing the keys correctly.

Where Should Idempotency Keys Be Stored?

Options include:

Database

Best for most applications.

Pros:

  • Persistent
  • Reliable
  • Easy to query

Cons:

  • Slightly slower

Redis

Excellent for high-volume APIs.

Pros:

  • Extremely fast
  • TTL support
  • Easy expiration

Many APIs automatically expire keys after 24 hours.

In-Memory

Useful only during development.

Not recommended for production.

Restarting the server loses everything.

Common Mistakes

Reusing Keys

Every logical operation should have its own unique key.

Bad:

1
2
3
4
5

ABC123

used today

used tomorrow

Good:

1
2
3
4
5

New checkout

↓

Generate new UUID

Ignoring Request Differences

Suppose the first request is:

1

$50

The retry is:

1

$500

Same key.

Different body.

The server should reject this request because the key is being reused for a different operation.

Never Expiring Keys

Keeping millions of old keys forever wastes storage.

Most APIs expire them after:

  • 24 hours
  • 48 hours
  • 7 days

depending on business requirements.

Real-World Use Cases

Idempotency is valuable anywhere duplicate requests could have costly consequences.

Examples include:

  • Payment processing
  • Bank transfers
  • Order creation
  • Hotel reservations
  • Flight bookings
  • Ticket purchases
  • Subscription billing
  • Inventory updates
  • Webhook processing
  • Email sending
  • Message queues

If performing the same action twice could create an incorrect outcome, idempotency is worth considering.

When You Don’t Need Idempotency

Not every endpoint needs an idempotency key.

For example:

1

GET /posts

No state changes.

No duplicates.

No problem.

Likewise, endpoints such as:

  • Search
  • Filtering
  • Reading reports
  • Viewing profiles

are already naturally idempotent.

Reserve idempotency mechanisms for operations where retries could create unintended side effects.

Final Thoughts

Idempotency isn’t just an implementation detail—it’s a reliability feature.

In distributed systems, retries are normal. Networks are unreliable, users click buttons more than once, and clients retry requests automatically. Instead of hoping those situations never happen, design your APIs to handle them gracefully.

By using idempotency keys, storing responses, validating retries, and choosing the right storage strategy, you can prevent duplicate orders, repeated payments, and other costly errors.

A resilient API isn’t one that never receives duplicate requests.

It’s one that produces the correct outcome even when duplicate requests inevitably arrive.

The next time you design a POST endpoint, ask yourself:

“What happens if this request is sent twice?”

If the answer is “something bad,” it’s probably time to add idempotency.

API Design, Software Testing
Idempotency API Design Software Testing
This post is licensed under CC BY 4.0 by the author.