Roadmap FinTech

Part 10 · FinTech engineering

APIs, idempotency and reliabilityواجهات البرمجة والموثوقية APIs

In payments the connection will drop mid-operation. The customer's app does not know if it worked, so it tries again, and without protection they get charged twice.

4 minute read

Share this:

An API is the road your software talks to somebody else's software over. In money systems it carries an extra requirement ordinary websites do not have: knowing for certain whether money moved or not.

A network timeout does not say whether an operation ran. It says only that no answer arrived.

The connection dropped while I was paying. Was I charged or not?

Every user has lived this, and it is also the hardest problem in payments engineering.

Picture your app: it sent a payment request to the bank, and the network died before the reply arrived. What happened? You do not know. The request may never have arrived. Or it may have arrived, executed, and had its reply lost on the way back.

Both possibilities look completely identical from your side. Nothing you hold distinguishes them.

So what do you do? Retry, and you may charge the customer twice. Do not retry, and you may leave a payment incomplete and a customer waiting.

The solution the industry settled on is elegant: send a unique identifier with every request, generated by you. When you retry, you send the same identifier. The other side sees it has already processed that identifier and returns the previous result instead of executing again.

The property is called idempotency (الجمود), and it means repeating a request does not change the outcome.

Which gives the rule that governs this whole area: do not assume silence means failure. Silence means you do not know, and that is a third state your system has to handle explicitly.

Design for an unknown result

A customer app sends a SAR 100 debit. The server executes it, but the response connection breaks. The app sees an error and retries. Without protection, one outage costs the customer SAR 200.

Forbidding retries does not solve this. The first request might have failed before arrival, so a retry (إعادة المحاولة) is necessary. It must be safe whether the first attempt ran or not.

An idempotency key (مفتاح عدم التكرار) is a unique reference attached to the operation. The app creates a value such as pay_8f31. The server stores the key and result. A repeat returns that result without moving money again.

Fix the meaning of the key

Bind the key to customer, operation type, and inputs. Reusing it with a different amount should return a conflict, not silently create another operation.

Store an in-progress state as well. A retry can arrive while the first worker is active. The server should wait or report pending instead of starting a second worker. Retain keys longer than the expected retry window.

Generate one unpredictable key per financial intent, not per button click. The key does not replace authentication.

Retry with discipline

A retry sends the same request again after failure or timeout. Immediate endless retries turn a small provider outage into traffic that delays recovery.

Increase delay between attempts with a little randomness. Set a maximum and move exhausted work into later review. Do not retry a clear business decline such as insufficient funds.

Separate transient error, business rejection, and unknown result. A 500 may permit an idempotent retry. A 400 needs corrected input. A disconnected response may require a status query. Do not hide those states behind one generic failure.

Do not trust webhook (خطّاف الويب) delivery

A webhook is a message a provider sends when an event changes. It avoids constant polling but is neither guaranteed nor ordered.

It may arrive late, twice, or after a newer event. It may never arrive. Store event identifiers, ignore duplicates, and compare event time with current state. An old message must not move a settled payment back to pending.

Verify the signature before trusting content. The provider signs the body and time with a shared secret. Your server recalculates and safely compares it. An unsigned “payment succeeded” endpoint lets anyone claim free goods.

Check timestamp freshness to prevent replay of an old valid message. Retain the original payload for investigation under the privacy policy, without logging secrets or full card details.

Confirm with the source

For high-value final changes, query the provider by identifier after a success notification. Confirm amount, currency, account, and status. This stops a malformed integration from producing the wrong ledger entry.

Run a periodic job that checks open operations with the provider. It recovers missed webhooks. Cases pending beyond a defined age become owned exceptions.

Polling frequency should match operation timing and provider limits. Instant payments need seconds; bank settlement may need hours.

Isolate a failing partner

One provider can fail while balance and history remain available. Use short timeouts, connection limits, and queues for delay-tolerant work rather than blocking every screen.

A dead-letter queue holds messages that exhausted retries. It needs alerts, an owner, and a safe replay procedure. Preserve identifiers and context without unnecessary secrets.

Define what stops when risk infrastructure fails. New transfers may pause while account viewing remains available. Never turn an outage into universal approval.

Give customers an honest state

Do not show “failed” when the outcome is unknown. Show “checking,” prevent a new financial intent, and provide a support reference. Update the state after confirmation.

Notify customers if success arrives after they leave. On final failure, release any hold and provide an actionable reason. Support needs a direct lookup path.

Test failure deliberately

Break connectivity before arrival, after provider execution, before local storage, and during webhook handling. Deliver events in reverse order and twice. Every test should end with one movement and an explainable state.

Connect request ID, idempotency key, provider ID, and ledger ID in one trace. Engineers can follow it across services, and auditors can prove a retry did not duplicate money.

A reliable API treats network failure as routine. It relies on stable operation identity, queryable results, and history that never repeats the financial effect.

New words here

Idempotency keyمفتاح عدم التكرار
A unique reference sent with a request so that repeating it does not repeat the action.
Webhookخطّاف الويب
A message another service sends to your server when something happens, instead of you constantly asking.
Retryإعادة المحاولة
Sending the same request again after a failure or timeout. Safe only if the operation is idempotent.

The short version

  • Assume every request may arrive twice. Design so that the second time changes nothing.
  • Always verify webhook signatures. Never trust a payment notification you cannot prove came from the provider.
  • Webhooks arrive late, out of order, more than once, or not at all. Confirm against the provider's API rather than trusting delivery.
  • Keep failed messages in a separate queue so a broken partner cannot silently lose events you will need to explain.

Check your understanding

Answer before reading the explanation. Getting one wrong here is worth more to you than a lucky guess.

  1. 1.You sent a payment request and the network dropped before the reply arrived. Was the customer charged?
  2. 2.You retry with the same idempotency key. What should the server do?
  3. 3.The same idempotency key arrives with a different amount. What is the correct response?
  4. 4.A webhook arrives saying a payment was reversed. Can you act on it directly?
  5. 5.The result is genuinely unknown. What do you show the customer?

This is a learning guide built from field experience. It is not legal, regulatory or investment advice, and it does not replace qualified professional counsel. FinTech rules change quickly and vary by country, so check the primary source before you make a decision that matters.

Back to the roadmap