aleksdolgop.dev

What's coming

I've spent seven years writing backends for systems where mistakes cost money. Payment flows, crypto, high load. Along the way I collected a list of techniques I first saw in large systems and later used in my own work.

That's what I'm going to write about.

Every breakdown follows the same shape. First the problem, then the solution any decent developer would write. Then I show at what scale it stops working and what to do instead. I always end with what the technique costs you, because none of them are free.

Most breakdowns come with a repository. You can run it and watch the naive version fall apart under load.

The first six posts

I'm starting with techniques I picked up from Telegram. The reason is simple: there they run for hundreds of millions of users, not in a diagram from someone's slide deck. I spent five years writing a server-side implementation of their protocol, so I know where the hard parts actually are.

  1. Your client was offline for a day. Now what?

    Everyone building mobile apps, dashboards or integrations runs into this. We'll look at five implementations: Telegram, Binance, Bybit, VK, X. All different, each with its own price. And we'll talk about why duplicates during recovery are unavoidable no matter which one you pick.

  2. Idempotency: what the reference model leaves out

    Everyone knows how Stripe does it. But it says nothing about what happens when the response never reaches you and never will. The operation may have gone through, or not. Let's see how that gap gets closed.

  3. Channels vs. chats

    Why broadcasting to a million subscribers is built completely differently from a conversation between two people. Plus why you can link to a message in a channel but not to one in a private chat.

  4. Upload once, send a hundred times

    About files. Uploading separated from sending, deduplication, resumable transfers. And a neat trick: serving content through someone else's CDN so the CDN operator can't read it.

  5. The API changes, the clients don't

    The whole schema gets versioned, not the endpoint paths. That's how dozens of client versions coexist for years without anyone maintaining three parallel APIs.

  6. Request batches and dependencies between them

    You send ten requests, the third depends on the first. The server will run them in whatever order it likes. There's a way to say “this one after that one”, and you rarely see it in REST.

After that

Techniques from other systems: exchanges, social networks, payment gateways.

Hybrid fan-out and the question of where to draw the line between a regular author and a popular one. The check-then-insert race and three ways to kill it. Counters you can't count exactly, and how to explain that to your product manager. Rate limits measured in weights instead of request counts. Request signing with a validity window. Absolute values instead of deltas.

Assembled modules

Not isolated techniques but working pieces in full. I'll put them on GitHub, feel free to fork.

File service. Post feed. Chat. Payment module.

One caveat: these will be references, not libraries. I can't maintain them for years and won't pretend otherwise. But you can see how they're built and take what you need.

In parallel

I'm writing an open MTProto implementation. Telegram published the protocol and the clients but not the server, and there still isn't an open one.

Doing it layer by layer: schema codegen first, then crypto, transport, and the server itself. Progress goes in the channel, code on GitHub.

The order isn't fixed

Nothing here is set in stone. If you need a specific topic sooner, say so in the comments or in the chat. I'll move it up.

And if you're working on something similar right now and a breakdown is missing a piece, tell me. I'll add it.

Telegram GitHub me@aleksdolgop.dev