Designing a Chat System Like WhatsApp / iMessage
Difficulty: Intermediate Prerequisites:Message Queues, Caching, and WebSockets
TL;DR
A chat system delivers messages in real-time using WebSockets for online users and stores-then-forwards for offline users.
flowchart LR
SENDER["Sender"]:::client
WS["WebSocket Servers"]:::service
CHAT["Chat Service"]:::service
STORE[("Message Store<br/>Cassandra")]:::data
K["Kafka<br/>fan-out"]:::async
PUSH["Push Notifications<br/>FCM APNs"]:::external
RECEIVER["Receiver"]:::client
SENDER -->|"1. Open WebSocket"| WS
WS -->|"2. Forward message"| CHAT
CHAT -->|"3. Persist message"| STORE
CHAT -->|"4. Publish to fan-out"| K
K -->|"5. Fan out"| WS
CHAT -->|"6. Push offline alert"| PUSH
WS -->|"7. Deliver to recipient"| RECEIVER
classDef client fill:#4c3a5e,stroke:#818cf8,color:#e2e8f0
classDef service fill:#1a3a2a,stroke:#4ade80,color:#e2e8f0
classDef async fill:#AB47BC,stroke:#4A148C,color:#fff
classDef data fill:#3b3520,stroke:#fbbf24,color:#e2e8f0
classDef external fill:#4a1942,stroke:#f472b6,color:#e2e8f0
In 3 sentences: Clients maintain a persistent WebSocket connection to the server. When a message is sent, the server persists it, looks up which server the receiver is connected to, and pushes it down their WebSocket. If the receiver is offline, the message waits in a queue and a push notification is sent.
Understanding the Problem
π¬ What is a chat system? A real-time messaging platform that lets users send text, images, and files to individuals or groups. Messages must be delivered reliably (even if the recipient is offline), ordered correctly, and displayed in real-time. Think WhatsApp, Telegram, Facebook Messenger, or Slack. The hard parts: guaranteed delivery across flaky mobile networks, real-time push without polling, group fan-out at scale, and end-to-end encryption.
Naive First Cut
flowchart LR
SENDER["Sender"]:::client
API["Chat API"]:::service
DB[("Messages DB<br/>one table")]:::data
RECEIVER["Receiver<br/>polls every 5s"]:::client
SENDER --> API
API --> DB
RECEIVER --> API
classDef client fill:#4c3a5e,stroke:#818cf8,color:#e2e8f0
classDef service fill:#1a3a2a,stroke:#4ade80,color:#e2e8f0
classDef data fill:#3b3520,stroke:#fbbf24,color:#e2e8f0
Sender POSTs message to an API, stored in a DB. Receiver polls the API every 5 seconds for new messages.
Why this breaks:
- Polling is wasteful - 500M users polling every 5s = 100M QPS of mostly-empty responses. Massive cost, terrible latency.
- 5-second delay feels laggy - real-time chat needs sub-second delivery.
- Single DB for all messages - billions of messages/day, one table collapses.
- No offline handling - if receiver is offline when message arrives, when do they get it?
- No ordering guarantee - if two messages arrive out of order at the DB, display is wrong.
- Group messages multiply the problem - 256-member group = 256 deliveries per message.
Prior Art
- WhatsApp Architecture (InfoQ) - Erlang-based, 2M connections per server, XMPP-derived protocol, store-and-forward for offline delivery.
- Facebook Messenger Iris - ordered log storage (like Kafka) per conversation. Messages appended to a per-user ordered log. Clients sync via sequence numbers.
- Discord How Messages Are Stored - migrated from MongoDB to Cassandra to ScyllaDB. Partition per channel + bucket.
- Signal Protocol - end-to-end encryption with double-ratchet. Pre-keys for offline delivery. The gold standard for E2E chat encryption.
- Slack Real-Time Messaging - WebSocket connections for real-time, application-level edge cache (Flannel) for fast channel hydration.
Technology Choices
| Tier | Purpose | Primary pick | Alternatives |
|---|---|---|---|
| Real-time transport | Push messages to online clients | WebSocket (long-lived) | SSE, MQTT (IoT/mobile-optimized), gRPC streaming |
| Connection management | Track whoβs online on which server | Redis Pub/Sub + connection registry | Kafka, custom session store |
| Message storage | Durable, ordered message log | Cassandra (partition per conversation) | ScyllaDB, DynamoDB, TiDB |
| Message queue | Decouple sender from fan-out | Kafka (per-user topic or partition) | SQS, RabbitMQ, Pulsar |
| Offline delivery | Store messages until recipient connects | Redis sorted set per user | SQS per user, Cassandra unread table |
| Presence | Whoβs online | Redis with TTL per user | Dedicated presence service |
| Media storage | Images, files, voice notes | S3 / GCS with CDN | MinIO, Azure Blob |
| Push notifications | Offline users | FCM + APNs | OneSignal, SNS |
| E2E encryption | Message privacy | Signal Protocol (Double Ratchet) | Custom, Noise Protocol |
Functional Requirements
Core:
- Users can send messages (text) to another user in real-time (1:1 chat).
- Users can create groups and send messages to all group members.
- Messages are delivered reliably even if the recipient is offline (store-and-forward).
Below the line:
- Read receipts, typing indicators
- Media messages (images, video, voice)
- End-to-end encryption
- Message search, reactions, threads
- Voice/video calling
Non-Functional Requirements
Core:
- Real-time delivery - P99 < 500ms for online-to-online message delivery.
- Reliability - zero message loss. Once the server acks, the message WILL be delivered eventually.
- Ordering - messages within a conversation appear in send order.
- Scale - 500M DAU, 100B messages/day (WhatsApp scale).
Below the line:
- Sub-100ms delivery latency
- Exactly-once delivery (at-least-once + client-side dedupe is acceptable)
- Multi-device sync (web + mobile + desktop)
Scale Estimation (Back-of-Envelope)
- Users: 500M DAU, 50M concurrent connections at peak
- Write QPS: 100K messages/sec sustained, 10B messages/day
- Read QPS: 200K message fetches/sec (history sync + offline drain)
- Storage: ~5TB message storage/year (compressed, Cassandra)
- Bandwidth: ~500 Gbps aggregate WebSocket traffic at peak
Core Entities
- User - identified by phone number or userId. Has online/offline status.
- Conversation - a 1:1 or group thread. Has a unique
conversationIdand list of participants. - Message - text content with
messageId,senderId,conversationId,timestamp,status(sent/delivered/read). - Connection - a live WebSocket session mapping
userId β serverId:connectionId.
API / System Interface
WebSocket: wss://chat.example.com/ws
β Client authenticates on connect (JWT)
β Bidirectional: send messages, receive messages, typing, presence
REST (fallback + media):
POST /v1/messages β send a message (fallback if WS down)
GET /v1/conversations/:id/messages?after=<seqNo> β sync history
POST /v1/media/upload β upload image/file, get a mediaUrl
POST /v1/groups β create group
Wire format (over WebSocket):
{"type": "message", "to": "conv_123", "text": "hello", "clientMsgId": "uuid"}
{"type": "ack", "messageId": "msg_456", "status": "delivered"}
{"type": "typing", "conversationId": "conv_123", "userId": "u_789"}
Security: WebSocket authenticated via JWT on handshake. clientMsgId is for client-side dedupe (idempotency). Server generates the authoritative messageId and timestamp.
High-Level Design
1) User sends a 1:1 message (both online)
New components we need:
- WebSocket Servers - maintain persistent two-way connections with every online user.
π‘ WebSocket = a persistent connection that stays open so the server can push messages instantly without the client asking. Unlike HTTP (ask β answer β done), WebSocket keeps the line open. Learn more β - Chat Service - the brain. Receives messages, persists them, and figures out where the recipient is connected.
- Message Store (Cassandra) - permanent storage for all messages. Partitioned by conversation so βload chat historyβ is a single partition read.
π‘ Cassandra is a distributed wide-column database designed for heavy writes. Partitioning by conversation_id means loading a chat history is a single-partition read - O(1) regardless of total messages in the system. - Connection Registry (Redis) - a fast lookup table mapping
userId β which WebSocket server they're on. When a message arrives for Bob, we check Redis to find which server is holding Bobβs connection.
flowchart LR
SENDER["Sender"]:::client
WS1["WebSocket Server A"]:::service
CHAT["Chat Service"]:::service
STORE[("Message Store<br/>Cassandra")]:::data
ROUTE["Connection Registry<br/>Redis"]:::data
WS2["WebSocket Server B"]:::service
RECEIVER["Receiver"]:::client
SENDER -->|"1. Open WebSocket"| WS1
WS1 -->|"2. Forward message"| CHAT
CHAT -->|"3. Persist message"| STORE
CHAT -->|"4. Lookup receiver server"| ROUTE
ROUTE -->|"5. Route to Server B"| WS2
WS2 -->|"6. Deliver to recipient"| RECEIVER
classDef client fill:#4c3a5e,stroke:#818cf8,color:#e2e8f0
classDef service fill:#1a3a2a,stroke:#4ade80,color:#e2e8f0
classDef data fill:#3b3520,stroke:#fbbf24,color:#e2e8f0
Step-by-step flow:
- Sender types βHey, are you free tonight?β and hits send β message travels over their open WebSocket connection to Server A
- Server A forwards the message to the Chat Service
- Chat Service persists the message to Cassandra (partition key =
conversationId, so all messages in a chat live together) - now itβs durable, even if everything crashes - Chat Service asks Redis: βWhich server is the receiver connected to?β β answer: Server B
- Chat Service pushes the message to Server B (via internal gRPC or pub/sub)
- Server B pushes the message down the receiverβs WebSocket β message appears on their screen instantly
- Receiverβs app sends back a
deliveredacknowledgment β this receipt flows back to the sender so they see the double-check ββ
Why WebSocket instead of HTTP polling? With polling, each user would hit our servers every 2 seconds asking βany new messages?β - for 500M users, thatβs 250M requests/second of mostly-empty responses. WebSocket keeps a persistent connection open so the server pushes messages the instant they arrive - zero wasted requests, sub-second delivery.
2) Receiver is offline - store and forward
New components we need (in addition to the ones above):
- Offline Queue (Redis sorted set) - when the receiver isnβt connected, we park message IDs here. Scored by sequence number so when they reconnect, we drain messages in perfect order.
- Push Service - sends push notifications to wake up the userβs phone.
π‘ Think of it as the βtap on the shoulderβ that tells the user to open the app. - FCM / APNs - Firebase Cloud Messaging (Android) and Apple Push Notification service (iOS). External services that deliver notifications to locked phones.
π‘ FCM doesnβt βknowβ a message arrived - YOUR server tells FCM to send the push. When Bob installs the app, FCM gives his device a unique token. Your server stores this token. When Bob is offline and a message arrives, your server calls FCMβs API with Bobβs token + notification content. FCM maintains its own persistent connection to every Android device in the world and routes the push through that always-on channel. APNs works the same way for iOS. Learn more about real-time communication β
How does the notification show the actual message text (with E2E encryption)?
For E2E encrypted apps like WhatsApp/Signal, FCM does NOT carry the message content (the server canβt read it). Instead:
- Server sends a silent data message via FCM - just a βwake up, you have a new messageβ signal with sender ID and message reference
- FCM wakes up the appβs background process on the device
- The app connects to the server, pulls the encrypted message, and decrypts it locally on the device
- The app constructs the notification itself (βAlice: Hey, are you free?β) and hands it to the OS for display
For non-E2E apps, the server CAN send the message text directly in the FCM payload (notification message type) - simpler but less secure.
flowchart LR
SENDER["Sender"]:::client
CHAT["Chat Service"]:::service
STORE[("Message Store")]:::data
OFFLINE[("Offline Queue<br/>Redis sorted set")]:::data
PUSH["Push Service"]:::service
FCM["FCM and APNs"]:::external
SENDER -->|"1. Send message"| CHAT
CHAT -->|"2. Persist message"| STORE
CHAT -->|"3. Queue for offline user"| OFFLINE
CHAT -->|"4. Trigger push alert"| PUSH
PUSH -->|"5. Deliver via FCM APNs"| FCM
classDef client fill:#4c3a5e,stroke:#818cf8,color:#e2e8f0
classDef service fill:#1a3a2a,stroke:#4ade80,color:#e2e8f0
classDef data fill:#3b3520,stroke:#fbbf24,color:#e2e8f0
classDef external fill:#4a1942,stroke:#f472b6,color:#e2e8f0
Step-by-step flow:
- Chat Service checks the Connection Registry β receiver is NOT online (no WebSocket entry found)
- Message is still persisted to Cassandra (same as before - always store first, deliver second)
- Message ID is added to the receiverβs offline queue in Redis (sorted by sequence number for ordering)
- Push Service sends a notification via FCM/APNs: βNew message from Aliceβ β phone buzzes
- Later, receiver opens the app and reconnects via WebSocket β server drains the offline queue, sending all pending messages in order
- Receiverβs app acknowledges each message β server removes them from the offline queue
Why store-and-forward instead of just βretry laterβ? Mobile networks are unreliable. A user might be offline for hours (on a flight, in a tunnel, phone dead). Store-and-forward guarantees zero message loss - once the server acknowledges receipt from the sender, that message WILL reach the recipient eventually, no matter how long it takes.
3) Group message fan-out
New components we need (in addition to the ones above):
- Kafka - an event bus for group message fan-out.
π‘ We use Kafka here because group messages need to be delivered to N members reliably. If a fan-out worker crashes mid-delivery, Kafka retries automatically - no message gets lost. Learn more β - Fan-out Workers - consume group message events and deliver to each member individually (online β push via WebSocket, offline β queue + push notification).
flowchart LR
SENDER["Sender"]:::client
CHAT["Chat Service"]:::service
STORE[("Message Store")]:::data
K["Kafka<br/>fan-out topic"]:::async
FAN["Fan-out Workers"]:::service
WS["WebSocket Servers"]:::service
MEMBERS["Group Members"]:::client
SENDER -->|"1. Send group message"| CHAT
CHAT -->|"2. Store single copy"| STORE
CHAT -->|"3. Publish fan-out event"| K
K -->|"4. Process group delivery"| FAN
FAN -->|"5. Push to online members"| WS
WS -->|"6. Deliver to each member"| MEMBERS
classDef client fill:#4c3a5e,stroke:#818cf8,color:#e2e8f0
classDef service fill:#1a3a2a,stroke:#4ade80,color:#e2e8f0
classDef async fill:#AB47BC,stroke:#4A148C,color:#fff
classDef data fill:#3b3520,stroke:#fbbf24,color:#e2e8f0
Step-by-step flow:
- Sender sends a message to group
conv_123(256 members) β hits Chat Service - Chat Service stores ONE copy of the message (partition key =
conv_123) - not 256 copies! - Publishes a fan-out event to Kafka: βdeliver message M to these 256 membersβ
- Fan-out workers consume the event and look up each memberβs connection - online members get real-time WebSocket delivery, offline members get the offline queue + push notification treatment
- If a fan-out worker crashes, Kafka retries - at-least-once delivery is guaranteed
Why store once, fan-out on delivery? Writing 256 copies of the same message would waste massive storage. Instead, we store one copy and fan out references (message IDs) to each memberβs timeline. This also makes edits and deletes trivial - change one row, and everyone sees the update.
Potential Deep Dives
Deep Dive 1 - How to handle 2M WebSocket connections per server
Problem: A chat system with 50M concurrent users needs to maintain 50M persistent WebSocket connections. If each server handles 500K connections, thatβs 100 servers just for connection holding. The challenge: each connection is stateful (long-lived TCP), consumes memory, and requires efficient event handling.
Bad - one thread per connection (Java BIO / traditional blocking I/O).
Classic Java ServerSocket.accept() β spawn a thread per client. At 10K threads you hit OS limits, context-switching overhead makes the CPU thrash, and each thread stack takes ~512KB. 2M threads Γ 512KB = 1TB RAM. Impossible.
Good - NIO event loop model (Netty, Node.js, Go goroutines).
Instead of one thread per connection, use a small pool of threads (event loops) that multiplex thousands of connections using OS-level I/O selectors (epoll on Linux, kqueue on macOS).
What Netty is: An asynchronous, event-driven network framework for Java. It implements the Reactor pattern - a single thread monitors many sockets, and only wakes up when thereβs data to read/write. No blocking, no idle threads.
Event Loop (1 thread) monitors 100K connections via epoll
β Connection has data? β Read it, process, respond
β Connection idle? β Costs nothing (just a file descriptor)
Real numbers:
- Each idle WebSocket = ~10KB RAM (file descriptor + small read/write buffers)
- 2M connections Γ 10KB = 20GB RAM (fits in a 64GB server)
- Netty can handle 1-2M connections per JVM instance on modern hardware
- Goβs goroutines achieve similar density (goroutine = ~4KB stack vs Java thread = 512KB)
Tech used in production:
- WhatsApp: Erlang/OTP (lightweight processes, similar to goroutines - famously ran 2M connections per server)
- Discord: Elixir/Erlang on the gateway, Rust for hot paths
- Slack: Java + Netty for WebSocket gateway (project βFlannelβ)
- Signal: Java + Netty
- WeChat: C++ custom framework
Great - tiered architecture separating connection from logic.
At extreme scale (100M+ connections), even Netty hits limits on a single machine. The problem: the server holding connections ALSO processes messages (routing, persistence, fan-out). Under load, message processing slows down AND connection handling suffers - they compete for the same CPU/memory.
The solution: split into two independent layers, each doing one job.
Edge Tier (connection holding) - the βreceptionistβ:
- ONLY manages TCP/WebSocket connections, TLS handshake, and heartbeat pings
- Does NO business logic - just holds open connections and passes messages through
- Extremely lightweight: each connection costs ~10KB (just a file descriptor + buffer)
- Can hold 2M+ connections per node because itβs barely doing anything per connection
- Built with: Envoy proxy, HAProxy, custom Go/Rust services, or Netty with minimal handlers
Logic Tier (message processing) - the βbrainβ:
- Receives raw messages from edge tier via internal gRPC
- Handles all business logic: routing, persistence to Cassandra, fan-out to group members, push notifications
- Stateless - scales horizontally based on message throughput
- Doesnβt hold any WebSocket connections - just processes and responds
How a message flows through both tiers:
- Aliceβs phone is connected to Edge Server #3 via WebSocket
- Alice sends βHey Bobβ β Edge Server #3 receives the raw bytes
- Edge Server #3 forwards to Logic Tier via internal gRPC: βmessage from userId=alice, payload=Hey Bobβ
- Logic Tier stores in Cassandra, then checks Connection Registry: βBob is on Edge Server #7β
- Logic Tier sends to Edge Server #7: βdeliver this to Bobβs WebSocket connectionβ
- Edge Server #7 pushes the message down Bobβs WebSocket
- If Bob is offline β Logic Tier calls Push Service instead (FCM/APNs)
The Connection Registry (Redis) ties both tiers together:
Redis Hash: connection_registry
alice β edge-server-3:conn-8842
bob β edge-server-7:conn-1204
carol β edge-server-3:conn-9921
When Logic Tier needs to deliver to Bob, it looks up this registry and routes to the correct edge server. When Bob disconnects, Edge Server #7 removes the entry.
Why this is better than one server doing everything:
- Adding more connections = adding cheap, lightweight edge nodes (no processing overhead)
- A slow DB write in Logic Tier doesnβt block Edge Tier from handling new connections/pings
- If an edge node crashes: clients reconnect to another edge node. No messages are lost (Logic Tier handles durability separately)
- During idle hours (3 AM): connections exist but messages are rare. Edge handles the load efficiently, Logic Tier is mostly idle
Real-world implementations:
- WhatsApp: Erlang nodes at edge, backend services for routing/storage
- Discord: βGatewayβ servers (Elixir) hold connections, βGuildβ servers handle message logic
- Slack: βFlannelβ is their edge/cache layer, backend services do the real work
Deep Dive 2 - Message ordering in distributed systems
Problem: Alice sends βHelloβ then βHow are you?β in quick succession. These hit different server instances (load balanced). Or one arrives via WebSocket, another via a retry. Bob must see them in the correct order. Out-of-order messages make conversations nonsensical.
Why this is hard: In a distributed system, thereβs no global clock. Server Aβs timestamp might be 50ms ahead of Server B. Network latency varies. Messages can be retried out of order.
Bad - rely on server timestamps.
Each server stamps the message with System.currentTimeMillis() on arrival. Sort by timestamp on display.
Fails because:
- Clock skew between servers (NTP syncs every few seconds, drift is 10-50ms)
- Aliceβs βHelloβ hits Server A at T=1000, βHow are you?β hits Server B whose clock reads T=999. Bob sees them reversed.
- Even on one server, if two messages arrive in the same millisecond, order is random.
Good - per-conversation monotonic sequence number.
Assign a strictly increasing seqNo per conversation. Every message in a conversation gets the next number in sequence.
Implementation: Redis INCR on key conv_seq:{conversationId}.
Alice sends "Hello" β server does INCR conv_seq:alice_bob β gets 42
Alice sends "How are you?" β server does INCR conv_seq:alice_bob β gets 43
Bobβs client sorts by seqNo regardless of arrival order. Even if msg 43 arrives before 42 (network jitter), the UI holds 43 and renders after 42 arrives.
Why Redis INCR? Atomic, single-threaded, sub-ms. Even at 100K messages/sec across all conversations, one Redis cluster handles it because each conversation is an independent key (no contention across conversations).
What about gaps? If Bob receives seqNo 42 then 44 (missed 43), client knows thereβs a gap and requests: βgive me message 43 for this conversation.β Server fetches from the message store.
Great - sequence numbers + client vector clock + multi-device sync.
For apps with multiple devices (phone + web + desktop), ordering gets harder. User sends from phone (seqNo 42), then from desktop (seqNo 43). Both devices need to converge.
The approach (used by WhatsApp, Slack, Facebook Messenger):
- Server is the source of truth for sequence numbers. Server assigns seqNo on receipt - NOT the client.
- Each device maintains a cursor:
lastSyncedSeqNo. On reconnect, device says βgive me everything after seqNo 38β and server sends the delta. - Client embeds
lastSeenSeqNoin outgoing messages so the server can detect if the client missed something and proactively push missing messages. - Conflict resolution for near-simultaneous sends from multiple devices: Both get seqNos from the same atomic counter, so theyβre naturally ordered by who hit the server first. No conflict possible at the ordering level.
Tech used in production:
- WhatsApp: Server-assigned message IDs + per-chat ordering. Each message has a globally unique ID + per-conversation sequence.
- Slack: Uses a
ts(timestamp) as the unique message ID within a channel. Server-generated, monotonically increasing per channel. Format:1234567890.123456. - Discord: Snowflake IDs (time-based, globally unique). Messages sorted by Snowflake ID which is inherently time-ordered since timestamp is the most significant bits.
Deep Dive 3 - Reliable delivery with at-least-once + client dedupe
Problem: Network is unreliable. Message might be delivered twice if the ack is lost.
In simple terms: The internet is flaky. A message might arrive twice if the βgot itβ confirmation gets lost. We need to make sure Bob sees each message exactly once, even if the system retries delivery.
Flow:
Sender β Server: message (clientMsgId: "abc")
Server β Sender: ack (messageId: "msg_1", clientMsgId: "abc")
Server β Receiver: message (messageId: "msg_1")
Receiver β Server: delivered ack (messageId: "msg_1")
What if receiverβs ack is lost? Server retries delivery. Receiver sees msg_1 twice. Client dedupes by messageId - if already in local DB, ignore.
What if senderβs send is retried? Server checks clientMsgId: "abc" against a short-lived dedupe cache. If seen, returns the same messageId without re-storing.
Result: at-least-once from server side, exactly-once from userβs perspective (client dedupe).
Deep Dive 4 - How to sync message history across devices
Problem: User has phone + web + desktop. All three must show the same messages.
In simple terms: You send a message from your phone. When you open WhatsApp on your laptop 5 minutes later, that same message should be there. All your devices need to stay in sync.
Solution: pull-based sync with sequence numbers.
- Each conversation has a
maxSeqNo. - Each device tracks
lastSyncedSeqNoper conversation. - On app open, device sends
GET /conversations/:id/messages?after=lastSyncedSeqNo. - Server returns the delta. Device applies locally.
- Real-time messages come via WebSocket; device increments its local seqNo on receipt.
This is the βordered logβ model (Facebook Iris). The server is the source of truth; clients are materialized views with a cursor.
Deep Dive 5 - Group fan-out: write amplification vs read amplification
Write amplification (push model):
In simple terms: When you send a message to a 500-person group, should we write 500 copies (one per member) or write one copy and let each member fetch it? Each approach has trade-offs.
- On group message, write a copy to each memberβs inbox.
- 256-member group Γ 1000 messages/day = 256K writes/day for one group.
- Pro: reads are fast (each user reads their own inbox).
- Con: massive write cost at scale. Celebrity groups with 100K members are catastrophic.
Read amplification (pull model):
- Store one copy per conversation.
- On read, userβs client fetches from the conversationβs log.
- Pro: one write per message regardless of group size.
- Con: each read must merge multiple conversationsβ logs.
Hybrid (what WhatsApp/Discord do):
- Small groups (β€256): push model. Fan-out is bounded and fast.
- Large channels (1000+): pull model. Store in channel log, clients fetch on demand.
- Threshold is configurable per platform.
Final Architecture
flowchart TD
CLIENTS["Mobile and Web Clients"]:::client
LB["Load Balancer<br/>sticky by userId"]:::edge
WS["WebSocket Servers<br/>Netty edge tier"]:::service
CHAT["Chat Service"]:::service
REG[("Connection Registry<br/>Redis")]:::data
STORE[("Message Store<br/>Cassandra")]:::data
OFFLINE[("Offline Queue<br/>Redis sorted set")]:::data
K["Kafka<br/>fan-out and events"]:::async
FAN["Fan-out Workers"]:::service
PUSH["Push Service"]:::service
MEDIA[("S3 and CDN<br/>media")]:::data
FCM["FCM and APNs"]:::external
CLIENTS -->|"Open WebSocket"| LB
CLIENTS -->|"Presigned upload"| MEDIA
LB -->|"Sticky route by user"| WS
WS -->|"Forward to chat logic"| CHAT
CHAT -->|"Lookup receiver server"| REG
CHAT -->|"Persist message"| STORE
CHAT -->|"Queue for offline user"| OFFLINE
CHAT -->|"Publish group fan-out"| K
K -->|"Process group delivery"| FAN
FAN -->|"Push to online members"| WS
CHAT -->|"Trigger push alert"| PUSH
PUSH -->|"Deliver via FCM APNs"| FCM
classDef client fill:#4c3a5e,stroke:#818cf8,color:#e2e8f0
classDef edge fill:#1e3a5f,stroke:#60a5fa,color:#e2e8f0
classDef service fill:#1a3a2a,stroke:#4ade80,color:#e2e8f0
classDef async fill:#AB47BC,stroke:#4A148C,color:#fff
classDef data fill:#3b3520,stroke:#fbbf24,color:#e2e8f0
classDef external fill:#4a1942,stroke:#f472b6,color:#e2e8f0
How it works end-to-end:
- Client opens WebSocket β connects through Load Balancer (sticky by userId) to a WebSocket Server
- Sender sends message β WebSocket Server forwards to Chat Service
- Chat Service persists message β writes to Cassandra (Message Store) with a per-conversation sequence number
- Connection Registry checked β Redis lookup finds which WebSocket Server holds the recipient
- Kafka fan-out for groups β message event published to Kafka, Fan-out Workers push to each memberβs WebSocket Server
- Recipient online β message delivered in real-time through their WebSocket connection
- Recipient offline β message queued in Redis sorted set (Offline Queue) and push notification sent via FCM/APNs
- Recipient reconnects β drains Offline Queue in order, syncs from last seen sequence number
Summary
| Decision | Choice | Why |
|---|---|---|
| Transport | WebSocket | Real-time bidirectional, sub-second delivery |
| Message store | Cassandra | Partition per conversation, append-only, handles billions |
| Connection registry | Redis | Sub-ms lookup of βwhich server has user Xβ |
| Offline delivery | Redis sorted set + push notification | Ordered drain on reconnect |
| Group fan-out | Kafka β workers | Async, retryable, doesnβt block sender |
| Ordering | Per-conversation sequence number | Simple, no clock dependency |
| Delivery guarantee | At-least-once + client dedupe | Zero message loss, no duplicates visible to user |
| Multi-device | Pull sync with seqNo cursor | Ordered log model (Facebook Iris) |
Key Technologies Mentioned
| Term | What it is |
|---|---|
| WebSocket | A persistent two-way connection between client and server. Unlike HTTP (request β response β done), WebSocket stays open so the server can push messages to the client anytime. |
| Cassandra | A distributed NoSQL database optimized for fast writes. Stores data across many machines. Perfect for append-only message logs. |
| Kafka | A distributed event streaming platform. Producers write events, consumers read them. Used here to decouple message sending from delivery fan-out. |
| Redis | In-memory key-value store (< 1ms reads). Used here for connection registry (which user is on which server) and offline message queues. |
| FCM / APNs | Firebase Cloud Messaging (Android) and Apple Push Notification service (iOS). How you send push notifications to phones when the app is closed. |
| Sequence Number | A monotonically increasing integer per conversation. Guarantees message ordering regardless of clock differences between servers. |
| Store-and-Forward | Pattern where the server stores a message durably first, then delivers it when the recipient is available. Ensures zero message loss. |
| Fan-out | Delivering one message to multiple recipients (group chat). βFan-out on writeβ = copy to each inbox. βFan-out on readβ = store once, each client fetches. |
Whatβs Expected at Each Level
This section helps you calibrate your depth. You donβt need to cover everything - just know whatβs expected for your level.
Mid-level
Design basic 1:1 messaging with a server relaying messages. Propose WebSocket for real-time delivery. Understand offline message storage and why polling is wasteful. With prompting, discuss how to handle group messages by fanning out to multiple recipients.
Senior
Propose Cassandra for message storage (partition by conversation). Explain connection-level routing - how does a message find the right WebSocket server? Discuss read receipts, message ordering guarantees (per-conversation sequence numbers), and offline delivery queues. Articulate why eventual consistency is acceptable for message delivery.
Staff+
Address end-to-end encryption key exchange (Signal protocol double-ratchet), multi-device sync with ordered-log cursors, and message fan-out for large groups (1000+ members) using the hybrid push/pull model. Discuss graceful degradation when the chat service is overloaded (backpressure on WebSocket connections). Cover message retention policies and GDPR right-to-deletion across replicated stores.
π― Key Takeaways
- WebSocket for real-time delivery - persistent connection, server pushes
- Cassandra for message storage - partitioned by conversation for fast reads
- Store-and-forward for offline users - deliver when they reconnect
- Connection registry in Redis routes messages to the right WebSocket server
Related Designs
- Notification System - similar multi-channel delivery + WebSocket patterns
- Twitter Feed - fan-out and real-time updates
- Stock Broker - Kafka event streaming + exactly-once semantics
Related Concepts
Understand the building blocks used in this design:
- WebSockets vs SSE β β persistent connections deliver messages in real time both ways
- Message Queues β β buffers and routes messages between senders and recipients
- Fan-Out Patterns β β delivers a single group message to every member of the conversation
- Consistent Hashing β β maps each user to a connection server so messages find the right socket
Discussion
Newest first