Caching Strategies Explained: Cache-Aside, Write-Through and Write-Back
Caching strategies and their trade-offs: cache-aside, read-through, write-through, write-back and write-around, plus eviction policies, TTLs and invalidation.
Kafka vs RabbitMQ vs SQS
Kafka, RabbitMQ and Amazon SQS compared: log vs queue models, ordering, replay, throughput, delivery guarantees and operations, and when to use each.
Key takeaways
- Kafka is a durable, replayable log built for high-throughput event streaming.
- RabbitMQ is a flexible message broker with rich routing for task queues and messaging patterns.
- SQS is a fully managed queue with almost no operations, ideal for decoupling services on AWS.
On this page
Message systems let services communicate asynchronously: a producer sends a message and moves on, and consumers process it later. Kafka, RabbitMQ and SQS are three of the most popular choices, and they are built on different ideas.
The key difference: log vs queue
- A queue (RabbitMQ, SQS) delivers each message to a consumer, which acknowledges it, and then the message is gone.
- A log (Kafka) appends messages to a durable, ordered log. Logs suit fan-out workloads such as news feeds, where many consumers read the same events. Consumers track their own position and can re-read history. Many consumer groups can read the same data independently.
Comparison
| Kafka | RabbitMQ | SQS | |
|---|---|---|---|
| Model | Partitioned, replayable log | Broker with exchanges and queues | Managed queue |
| Ordering | Per partition | Per queue (with caveats) | Standard: best effort; FIFO queues: ordered |
| Replay | Yes, within retention | Not by default | No |
| Throughput | Very high | High | High, scales automatically |
| Routing | Topics and partitions | Rich: direct, topic, fanout, headers | Simple; combine with pub/sub services |
| Operations | Significant (unless managed) | Moderate | Minimal |
When to use Kafka
- Event streaming and event-driven architectures
- Many independent consumers of the same data
- Replaying history to rebuild state; see event sourcing and CQRS
- High-volume logs, metrics and analytics pipelines
When to use RabbitMQ
- Background job and task queues
- Complex routing between services
- Request/reply messaging and priorities
- Lower-latency messaging at moderate scale
When to use SQS
- Decoupling services on AWS with minimal operational effort
- Buffering spikes in load between producers and workers
- Simple work queues where replay isn’t needed
Delivery guarantees
All three can deliver messages at least once, which means duplicates are possible. Design consumers to be idempotent; see our guide to idempotency keys. “Exactly-once” processing is possible in specific setups but requires careful design end to end.
In a system design interview
Use a queue to smooth spikes and decouple slow work, such as sending notifications (see designing a notification system) or recording analytics (see designing a URL shortener). Choose a log when multiple consumers need the same events or replay matters.
Frequently asked questions
Is Kafka a queue?
Not exactly. It’s a distributed log that can be used like a queue through consumer groups, while also supporting replay and multiple readers.
Which is easiest to operate?
SQS, because it’s fully managed. Managed offerings also exist for Kafka and RabbitMQ.
Can I use more than one?
Yes. Many systems use Kafka for event streaming and a queue for background jobs.
Sources
Every article is edited by a human and checked against our editorial policy. Spotted a mistake? Tell us.
Keep reading
Rate Limiting Algorithms: Token Bucket, Leaky Bucket and Sliding Windows
Rate limiting algorithms compared: fixed window, sliding log, sliding window counter, token bucket and leaky bucket, with code and distributed design tips.
Idempotency Keys: Designing Safe Retries
How idempotency keys make retries safe: why duplicate requests happen, how to store and check keys, handling concurrent requests, expiry and common mistakes.