Design a News Feed

A news feed shows each user recent posts from the accounts they follow, fast. The central design choice is when to do the work: fan-out on write copies each new post into every follower’s precomputed feed, making reads cheap; fan-out on read assembles the feed when the user opens the app, making writes cheap. Production systems usually combine both, then add ranking, caching and cursor-based pagination on top.

1. Requirements

Functional: publish posts (text, images, video), follow accounts, view a feed of followed accounts’ posts, newest or ranked first, with infinite scrolling.

Non-functional: feed loads in a few hundred milliseconds, publishing feels instant, and the system survives traffic spikes. Slight delays in a post appearing in followers’ feeds are acceptable: this is a classic case where eventual consistency is fine.

2. The core data

• Users and follows (who follows whom): a relational store or graph-friendly table.
• Posts: ID, author, timestamp, content and media references, sharded by post ID or author.
• Feeds: for each user, an ordered list of post IDs, usually held in memory.
• Media: object storage delivered through a content delivery network.

3. Fan-out on write (push)

When someone posts, a background worker looks up their followers and inserts the new post ID at the top of each follower’s feed list.

• Reads are fast: opening the app is one cache read of a precomputed list.
• Writes multiply: a post by an account with 10,000 followers means 10,000 inserts.
• Wasted work: feeds are updated even for users who never log in.

Run the fan-out asynchronously through a queue so publishing returns immediately; our comparison of Kafka, RabbitMQ and SQS covers the options.

4. Fan-out on read (pull)

When a user opens the feed, fetch recent posts from each account they follow and merge them.

• Writes are cheap: a post is stored once.
• Reads are expensive: a user following 500 accounts triggers 500 lookups, merged and sorted, on every refresh.

5. The hybrid, and the celebrity problem

An account with tens of millions of followers makes fan-out on write explode: one post, tens of millions of inserts. The common answer:

• Push posts from ordinary accounts into followers’ feeds.
• Pull posts from very large accounts at read time and merge them in.
• Skip inactive users during fan-out and rebuild their feed if they return.

6. Feed storage and hydration

Store each feed as a capped list of post IDs, for example the latest 500, in an in-memory store. A sorted set keyed by user, with timestamps or scores, makes inserts and range reads cheap. When a feed is requested:

1. Read a page of post IDs from the feed cache.
2. Hydrate them: batch-fetch post content, author details and counts, mostly from caches.
3. Filter out deleted posts, blocked authors and anything the user has hidden.

Keep the post and user caches warm with the patterns in our caching strategies guide; a feed page can touch dozens of objects.

7. Ranking

A purely chronological feed is simple and predictable. A ranked feed scores candidate posts on signals such as recency, the viewer’s relationship with the author, predicted interest and content type, often with a machine-learning model. A typical pipeline gathers a few hundred candidates, scores them, then applies rules for diversity and freshness.

Ranking for engagement has side effects on users as well as metrics. It’s worth understanding why endless feeds keep people scrolling before tuning purely for time spent; features like “you’re all caught up” and a chronological option exist for good reasons.

8. Pagination

Use cursor-based pagination, not offsets. Return a cursor (for example, the score or ID of the last item served) and fetch the next page after it. Offsets break as new posts arrive: items shift and users see duplicates or gaps.

9. Counters and interactions

Likes, comments and shares are write-heavy. Increment counters asynchronously, batch them and accept that displayed counts lag slightly. Showing rounded counts for big numbers (“1.2K likes”) hides small delays, and users rarely need exact values. Notifications for interactions go through a separate pipeline; see our notification system design.

10. Scaling and reliability

• Keep feed workers stateless and scale them with queue depth.
• Replicate caches; if a feed cache is lost, rebuild it from the follow graph and recent posts.
• Rate-limit publishing and fan-out to protect the system during spikes.
• Monitor fan-out lag: the time between a post and its appearance in followers’ feeds.

Frequently asked questions

Is fan-out on write or fan-out on read better?

Neither on its own. Fan-out on write gives fast reads for most users; fan-out on read avoids the cost of accounts with huge followings. Large systems use a hybrid.

Why store post IDs instead of whole posts in feeds?

IDs are small, so millions of feeds fit in memory, and edits or deletions only need changing in one place.

How do you handle deleted posts?

Remove them from the post store and filter them out during hydration. Stale IDs in feed caches then simply disappear from results.