Wiki
Core Architecture
The X Recommendation Algorithm, as detailed in the `twitter/the-algorithm` repository, is architected as a comprehensive system of interconnected services and jobs. It is designed to efficiently gener
Updated December 24, 2025v1
Page Viewers
1170
0 users~1170 guests
Guest Views
~1170 views from anonymous visitors
The X Recommendation Algorithm, as detailed in the twitter/the-algorithm repository, is architected as a comprehensive system of interconnected services and jobs. It is designed to efficiently generate and serve diverse content feeds across various X product surfaces, including the "For You Timeline," Search, Explore, and Notifications. This complex system is built upon a foundation of shared data, machine learning models, and robust software frameworks. For a general introduction, refer to the Overview.
Shared Architecture Components
The core of the algorithm relies on a set of shared components that underpin various recommendation surfaces. These components are categorized into Data, Models, and Software Frameworks, enabling consistency and reusability across the platform.
Data
| Type | Component | Description |
|---|---|---|
| Data | tweetypie | Core service that handles the reading and writing of post data. (tweetypie/server/README.md) |
| Data | unified-user-actions | Real-time stream of user actions on X, providing crucial behavioral data. (unified_user_actions/README.md) |
| Data | user-signal-service | A centralized platform designed to retrieve both explicit (e.g., likes, replies) and implicit (e.g., profile visits, tweet clicks) user signals. (user-signal-service/README.md) |
Models
| Type | Component | Description |
|---|---|---|
| Model | SimClusters | Facilitates community detection and generates sparse embeddings for users and content into those communities. (src/scala/com/twitter/simclusters_v2/README.md) |
| Model | TwHIN | Provides dense knowledge graph embeddings for Users and Posts, enhancing relational understanding within the platform. (https://github.com/twitter/the-algorithm-ml/blob/main/projects/twhin/README.md↗) |
| Model | trust-and-safety-models | A suite of models dedicated to detecting NSFW (Not Safe For Work) or abusive content, ensuring platform integrity. (trust_and_safety_models/README.md) |
| Model | real-graph | Predicts the likelihood of one X User interacting with another User, a foundational component for social recommendations. (src/scala/com/twitter/interaction_graph/README.md) |
| Model | tweepcred | Implements a Page-Rank like algorithm to calculate and assess X User reputation. (src/scala/com/twitter/graph/batch/job/tweepcred/README) |
| Model | recos-injector | A streaming event processor that builds input streams for services based on the GraphJet framework (e.g., User-Tweet-Entity Graph). (recos-injector/README.md) |
| Model | graph-feature-service | Serves graph features for a directed pair of users (e.g., how many of User A's following liked posts from User B), enriching model inputs. (graph-feature-service/README.md) |
| Model | topic-social-proof | Identifies topics related to individual posts, supporting content categorization and relevance. (topic-social-proof/README.md) |
| Model | representation-scorer | Computes scores between pairs of entities (Users, Posts, etc.) using embedding similarity, aiding in content matching. (representation-scorer/README.md) |
Software Framework
| Type | Component | Description |
|---|---|---|
| Software framework | navi | A high-performance, machine learning model serving framework written in Rust. (navi/README.md) |
| Software framework | product-mixer | A flexible software framework specifically designed for building feeds of content, used by various product surfaces. (product-mixer/README.md) |
| Software framework | timelines-aggregation-framework | Provides a framework for generating aggregate features, supporting both batch and real-time processing. (timelines/data_processing/ml_util/aggregation_framework/README.md) |
| Software framework | representation-manager | A service responsible for retrieving various embeddings (e.g., SimClusters and TwHIN). (representation-manager/README.md) |
| Software framework | twml | A legacy machine learning framework built on TensorFlow v1, still utilized by some components. (twml/README.md) |
Product Surfaces
Beyond the shared infrastructure, the repository also details the specific implementations for key product surfaces, namely the "For You Timeline" and "Recommended Notifications."
For You Timeline
The "For You Timeline" is a highly personalized feed designed to show users content they are most likely to engage with. It orchestrates a complex flow of candidate generation, ranking, and filtering to achieve this.
The diagram below illustrates how major services and jobs interconnect to construct a For You Timeline:
The core components involved in building the "For You Timeline" are:
Candidate Source
| Type | Component | Description |
|---|---|---|
| Candidate Source | search-index | Responsible for finding and ranking "In-Network" posts, contributing approximately 50% of the posts in the timeline. (src/java/com/twitter/search/README.md) |
| Candidate Source | tweet-mixer | Acts as a coordination layer for fetching "Out-of-Network" tweet candidates from various underlying compute services. (tweet-mixer) |
| Candidate Source | user-tweet-entity-graph (UTEG) | Maintains an in-memory User-to-Post interaction graph and identifies candidates through traversals of this graph. It is built on the GraphJet↗ framework. (src/scala/com/twitter/recos/user_tweet_entity_graph/README.md) |
| Candidate Source | follow-recommendation-service (FRS) | Provides users with recommendations for accounts to follow, including posts from those suggested accounts. (follow-recommendations-service/README.md) |
Ranking
| Type | Component | Description |
|---|---|---|
| Ranking | light-ranker | A ranking model primarily used by the search index (Earlybird) to perform an initial, lighter ranking of posts. (src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird/README.md) |
| Ranking | heavy-ranker | A neural network dedicated to a more sophisticated ranking of candidate posts, serving as one of the main signals for selecting timeline posts after candidate sourcing. (https://github.com/twitter/the-algorithm-ml/blob/main/projects/home/recap/README.md↗) |
Post mixing & filtering
| Type | Component | Description |
|---|---|---|
| Post mixing & filtering | home-mixer | The primary service responsible for constructing and serving the Home Timeline, built upon the product-mixer framework. (home-mixer/README.md) |
| Post mixing & filtering | visibility-filters | Ensures content compliance with legal regulations, enhances product quality, builds user trust, and protects revenue through hard-filtering, visible product treatments, and coarse-grained downranking. (visibilitylib/README.md) |
| Post mixing & filtering | timelineranker | A legacy service that provides relevance-scored posts from the Earlybird Search Index and the UTEG service. (timelineranker/README.md) |
Recommended Notifications
Recommended Notifications focuses on proactively surfacing relevant content to users through push notifications.
The core components for Recommended Notifications are:
| Type | Component | Description |
|---|---|---|
| Service | pushservice | The main recommendation service at X, specifically used to deliver recommendations to users via notifications. (pushservice/README.md) |
| Ranking | pushservice-light-ranker | A light ranker model utilized by pushservice to rank posts. It bridges candidate generation and heavy ranking by pre-selecting highly relevant candidates from a large initial pool. (pushservice/src/main/python/models/light_ranking/README.md) |
| Ranking | pushservice-heavy-ranker | A multi-task learning model designed to predict the probabilities that target users will open and engage with the sent notifications. (pushservice/src/main/python/models/heavy_ranking/README.md) |