DEV Community

Cover image for The Public Code Behind xAI's "For You" Feed
Mehmet Aras
Mehmet Aras

Posted on

The Public Code Behind xAI's "For You" Feed

#ai #opensource #machinelearning #socialmedia

Turkish version: blog.arasmehmet.com

Disclaimer: This post is based on the public
xai-org/x-algorithm repository. It describes what the
released code shows and where that public release stops.

Open X, scroll "For You," and the simple picture is easy: posts go in, the best-looking ones rise.

Look at the repo, and that picture changes. "For You" does not look like a place that only scores posts and sorts them. The code shows viewer guesses, post judgments, reply scores, and items placed into the list after ranking.

That is the interesting shift: the story is not just ranking logic. It is the information carried around the feed.

Not gender knowledge. A gender guess.

The code does not say "X knows your gender." It shows something narrower: a gender-guess step in the Phoenix feed path. Not certainty. A label and a confidence score.

That matters because the value is not about the post. It travels with the request for the person asking for the feed.

The step is not always active. When it runs, a new account gets a fresh prediction if there is no saved result. For an older account, this file only uses the saved guess. Nearby, there is also a slot for demographics. The repo shows the slot, not the full contents.

A viewer signal this close to the feed does not look like decoration. The repo does not show the final use. But it does show the kind of information the feed path is prepared to carry.

Posts carry judgment fields too

The post side is easier to see.

Grox returns fields like quality score, description, tags, topic categories, whether an image is editable by Grok, slop score, and minor-related score.

The names matter. They do not read like private machine internals. They read like judgments attached to a post.

This is where "content understanding" turns into data on the post. The exact prompt Grox receives is not in the repo. The output fields are visible.

Replies get scored too

Replies have their own Grox scorer.

Score: 0, 1, 2, 3. The scorer looks at the surrounding conversation, not just the reply as a standalone sentence.

That does not tell us where the score appears in the product. But it changes the picture: a reply is not only a comment under a post. It is also content to evaluate.

The feed is not just ranked. It is arranged.

A ranked post list is only part of it. The code shows items from other sources being placed into that list.

The For You code lists ads, Who to Follow suggestions, prompts, and push-to-home posts as separate sources. Prompts and Who to Follow suggestions are inserted into the list. A push-to-home item is placed at the very top when present.

The push-to-home path is the cleanest example. When the request names a tweet for push-to-home, the code builds a special feed item for it. If that tweet starts a conversation, the code attaches up to three top repliers.

Ads and prompts come with context too. Ads receive fields like country, language, IP address, user agent, user roles, device id, mobile device id, and mobile ad id. Prompts receive user and device context plus the prompt types supported by the app.

So the feed is not just a scoreboard. It is scored posts plus inserted modules, special items, and context passed to other systems.

What the public repo leaves out

The public repo is useful because it shows the rough shape of the system. It also shows where it stops.

The clearest gap is home-mixer. The code references a shared helper module 18 times across 16 Rust files, including the part that builds the Phoenix request. But that module is not in the released tree. The simple result: home-mixer does not compile as-is.

This does not look like a tiny missing detail. The feed code leans on a shared helper layer, and that layer is not visible in the repo.

Grox has similar edges. Some files refer to prompt, config, or model folders; those folders are not in the repo. Phoenix shows the same pattern: the docs explain how to run a system, but the code needed to run it is not visible in the repo.

So it is better to think about the repo in two parts: what it shows, and what it leaves out. The visible part gives serious clues about the feed. The missing part stops us from proving exactly how everything connects in the live system.

Closing

The repo's real message is this: "For You" does not just pick posts. It carries information.

That information includes viewer guesses, post judgments, reply scores, inserted feed items, and context for ads and prompts. But the repo does not show exactly how all of those pieces are used in the live system.

That is the difference: "For You" looks less like a single ranking rule and more like a wider system carrying context around the feed.

Get in touch: arasmehmet.com

Where this lives in the code

Release frame

  • release note: README.md:L26-L42
  • Grox release description: README.md:L34
  • Phoenix ranking summary: README.md:L53-L56
  • public-release boundary examples:
    • home-mixer/lib.rs:L1-L13
    • grox/classifiers/content/banger_initial_screen.py:L8-L13
    • phoenix/README.md:L210-L238

Viewer context

  • inferred-gender behavior:
    • home-mixer/query_hydrators/user_inferred_gender_query_hydrator.rs:L30-L47
    • "new user" cutoff: home-mixer/query_hydrators/user_inferred_gender_query_hydrator.rs:L56
  • demographics behavior:
    • home-mixer/query_hydrators/user_demographics_query_hydrator.rs:L14-L24
    • query fields: home-mixer/models/query.rs:L72-L77
  • connected in Phoenix:
    • modules declared: home-mixer/query_hydrators/mod.rs:L17-L18
    • added to Phoenix query steps: home-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs:L228-L231

Post judgments

  • Grox result fields:
    • grox/classifiers/content/banger_initial_screen.py:L30-L38
    • parsed into category results: grox/classifiers/content/banger_initial_screen.py:L102-L155
  • connected in Grox:
    • stream includes this check: grox/generators/stream_generator.py:L71-L78
    • plan for this check: grox/plans/plan_initial_banger.py:L14-L37
    • main plan includes it: grox/plans/plan_master.py:L18-L34
    • engine starts the main plan: grox/engine.py:L51-L55
  • saved output:
    • metrics and fields: grox/tasks/task_pub.py:L166-L219, grox/tasks/task_pub.py:L308-L325
    • write call: grox/tasks/task_pub.py:L339-L340
  • missing prompt body:
    • grox/classifiers/content/banger_initial_screen.py:L13
    • no grox/prompts/ directory in this checkout

Reply scoring

  • reply scorer:
    • grox/classifiers/content/reply_ranking.py:L29-L49
    • thread view and signals: grox/classifiers/content/reply_ranking.py:L53-L72
    • fallback sampling: grox/classifiers/content/reply_ranking.py:L74-L88
    • score cleanup: grox/classifiers/content/reply_ranking.py:L103-L158
  • connected in Grox:
    • grox/tasks/task_rank_replies.py:L11-L31
    • stream eligibility: grox/generators/stream_generator.py:L58-L68
    • recovery stream: grox/generators/stream_generator.py:L173-L178
    • plan: grox/plans/plan_reply_ranking.py:L10-L27
    • plan master: grox/plans/plan_master.py:L18-L34

Feed assembly

  • feed sources:
    • home-mixer/candidate_pipeline/for_you_candidate_pipeline.rs:L164-L177
    • selector constructed: home-mixer/candidate_pipeline/for_you_candidate_pipeline.rs:L179-L201
  • mixing and insertion:
    • home-mixer/selectors/blender_selector.rs:L30-L51
    • prompts: home-mixer/selectors/blender_selector.rs:L77-L85
    • Who to Follow: home-mixer/selectors/blender_selector.rs:L89-L100
    • push-to-home pinned at top: home-mixer/selectors/blender_selector.rs:L103-L113
  • push-to-home source:
    • home-mixer/sources/push_to_home_source.rs:L19-L67
    • top repliers: home-mixer/sources/push_to_home_source.rs:L70-L95
  • ad and prompt context:
    • home-mixer/sources/ads_source.rs:L41-L57
    • home-mixer/sources/prompts_source.rs:L18-L46
    • home-mixer/sources/prompts_source.rs:L49-L98

Public boundaries

  • home-mixer missing shared module:
    • module list: home-mixer/lib.rs:L1-L13
    • rg -n "crate::util" home-mixer --glob "*.rs" returns 18 hits across 16 Rust files
    • Phoenix request builder import: home-mixer/scorers/phoenix_scorer.rs:L8
    • Phoenix request builder call: home-mixer/scorers/phoenix_scorer.rs:L84
    • Phoenix request cache helpers: home-mixer/side_effects/phoenix_request_cache_side_effect.rs:L4-L6, home-mixer/side_effects/phoenix_request_cache_side_effect.rs:L77-L85
    • other examples pointing to missing crate::util: home-mixer/server.rs:L3-L9, home-mixer/scorers/ranking_scorer.rs:L3-L4, home-mixer/scorers/vm_ranker.rs:L1-L5, home-mixer/for_you_server.rs:L3-L5
  • Grox missing packages:
    • grox/classifiers/content/banger_initial_screen.py:L8-L13
    • grox/classifiers/content/reply_ranking.py:L9-L13
    • grox/classifiers/content/post_safety_screen_deluxe.py:L9-L14
    • no grox/prompts/, grox/config/, or grox/lm/ directory in this checkout
  • Phoenix README boundary:
    • phoenix/README.md:L210-L238
    • find phoenix -maxdepth 3 -type f returns only phoenix/README.md in this checkout

Top comments (0)