A Reinforcement Learning Approach to Job Search

Pramod Krishnachari — Sat, 18 Jul 2026 03:49:00 +0000

A Field Manual ·

Subject: Job search as MDP
Audience: The reflective applicant
Horizon: Built to last

A durable framework for applications, outreach, and networking treated as a partially-observable decision process you improve, week over week, instead of an act of faith you repeat until something works. Expanded with the funnel mathematics, the interview subproblem, the craft of outreach, and the machinery of long searches.

Part I: Foundations - the problem, formalized

§ 01 - The Frame

Reinforcement learning is the right vocabulary for job search because everything that makes job search painful is structurally a reinforcement learning problem. You take an action; submit, message, post, prepare and a long, ambiguous delay later you observe a faint, noisy signal that may or may not have anything to do with what you actually did. The reward is sparse. The state is partially observable. The credit assignment problem is brutal: was it the resume edit, the warm intro, the timing, the market, the recruiter's mood? You will never know for any single instance. But across many instances, with disciplined logging and the willingness to update, patterns emerge that a non-learner cannot see.

The candidate who fails is not lazy. They are usually working very hard. They fail because they treat each application as an independent attempt rather than a sample drawn from a distribution they could be learning. They keep pulling the same lever and waiting. They confuse activity with adaptation. The framework here is the opposite: fewer actions, taken more deliberately, logged carefully, reviewed weekly, with the policy updated explicitly.

Against the supervised mindset

Most of us were trained by school into a supervised learning model of effort: submit the assignment, receive the grade, the grade reflects the work. The job market does not grade homework. It samples policies. A single application is not an exam you passed or failed it is one draw from a noisy process whose parameters you can only estimate across many draws. Internalizing this changes everything downstream. A rejection stops being a verdict on your worth and becomes what it actually is: one low information sample. A silence stops being an insult and becomes a data point with a known base rate. The emotional relief this reframe provides is not a side benefit; it is load-bearing, because the agent's morale is part of the state, and the state determines what actions are even possible next week.

You cannot solve the job market. You can become a better than baseline learner inside it. That is enough.

Three searches run in parallel

It clarifies the whole enterprise to notice that you are not running one search but three, on different time horizons, and they share a budget:

The transactional search — get an offer for the current opening. Horizon: weeks to months. This is what everyone means by "job search," and it is where the funnel math of §03 lives.
The capability search — become a stronger candidate: skills, artifacts, interview craft, sharper positioning. Horizon: months. Every hour here changes the state so that every future transactional action has a higher expected return.
The network search — build and warm the graph of people who will power this search and every search after it. Horizon: months to decades. This is the high-discount-factor game of §11.

Candidates in a panic collapse all three into the first and starve the other two which is precisely backwards, because the second and third searches are what make the first one tractable. The weekly allocation in §06 deliberately funds all three. Hold that structure even when the transactional search screams for every available hour. Especially then.

One honest caveat

RL in the literature usually assumes a stationary or slowly-changing environment. The job market is non-stationary listings appear and vanish, your skills compound, your story improves with each interview, the macro shifts under your feet. The right mental model is therefore not "find the optimal policy" but "maintain a policy that improves faster than the environment drifts." That is achievable. Most candidates' policies don't improve at all, which is why a disciplined learner holds an edge that persists even in bad markets. Section §14 treats the drift directly.

Section notes

Note: Terminology, briefly

State: what you can observe about your situation.
Action: what you do this week.
Reward: measurable signal of progress.
Policy: your rule for choosing actions given the state.
Update: the change you make to the policy after observing reward.
Full glossary in Appendix A.

Note: A note on identity

Framing yourself as an agent inside a learning loop is psychologically protective. Rejection becomes signal, not verdict. Silence becomes data, not judgment. You are the scientist, not the specimen.

Note: Why three searches

The decomposition is also a diagnostic. A search that feels stuck is almost always over invested in one of the three. All transaction, no capability: you plateau. All capability, no transaction: you hide in preparation. All network, no transaction: pleasant conversations, no offers.

§ 02 The MDP: A Full Specification

Formalize the problem before you try to optimize inside it. The point is not mathematical rigor it is to make the moving parts visible. If you cannot name your state, action, and reward, you cannot improve them.

State `s`

What you observe at the start of each decision period call it Monday morning. The state is necessarily partial; you cannot see inside any company's hiring intent. But you can see your own posture clearly enough, if you look at it on purpose.

Observable state variables

Variable	Meaning	Source
Pipeline shape	Count of leads at each stage: cold, applied, replied, screening, technical, onsite, offer	Replay buffer
Pipeline EV	Expected offers implied by the pipeline (§03) the single most decision-relevant number	Computed
Reply rate (30d)	Substantive responses ÷ outbound actions in the trailing window	Replay buffer
Channel returns	Per-channel reply rates and stage progressions	Replay buffer
Network warmth	People whose warmth score (§11) is high enough that they'd reply within 48h	Network sheet
Capacity	Hours available this week for search work, after life and other obligations	Calendar
Morale	Self-reported 1–5; tracks burnout before burnout tracks you	Daily check-in
Runway	Days until a hard constraint binds — visa clock, savings, lease, deadline	Known

The state audit — a twenty-minute protocol

Once, at the start of the search, and again monthly, answer four questions in writing. This is the honest initialization without which every downstream estimate is fantasy:

Positioning, one sentence. "I am a ____ who is unusually good at _, looking for _ roles at ____ kinds of companies." If the sentence takes more than one attempt to say aloud, the positioning is not yet real — and every message you send will leak that uncertainty.
Proof, three artifacts. The three concrete things a stranger could inspect in five minutes that make the positioning credible: a project, a write-up, a deployed thing, a talk. If you cannot name three, the capability search (§01) needs hours this week.
Warm network, honest count. Not connections people who would reply within 48 hours if you wrote today. Most people's honest number is smaller than they think, and knowing it changes the allocation.
The binding constraint. Which single variable most limits you right now: reply rate (top of funnel), conversion (middle), warm network (channel access), or time (runway)? The week's plan should visibly attack the binding constraint, not the comfortable one.

Action `a`

The set of things you can choose to spend your next hour on. The actions differ in cost, expected reward, variance, and time-to-signal. Most candidates use only two or three of these and wonder why their policy plateaus.

A₁ · Cold application
Submit to a public posting with a generic resume. Cheap, low-signal, occasionally necessary for funnel volume. Option-buying, not strategy.

A₂ · Tailored application
Customize resume bullets and write a real cover note for one specific role. 4–10× the reply rate of A₁ in most reported data, at roughly 3× the cost.

A₃ · Direct outreach
Email or LinkedIn message to a hiring manager, recruiter, or team member. Cold but personalized. The highest-variance action; returns are bimodal. Craft in §10.

A₄ · Warm intro request
Ask someone in your network to forward you. Highest expected value per attempt by a wide margin; rate-limited by network warmth, which is why §11 exists.

A₅ · Alumni / community outreach
Reach out for a conversation, not a job. Long horizon. Builds the network that powers A₄ later.

A₆ · Content / visibility
A post, a write-up, a public artifact. Pull rather than push. Pays slowly, compounds, and is fully off-policy it reaches people you would never have written to.

A₇ · Skill / portfolio work
Build something. Strictly, an intrinsic-reward action — it changes the state by making you a better agent, raising the return of every future action.

A₈ · Interview prep
Mocks, drills, story refinement. Only converts once you have screens to convert; underdone before pipeline exists, overdone after. Treated fully in §12.

A₉ · Follow-up
Re-touch stale threads. Nearly free, chronically neglected, and among the highest hourly-return actions in the entire space. The ladder is specified in §10.

Composite actions: the campaign

In RL these are called options: temporally extended macro-actions composed of primitives. In practice, your highest-value targets deserve campaigns, not single touches. A campaign against one target company bundles: 30–45 minutes of research (§09) → a tailored application → direct outreach to the hiring manager referencing something specific → a warm-intro attempt if any path exists → the follow-up ladder. Log it under one campaign ID and evaluate it as a unit — because the touches interact, and crediting them separately misreads what happened. Reserve campaigns for your top ten targets; everything else gets primitive actions.

Reward `r` and Transition `P(s′ | s, a)`

The true terminal reward is +OFFER, so rare and delayed that learning purely from it is hopeless; the shaped substitute is built in §05. The transition function is mostly unknown and partially stochastic — you do not need to model it globally. You need to do two things instead: estimate its stage-to-stage structure from population data and then your own (that is §03), and sample from it carefully while remembering everything (that is §08).

Section notes

Note: Partial observability

You can never see a company's true hiring state — frozen reqs, internal candidates, a reorg mid-process. Treat all of it as a latent variable. Outreach often reveals state more than it advances it; that information is itself part of the return.

Note: A common error

Conflating action cost with action value. A cold application is cheap, but its expected reward is so low that ten of them rarely beat one warm intro. §03 makes this quantitative.

Note: On morale as state

Morale is not a soft variable; it is the variable that determines whether tomorrow's actions happen at all. A burned-out agent has a degenerate action space. Log it daily in ten seconds: one number, 1–5.

Note: Options, formally

Sutton, Precup & Singh (1999) introduced options as temporally extended actions with their own initiation sets and termination conditions. The campaign is exactly this: initiated on high fit targets, terminated on reply, rejection, or the D30 close out.

§ 03 Funnel Mathematics

This is the section that turns the framework from philosophy into arithmetic. The job funnel is a Markov chain: leads move through stages with estimable transition probabilities, and the value of a lead at any stage can be computed by backward induction from the terminal state. Once you can value stages, three previously vague questions become computable: how much is this action worth, how healthy is my pipeline, and what is the binding constraint this week?

Stage conversion bands

Calibration anchors first blended figures for technical roles at mid-market companies, drawn from widely reported community and recruiting data. They are starting priors, not truths; your buffer replaces them within six to eight weeks.

Stage transitions: prior bands (replace with observed)

Transition	Band	Midpoint
Tailored application → substantive reply	8-15%	`0.115`
Cold application → substantive reply	1-3%	`0.02`
Warm-intro'd candidacy → reply	40-70%	`0.55`
Reply → recruiter screen	50-70%	`0.60`
Screen → technical round	40-60%	`0.50`
Technical → onsite / final loop	35-55%	`0.45`
Onsite → offer	25-40%	`0.33`

Backward induction: what a stage is worth

Set the value of an offer to 1 and walk backwards, multiplying by each transition probability. Using the midpoints:

Formula: V_onsite = 0.33; V_technical = 0.15; V_screen = 0.074; V_reply = 0.045

And one more step back to the action itself:

Formula: V_tailored app ≈ 0.115 × 0.045 ≈ 0.005; V_warm intro ≈ 0.55 × 0.70 × 0.074 ≈ 0.028

Read those numbers slowly, because they carry three of the most important lessons in this manual.

One tailored application ≈ 1/200th of an offer. Nobody's single application "should have worked." The despair that follows any individual silence is a category error — the unit was never big enough to carry that much meaning. Volume discipline is not pessimism; it is literacy.
One warm-intro'd candidacy ≈ 1/35th of an offer roughly 5–6× the tailored application, before accounting for the better downstream conversion warm candidates typically enjoy. This is the entire quantitative case for spending real hours on §11 even when it produces nothing this week.
A cold application ≈ 1/1000th of an offer. Not zero — options are worth holding — but a search built primarily on this channel is a plan to send a thousand of them. Some people do. There are better plans.

Pipeline expected value

Value every live lead at its current stage and sum. This single number pipeline EV, the expected offers implied by your current pipeline is the most decision relevant statistic in the whole system.

Algorithm 2: Pipeline valuation & bottleneck diagnosis

V = {applied: 0.005, replied: 0.045, screen: 0.074,
     technical: 0.15, onsite: 0.33}   # replace with observed

EV = sum(V[lead.stage] for lead in pipeline if lead.alive)

# -- diagnosis --
if EV < 1.0:
    bottleneck = "top of funnel"     # go wide this week
elif share_of_EV(stages=[applied, replied]) > 0.6:
    bottleneck = "conversion"        # shift hours to prep & craft
elif share_of_EV(top_2_leads) > 0.6:
    bottleneck = "concentration"     # keep feeding the top even now
else:
    bottleneck = "none"               # execute; don't fidget

A worked example. Thirty applications awaiting reply, four live replies, three screens scheduled, one technical, one onsite: 30×.005 + 4×.045 + 3×.074 + 1×.15 + 1×.33 ≈ 1.03 expected offers. That pipeline is alive but thin — one bad onsite and it drops below 0.7. The operational rule that follows:

Key rule — The EV floor

If you want an offer this quarter with any margin for variance, keep pipeline EV at or above 1.5 at all times. Below the floor, the week's plan is top of funnel regardless of what else is happening. And because leads decay, EV decays — a pipeline you stop feeding loses roughly a third of its value every two to three weeks as threads go stale.

The refill lag: the classic mid-search catastrophe

The single most common structural error of an otherwise good search: interviews arrive, all hours flow to interview prep, top of funnel stops. Then the interviews fail as, at 33% per onsite, they usually individually do and the pipeline is empty. Refilling from cold takes four to six weeks. The candidate experiences this as a devastating regression; structurally it was a predictable consequence of a policy decision made a month earlier. The rule: weekly outbound never drops below 40% of normal volume, no matter the interview load. It feels wasteful during a hot streak. It is the cheapest insurance you will ever buy.

Capacity planning: working the math forward

The same arithmetic runs forward as a planning tool. Suppose the goal is two offers in twelve weeks. A weekly mix of five tailored applications, four warm intro attempts, and three direct outreaches is worth roughly 5×.005 + 4×.028 + 3×.008 ≈ 0.16 expected offers per week, about 1.9 over the quarter. In other words, the standard allocation of §06 is not arbitrary; it is approximately calibrated to a two-offer quarter at prior conversion rates. If your observed rates come in lower, the same equation tells you exactly how much wider or warmer the funnel must run. No more guessing whether you are "doing enough" the question has an answer.

Section notes

Note: Why these numbers move

Every band shifts with seniority, market heat, and fit quality. Early career and career switching candidates should expect the lower edges; candidates with rare, in demand specialties see the upper edges. The method is the point; the numbers are furniture.

Note: Value iteration, formally

This is one sweep of value iteration on a chain MDP with an absorbing terminal state. The full algorithm iterates until convergence; a funnel this shallow converges in one pass, which is why a spreadsheet suffices.

Note: Leads decay

A "screen scheduled" from five weeks ago that never happened is not worth Vscreen. Apply a staleness haircut: multiply a lead's value by 0.5 after 21 days without forward motion, and mark it dead at 45 unless explicitly revived.

Note — The comfort of base rates

At midpoint rates, a good onsite fails two times in three. Knowing this before it happens is the difference between "the system is working, variance happened" and a morale collapse that stalls the search for a month.

Part II: The engine: loops, rewards, and allocation

§ 04 The Algorithm: Daily & Weekly Loops

All the theory resolves to nested loops. The week is the unit of learning — policy updates happen there. The day is the unit of execution — and days need their own structure, because willpower is a morning resource and outreach quality decays with fatigue. First the learning loop, then the execution loop it contains.

Algorithm 1: Weekly policy iteration

# --- Initialization (one-time, ~2 hours) ---
channels = [cold_app, tailored_app, direct_outreach, warm_intro,
            alumni, content, skill_work, interview_prep, follow_up]
Q[c]            = 0.5   # optimistic init — forces fair sampling
N[c]            = 0
replay_buffer   = empty_log()
capacity_week   = 15     # honest hours, after life
explore_budget  = anneal(runway)   # 25% → 10% as deadline nears, §07

# --- The loop ---
while not terminal(state):

   # Monday: plan (30 min)
   state   = observe(pipeline_EV, reply_rate_30d, warmth, morale, runway)
   focus   = binding_constraint(state)          # §02 audit + §03 diagnosis
   targets = choose_channels(Q, N, capacity_week, explore_budget)

   # Tue–Fri: act (Algorithm 1b, daily)
   for action in targets:
      hypothesis = write_down_what_you_expect(action)
      execute(action)
      log(replay_buffer, action, hypothesis, timestamp=now)

   # Friday afternoon: observe & update (40 min)
   for entry in replay_buffer.recent(days=14):
      r = shaped_reward(entry)                   # §05
      N[entry.channel] += 1
      Q[entry.channel] += (r - Q[entry.channel]) / N[entry.channel]

   retire(channels, threshold=0.02, min_samples=20)
   introduce_variant(top_channel)                 # one per week, §07
   write_review(five_questions)                   # Appendix B

The daily loop: energy aware scheduling

Within the week, sequence work by cognitive cost, not by anxiety. Anxiety wants you to refresh the inbox and re read old threads; the policy wants your best hours on creation.

Algorithm 1b: The day

# Morning block (90–120 min, best energy) — CREATE
do(highest_cognitive_load)    # tailoring, writing, portfolio, prep

# Midday block (45–60 min) — TRANSMIT
send(outreach, applications)  # Tue–Thu, recipient's mid-morning, §10
send(follow_ups_due_today)    # the tickler column drives this

# Afternoon block (30–45 min) — ABSORB
research(next_targets)        # model-building, §09
log(everything);  morale_checkin(1..5)

# The Minimum Viable Day (bad days only)
if morale ≤ 2 or capacity_today < 30min:
    send(one_follow_up); log(one_row); stop_without_guilt()

The Minimum Viable Day deserves emphasis. Every long search contains bad days; the question is whether a bad day breaks the streak or bends it. One follow-up plus one log row keeps the system alive at a cost of ten minutes, protects the habit, and because follow ups convert at 15-30% occasionally turns the worst day of the month into the one that lands the interview. Design for your bad days and the good days take care of themselves.

The lead priority queue

Not all leads deserve equal attention, and attention is the scarcest resource in the system. Score every live lead weekly:

Formula: priority = fit_1–5 × V_stage × warmth × freshness

where freshness halves after 21 days without motion (§03). The top ten leads get active management — campaigns, research, prompt follow ups, real thought. Everything below the line gets batch treatment: templated follow ups on schedule, no bespoke effort. Review queue membership every Friday; leads earn their way up by responding and drift down by going quiet. This is triage, and triage is kindness to yourself it converts a shapeless mass of "I should really get back to all of these" into a ranked list with a bright line.

Key rule: The bare minimum, if you remember nothing else

Every outbound action gets logged before you do it, with a one-sentence hypothesis about why you expect it to work. Every Friday, you spend forty minutes reading the last two weeks of logs and asking one question: what would I do differently if I started over today? That is the entire algorithm. Everything else is refinement.

Section notes

Note: Why the hypothesis matters

Forcing a written prediction before the action is the single highest ROI habit in the system. Without it, retrospection collapses into hindsight bias. With it, you discover which of your beliefs were wrong the only thing you can actually update on.

Note: Sample efficient learning

Real RL agents need millions of episodes. You will have perhaps a hundred outbound actions a month. Every one must teach something or it is wasted. This is why the hypothesis log is non negotiable, and why §09's cheap simulation matters.

Note: Inbox discipline

Checking for replies is a variable ratio reward schedule the same mechanism as a slot machine. Uncontrolled, it will eat your morning block. Check twice daily at fixed times. The replies arrive at the same speed either way.

Note: Patience clause

Do not change the algorithm because of one bad week. Update the policy, yes the meta-procedure stays. Most candidates who quit the loop quit during week three, exactly when first signal arrives.

§ 05 Reward Shaping: Surviving Sparsity

The reward signal in job search is so sparse that learning from it directly is statistically hopeless. You may send two hundred applications and receive three offers across an entire search; credit assignment from offer back to action is impossible at that resolution. Shaping the reward defining intermediate signals that correlate with the terminal outcome is the central practical problem, and the central danger, because shaped rewards distort policy. Optimize "applications per week" and you will send junk. Optimize "connections made" and you will collect strangers. The art is choosing proxies honestly correlated with what you want.

Formula: r = w₁ · r_terminal + w₂ · r_stage + w₃ · r_engage + w₄ · r_process − w₅ · r_cost

Reward components — reference weights

Signal	What counts	Weight
Terminal	Written offer with terms	`+1000`
Final-round	Onsite / team loop	`+30`
Technical screen	Coding, case, system design	`+10`
Recruiter call scheduled	Calendar held by the company	`+5`
Substantive reply	A real human writing back, even to decline	`+1`
Profile signal	Target-company recruiter views your profile	`+0.3`
Process	Action executed as planned	`+0.05`
Reward-hack guard	Felt like going through the motions	`−0.5`
Burnout penalty	Day lost to search-related exhaustion	`−2`

The relative ordering matters more than the absolute numbers, and two design choices deserve comment. First, process reward is small but non zero: you need some credit for executing as planned even when nothing comes back, or you will quit during a six week silent stretch that base rates say to expect. But make it large and you will manufacture volume for its own sake. Second, the negative reward for going through the motions is the only mechanism preventing the system from optimizing itself into spam. Apply it ruthlessly on Friday: if a batch of actions felt mechanical, mark them down even if they hit the volume target.

The reward is a ledger, not a feeling

A week can feel terrible two rejections, five silences and score well: eleven quality actions, one screen scheduled, one warm intro secured, pipeline EV up 0.3. The inverse is also common: a week that felt busy and productive scores near zero because the activity was low-value churn. This divergence between felt sense and ledger is not a bug; it is the entire reason the ledger exists. Feelings integrate rejection with a weight of about ten and progress with a weight of about one. The ledger weighs them correctly. When they disagree, trust the ledger and note the disagreement in the Friday review, because a persistent gap in either direction is itself diagnostic.

Multi objective reward: the offer is a vector

The terminal reward is not actually scalar. An offer carries compensation, learning slope, visa and sponsorship security, manager quality, mission fit, brand equity, optionality. Two disciplines follow:

Write your scalarization weights before any offer exists day two of the search, per §16. Rank the components; assign rough percentages. The reason to do it early is that motivated reasoning is undefeated: the moment a shiny offer lands, an unwritten value function quietly rewrites itself to justify the exciting thing. A written one can be argued with, but not gaslit.
Let the vector shape targeting, not just deciding. If sponsorship security carries heavy weight, that filter belongs at the top of the funnel verified before hours are invested, not discovered at the offer stage. Every hour spent on a target that fails a hard constraint is reward leakage.

Warning: Reward hacking — the failure mode

The classic RL pathology: the agent maximizes the proxy rather than the objective. In job search: fifty generic messages to "build network," applications to roles you'd never accept to "keep volume up," cover letters with only the company name swapped. The dashboard looks healthy. The terminal reward never arrives. The test: if you cannot say in one sentence why this specific company, the action doesn't count toward the week — whatever the tracker says.

Section notes

Note: Why discount terminal reward at all

Undiscounted, the terminal reward provides nearly zero gradient it is too distant and too noisy. Intermediate signals are the compass. The trick is never mistaking the compass for the destination.

Note: Calibrating to your search

Rare events get heavy weights; frequent events light ones. If onsites become weekly, their weight should fall. The reward function is itself a thing you tune as the binding constraint moves.

Note: Potential-based shaping

Ng, Harada & Russell (1999): shaping rewards defined as differences of a potential function preserve the optimal policy. The stage rewards above approximate this they pay for progress between stages, which is why they distort less than paying for raw activity.

Note: Sponsorship as hard constraint

For candidates with visa clocks: sponsorship is not a preference with a weight it is a feasibility constraint. Model it as a filter, not a reward term. Constraints prune the action space; rewards rank what survives.

§ 06 Channels as Bandit Arms

Weekly channel allocation is a multi-armed bandit: each channel an arm with unknown expected reward, a fixed budget of pulls, and the eternal tension between exploiting the best known arm and learning whether another is better. The literature has real answers, and at this scale they fit in a spreadsheet.

Channels — cost, baseline rate, leverage profile

Channel	Time / action	Typical reply	Leverage	Time-to-signal
Cold application	10–15 min	1–3%	Volume	1–6 wk
Tailored application	45–75 min	8–15%	Quality	1–4 wk
Direct outreach	20–40 min	5–20%	High variance	1–14 d
Warm intro	30 min + intro lag	40–70%	Compounding	3–14 d
Alumni outreach	15–25 min	20–40%	Network build	1–3 wk
Content / visibility	2–5 hr / piece	n/a (pull)	Long-tail	wk–mo
Portfolio work	5–20 hr	indirect	State change	cumulative
Interview prep	1–3 hr	conversion	Conditional	with pipeline
Follow-up	5 min	15–30%	Free money	days

Three patterns leap out of the table once it is actually looked at. Follow-up is the highest hourly return action in the space five minutes, 15–30% conversion and most people skip it out of awkwardness, which is structurally irrational; the ladder in §10 removes the awkwardness by removing the improvisation. Warm intros dominate on expected value and are rate limited only by network warmth, which is a stock you can build (§11). Cold applications are option-buying: cheap lottery tickets that occasionally pay when your resume lands at the moment of need. Hold some options. Do not call it a strategy.

Allocation, formally: UCB and Thompson in a spreadsheet

Two classical algorithms, both one spreadsheet column, both better than intuition at this problem. Use UCB if you want determinism, Thompson if you want elegance under drift; either beats "whatever felt right."

Algorithm 3: Channel selection, two ways

# --- UCB1: pull the arm with the highest upper confidence bound ---
# spreadsheet: =Q + SQRT(2*LN(N_total)/N_c)
for slot in weekly_slots:
    score[c] = Q[c] + sqrt(2 * ln(N_total) / N[c])
    allocate(slot, argmax(score))       # uncertainty is a bonus

# --- Thompson: sample beliefs, allocate to the best sample ---
# per channel keep (successes s, failures f) in a sliding 8-wk window
for slot in weekly_slots:
    draw[c] = Beta_sample(s[c] + 1, f[c] + 1)
    allocate(slot, argmax(draw))        # explores in proportion to doubt

The intuition, in one sentence each. UCB: prefer the arm whose best plausible value is highest — which automatically favors under sampled channels until they've had a fair audition. Thompson: maintain a belief distribution per channel, roll the dice against your beliefs, and let doubt itself drive exploration. The sliding window (trailing eight weeks, §14) keeps both honest as the market and your own skill drift.

The reference allocation: a fifteen hour week

Before your own data exists, a defensible split that funds all three searches of §01:

5h — tailored applications to 4–6 high-fit roles, top targets run as campaigns (§02)
3h — warm-intro requests and alumni outreach, 8–12 messages
2h — direct outreach to hiring managers at 4–6 of those targets
2h — portfolio or skill work on the single artifact nearest to done
1h — follow-ups on everything the tickler says is due
1h — one piece of visible work: a post, a public commit, a write-up
1h — review, retrospection, next-week plan

Roughly 80% exploitation, 20% exploration. By month two the Q-values should be steering the split more than these defaults; by month three the defaults should be unrecognizable, because they will have become yours.

Section notes

Note: The follow-up arm, specifically

A templated, friendly nudge on every outbound thread per the §10 ladder. Two sentences, plus a value-add when you have one. Reply rates on the D7 bump frequently exceed the original send. The tickler column makes it automatic.

Note: Why not always exploit?

Because Q estimates are wrong in proportion to how little you've sampled, and the market drifts under you. An allocation that never explores converges confidently on a local optimum and then decays with the regime. §07 sets the budget.

Note: Where the table breaks

These are aggregates. AI native startups behave differently from incumbents; referral heavy cultures differ from portal driven ones. Trust your own buffer over this table once N > 15 per channel.

Note: Interview prep's odd shape

Prep has near zero value with an empty pipeline and enormous value with a full one. It is the only channel whose Q should be scheduled rather than learned its value is conditional, not stochastic. §12 sets the budgets.

§ 07 Exploration: Discipline, Decay, Curriculum

The exploration exploitation trade off is the central tension of RL and of any extended search. Pull only the arms you know and your ceiling is whatever those arms deliver. Explore relentlessly and no channel ever accumulates the reps to mature. Four rules of thumb that have aged well:

A fixed exploration budget, spent on purpose. A slice of weekly capacity reserved for the non-greedy: an untried channel, a different cover note structure, a company tier you assumed out of reach, a medium you don't use. Fixed in advance you do not get to skip exploration "because things are busy." Busyness is precisely when policies ossify.
One variant per week, top channel only. Take your highest-Q channel and introduce exactly one variation a different opener, a portfolio link in the signature, a tighter structure. One at a time, or you cannot attribute the result. Run it 2-3 weeks against the incumbent before declaring a winner.
The kill threshold. Once a channel or template has N ≥ 20 attempts and a reply rate under 2%, retire it. Do not "just try a few more." Twenty samples at zero is a statistically meaningful negative; honor it. (Retired is not deleted dead channels can be revived when the regime changes, §14.)
Anneal with the deadline. Exploration is an investment whose payoff needs remaining runway to be collected. With a hard clock a visa window, a savings floor, a lease the budget should decay: roughly 25–30% of capacity in the first third of the runway, 20% in the middle, 10% in the final third, where you ruthlessly exploit what the buffer has already proven. Exploring in week one of twelve is wisdom; discovering a fascinating new channel in week eleven is a hobby.

Note: Optimistic initialization - a small trick that matters

Start each new channel with an artificially high prior Q (0.5 against a realistic floor near 0.05). This forces the algorithm to give every arm a fair audition before concluding it doesn't work. Without it, two unlucky first outcomes on a new channel cause permanent, invisible under exploration the channel just quietly never gets pulled again.

Directed exploration: every experiment carries a question

Random exploration is wasteful at sample sizes this small. Better: name what you don't know, then design the week's exploration to find out.

You suspect early stage companies respond faster spend a week applying only to sub 25 person companies and compare reply lag against your baseline.
You suspect the resume, not the outreach, is the bottleneck send structurally identical outreach to two cohorts, one with the resume attached, one linking only a portfolio. Compare.
You've never tried visible work publish one short artifact and track inbound profile views, DMs, recruiter touches for two weeks.

The reward of an exploration is not whether it produced an interview. It is whether you can now answer a question you couldn't before. Log the question with the hypothesis; grade the answer on Friday.

Curriculum learning: sequence the difficulty

RL agents train on easy tasks before hard ones; so should you. Two curricula matter here. The interview curriculum: your first live interviews after a layoff or a degree are rusty regardless of preparation do not spend a dream company on interview number one. Sequence real processes at genuinely acceptable but lower stakes targets first, so the reps accumulate where the variance is affordable; §12 details it. The outreach curriculum: your first twenty cold messages will be your worst twenty. Send them to targets where a mediocre message costs little, and graduate the refined version to the top-ten list. One boundary: pursue only roles you would plausibly accept. There are enough genuinely acceptable lower stakes targets that manufacturing bad-faith interviews is both unnecessary and corrosive — to the market's trust and to your own story bank, which only compounds when the reps are real.

Section notes

Note: Upper confidence, informally

When in doubt between two channels, prefer the one you know less about. Uncertainty is not a cost at allocation time; it is a bonus, because resolving it has value beyond this week's pull.

Note: The dangerous middle

Worst of all worlds: continuing to use a channel you suspect is failing while never sampling it enough to retire it. Pick a side commit to twenty reps or kill it today.

Note: Annealing, formally

Epsilon greedy with a decaying epsilon; simulated annealing's temperature schedule. The shared idea: early randomness buys information while there is time to use it; late randomness is just noise added to a decision that needed conviction.

Note: Exploration ≠ novelty

The point is information, not stimulation. If a new tactic teaches nothing measurable, it wasn't exploration it was entertainment wearing exploration's clothes. The question-attached-to-every-experiment rule is the tell.

Part III: Memory & models - the buffer, the research, the craft

§ 08 The Replay Buffer

The replay buffer is where the system lives or dies. Without it, every week starts over. With it, every week stands on the accumulated evidence of all previous weeks — the same idea as experience replay in deep RL: store every transition; revisit later to extract more learning than the moment of action could give. A spreadsheet or a Notion database; tooling is irrelevant. What matters is that the schema is fixed and the hypothesis column is filled in before the action, not after.

Replay-buffer schema, v2

Field	Why it matters
Date (action)	Anchors time-to-signal calculations
Target (company · role)	Disambiguates leads across channels
Campaign ID	Groups the touches of a §02 campaign so they're evaluated as one option
Channel	The arm pulled; powers Q updates
Variant	Which resume / template / opener; powers the weekly A/B
Hypothesis	One sentence, written before sending: why this should work
Cost (minutes)	So reward-per-hour is computable, not just reward
Outcome	Categorical: no-reply · decline · reply · screen · technical · onsite · offer
Days to signal	Lag from send to first response of any kind
Stage log	Appended as the lead moves; a lead is a trajectory, not an outcome
Next action · date	The tickler — drives the daily follow-up queue automatically
Warmth	For network-mediated actions: the §11 score at time of ask
Lesson	Filled in at Friday review. One sentence. What did this teach.

The dashboard: six numbers, computed weekly

Outbound count — total quality actions sent (the volume floor of §03 checks against this).
Blended reply rate, trailing 30 days — the funnel's top-line health.
Per-channel reply rate — feeds the Q-values and the Algorithm 3 column.
Stage conversions, trailing 8 weeks — your personal replacements for the §03 prior bands.
Pipeline EV — the §03 sum; the number the Monday plan reads first.
Morale trend — seven-day average of the daily 1–5; two consecutive weeks under 2.5 triggers the §15 burnout protocol regardless of what else is going well.

A worked trace: four weeks, compressed

To make the machinery concrete, here is a fictional but representative slice of a buffer one month in ten rows of the sixty odd a real month produces:

Illustrative buffer excerpt

Wk	Target	Ch	Var	Hypothesis (compressed)	Outcome
1	Co. A · MLE	tailored	A	Pipeline work matches their stack post	no-reply
1	Co. B · DS	cold	—	Volume option; posted 2d ago	no-reply
1	Alum R.	alumni	—	Same program, adjacent role; easy yes	reply → call
2	Co. C · MLE	direct	A	EM posted about eval tooling; I built one	no-reply
2	Co. C · MLE	follow-up	D7	Bump; add repo link	reply → screen
2	Co. D · AI Eng	warm	—	R. offered forwardable intro	reply → screen
3	Co. E · MLE	direct	B	New opener: lead with artifact, not bio	reply
3	Co. F · DS	cold	—	Volume option	no-reply
4	Co. C · MLE	—	—	(stage log: screen → technical)	technical
4	Co. G · MLE	direct	B	Artifact-first opener, round two	reply

What the Friday reviews did with a month of this. Cold applications: 22 sent, zero substantive replies past the kill threshold; retired, hours reallocated. Direct outreach variant B (open with the artifact, not the biography): 3 replies from 9 sends against variant A's 1 from 8 not yet conclusive at these sample sizes, but strong enough to run B as default while A finishes its audition. The alumni call in week one became the warm intro in week two became the screen that became the technical a four week trajectory that would be invisible as anything but luck without the campaign ID tying it together. That is the buffer doing its actual job: making the slow causality of a search visible enough to invest in.

Key rule: The compound effect

By month three the buffer is the most valuable artifact you own: hundreds of micro experiments, the templates that worked, the people who replied, the questions asked, the mistakes annotated. Even if the search ends next week, the buffer seeds every future search you will ever run. Back it up monthly. Treat it like the asset it is.

Section notes

Note: Off policy learning

Read other people's buffers when you can: mentor debriefs, alumni calls, public post mortems, accounts from people six months ahead. Their data is noisy but nearly free it costs you a conversation, not a week.

Note: The ensemble

Two or three peers running the same system, sharing dashboards in a weekly thirty minute sync, triples the effective sample size and halves the loneliness. Federated learning, at kitchen table scale. §15 lists isolation among the failure modes for a reason.

Note: A note on grief

Logging rejections immediately, in a structured field, is one of the more emotionally regulating habits in this manual. It converts a feeling into a data point, and the data point is easier to carry.

Note: Don't over-instrument

Thirteen columns is the ceiling, not the floor. A buffer that takes longer to maintain than the actions it records will be abandoned by week five. Friction kills logging; logging is the system.

§ 09 Model-Based Methods - Research as Simulation

Everything so far has been model free: act, observe, update, repeat. But RL has a second tradition. A model-based agent builds an internal model of the environment and plans against it simulating actions cheaply before spending them expensively in the real world. In job search, the model is research, and its economics are extraordinary: a real outreach to a top target is a scarce, unrepeatable sample, while forty-five minutes of research lets you simulate a dozen versions of that outreach and discard the bad ones at zero sample cost.

What a target model contains

For each top ten target, forty five minutes assembles: what the team actually ships and on what stack; what the company says its problems are (earnings calls, engineering blogs, founder posts they will tell you); who the hiring manager is and what they have written or built; the shape of the org around the role; any path however indirect from your network into it. From this you can run the simulation that matters: if I send this specific message to this specific person, what plausibly happens? Most drafts die in simulation. Good. Each death cost nothing.

Modeling the process itself

The other model worth building is procedural: what does this company's loop look like? Recruiter screen, two technicals, a five round onsite? Take home or live coding? Ask the recruiter they will simply tell you. It is free information that most candidates never request, and it converts §12's prep from generic anxiety into targeted rehearsal. A candidate who knows the loop prepares the right thing at the right time; a candidate who doesn't prepares everything badly.

Warning: The trap - research as procrastination

Research feels like progress and costs no courage, which makes it the search's most seductive hiding place. The model is never the reward. Hard cap: forty five minutes per target, then the action ships. If you find yourself on minute ninety reading a founder's podcast transcripts, you are not modeling the environment; you are avoiding it.

The allocation rule that falls out: model based effort for the top ten (campaigns, §02), model-free speed for everything else. Depth where the value concentrates, breadth where it doesn't the same shape as the priority queue, because it is the same principle.

Section notes

Note: Dyna, informally

Sutton's Dyna architecture interleaves real experience with simulated experience from a learned model the agent practices against its imagination between real episodes. Research-then-draft-then-discard is exactly this loop, run by hand.

Note: Models go stale

A target model older than a quarter is a rumor. Re verify the load bearing facts is the role still open, is the EM still there before spending a campaign on them.

Note: Signal in the asking

Asking a recruiter about the loop structure isn't just information-gathering; it reads as seriousness. The question itself is a small positive action.

§ 10 The Craft: Anatomy of Outreach

Templates decay copied widely, they read as copied. Structure endures. What follows is the anatomy of outreach that gets answered: five components, each with a causal story, plus the follow up ladder that multiplies whatever the first touch earns. Learn the anatomy and you can generate fresh instances forever; that is the difference between owning a template and owning the craft.

The five components

The specific hook. The first line proves this message could not have been sent to anyone else: their recent post, their team's launch, the talk they gave, the mutual context. Its function is purely Bayesian it moves the reader's estimate of "mass send" from 95% to near zero in one sentence, and everything after is read differently because of it.
Context, one sentence, chosen not complete. Who you are — selected for relevance to them, not coverage of you. The biography is what the profile link is for.
The value or the ask — never the vague hope. Either give something (a relevant artifact, a genuinely useful observation) or ask something specific and answerable in two minutes. "Any opportunities?" transfers all the work to the reader and converts accordingly. "Did I read correctly that the team is moving retrieval in house? I built something adjacent and wrote up what broke" transfers none.
Friction removal. Make yes cheap: a question answerable by reply, or two concrete time windows, or "no response needed just thought you'd want the link." The easier the yes, the higher the rate.
Length discipline. Under 120 words for cold email; under 80 for LinkedIn. Every sentence past the limit is a tax on the reader that you pay in reply rate.

A compact skeleton, annotated

"Your post on [specific thing] landed — especially [detail]. [hook] I'm an MLE who's spent the last year on [adjacent problem]; [one-line proof with link]. [context+proof] Quick question: is the team [specific, informed question]? [2-min ask] Happy to share the write-up either way — no reply needed if timing's wrong. [friction removal]" — Four sentences. Every component present. Nothing to cut.

The forwardable email: the single best mechanic in networking

When asking a contact for an intro, do not ask them to compose anything. Send them a short, self contained note written to be forwarded in one click that introduces you, states the specific interest, and links the proof. Their entire cost drops to "Fwd: + one line of endorsement." You are not asking for their words; you are asking for their name on top of yours. This one mechanic doubles intro completion rates, because most intros die not from unwillingness but from the composition task sitting in a busy person's queue until it evaporates.

The follow up ladder

The ladder — every outbound thread, automatically

Rung	Timing	Content	Note
D7 bump	Day 7	Two sentences, light, zero guilt-tripping	Often out-converts the original send
D16 value-add	Day 14–18	A new reason to reply: an artifact, a relevant link, a progress note — never "just checking in"	The rung that separates craft from nagging
D30 close	Day 30	"Assuming timing isn't right — door's open, I'll keep following the work"	The closing note gets odd, real replies; graceful exits are rare enough to notice
Quarterly	Every 90d	Passive: thread moves to the network sheet, warmth maintenance only	Dead now ≠ dead forever; regimes change

Two mechanical notes that cost nothing to capture. Timing: send Tuesday through Thursday, mid-morning in the recipient's time zone Monday triage buries you, Friday decay loses you. The effect is small; the price is zero. The two touch open: a substantive comment on something the person published, a few days before the direct message, converts a cold name into a vaguely familiar one. Priming is real, and it is honest work — you did read the thing.

Section notes

Note: Why structure beats templates

A template is a cached output; the anatomy is the generator. Outputs go stale as the market sees them repeated. Generators don't, because each instance is grown fresh from the specific target.

Note: The awkwardness tax

Nearly everyone under follows up because it feels like imposing. Reframe: a busy person's silence is usually a queue, not a verdict, and a well made follow-up is a service to someone who meant to reply. The ladder converts this from an emotional decision into a scheduled one which is the entire trick.

Note: Voice check

Read every message aloud once before sending. If it doesn't sound like a person — your specific person it reads as a template no matter how original it is. The ear catches what the eye forgives.

Part IV: Long horizons - the network, the interview, the ending

§ 11 Networking: Eligibility Traces & the Giving Portfolio

RL distinguishes immediate reward from long-horizon return through the discount factor γ. Networking is the canonical high γ activity: almost nothing done in week one pays in week one. It pays in weeks eight through fifty-two, and in jobs two and three from now. This is exactly why most candidates abandon it, the local gradient feels flat and exactly why the disciplined ones compound past them.

Networking is not an action you take when you need a job. It is a state you maintain so that when you need a job, the action exists. The unwritten rule everyone learns too late

Warmth as an eligibility trace

RL's eligibility traces mark recently visited states so that credit can flow back to them; the marks decay with time unless refreshed. Relationships behave identically, and making the decay explicit turns "who could I ask?" from a vibe into a query. Each contact carries a warmth score in [0, 1] that decays weekly and refreshes on contact:

Algorithm 4: The warmth model

# weekly decay
warmth[c] = warmth[c] * 0.9          # λ ≈ 0.9 per week

# refresh on touch (cap at 1.0)
boost = {substantive_conversation: 0.6,
         gave_them_real_help:      0.8,
         thoughtful_reply:         0.3,
         light_touch:              0.1}   # a real comment, a shared link

# the ask threshold
if warmth[c] ≥ 0.6: eligible_for(intro_request)
elif warmth[c] ≥ 0.3: eligible_for(light_ask, re_warming)
else: re_warm_first()                # never open cold with an ask

The numbers are deliberately rough, the model's value is not precision but the discipline it enforces: at λ = 0.9, a strong conversation decays below the ask threshold in about six weeks, which matches lived experience uncomfortably well, and explains why the ask that follows eight months of silence lands so badly. Re-warm before you ask. Always. A give, a real comment, a short genuine update — then, later, the ask, which now arrives inside a relationship instead of interrupting its absence.

The principles, condensed and extended

Weak ties carry the novel information. Granovetter's 1973 finding, replicated for fifty years: leads come disproportionately from acquaintances, because close ties know what you know. The practical consequence: your second-degree network is the actual pool, and the forwardable email (§10) is the pump.
Give before you ask, and not as a trick. Tactical generosity is recognizable and reads as what it is. Real giving — intros made, feedback offered, useful things shared because they're useful — builds a ledger that survives long silences. Run a small giving portfolio: one or two unprompted helpful acts a month, chosen because you can, aimed nowhere in particular. Its returns are unattributable and large.
Specificity is a courtesy. "Can we chat about your career?" is work; "You moved from analytics into MLE — I'm at that fork, and I'd value twenty minutes on how you chose" is easy. Do the work of being specific so they don't have to.
Maintenance is a separate budget from acquisition. Most people only acquire. Three genuine check-ins a month to existing ties — an update, a question, no ask — outperforms ten new cold connections, because warmth compounds and cold contacts evaporate.
Measure the network, not the conversation. A coffee chat that produced no referral but left the graph bigger, warmer, and better-informed succeeded. The state variable is the network; conversations are just its update rule.

The monthly minimum

Acquire
4–6 new conversations — alumni, second-degree ties, people whose work you genuinely follow. Specific, low-cost asks.

Maintain
3–5 real check-ins with existing ties. No ask attached.

Reciprocate
1–2 unprompted acts of help — the giving portfolio's monthly deposit.

Visible
1 public artifact, so the people who would never be reached by push can still arrive by pull.

Section notes

Note: High γ, formally

Set γ near 1 and future reward counts nearly as much as present reward — under that lens networking is obviously correct. Set γ near 0 and it is provably a waste of the week. The candidates who "don't believe in networking" are not wrong about the evidence; they are running the wrong discount factor.

Note: Compound structure

New contacts unlock their contacts: linear input, roughly geometric output — the same shape as knowledge, and the structural reason patience here beats intensity.

Note: The warmth sheet

One tab: name, context, warmth, last touch, next touch, notes. Fifteen minutes weekly to decay and refresh. The "next touch" column is the network's tickler it makes maintenance a schedule instead of a mood.

Note: Read for context

Mark Granovetter, "The Strength of Weak Ties" (1973). Short, empirical, and still the cleanest statement of why acquaintances move careers.

§ 12 The Interview Sub MDP

Once the funnel produces interviews, a second decision process opens inside the first with its own states (scheduled → prepped → performed → retrospected → awaiting), its own actions, and much better economics: interview experience is dense reward compared to outreach, because every loop teaches whether you pass or fail. The tragedy is how much of that teaching evaporates unlogged. This section is about keeping it.

The question bank: experience replay for interviews

Within two hours of every interview before memory reconstruction sets in, which it does fast and flatteringly — log: every question asked, as close to verbatim as recall allows; your answer quality, graded 1–5 honestly; where you stumbled and why; what you asked them; the interviewer's energy at each segment. After five or six loops for a role family, you will have seen the majority of the distribution you will ever be asked — interview questions cluster far more than candidates expect. From that point, prep stops being generic anxiety and becomes rehearsal against your own observed distribution, weighted by your own graded weaknesses. This is experience replay doing exactly what it does in deep RL: extracting many updates from each expensive episode.

Algorithm 5: Post-interview protocol (≤ 2 hours after)

log(questions_verbatim)          # recall decays by half within a day
grade(each_answer, 1..5)         # honesty here is the whole point
note(stumbles, surprises, energy_reads)
update(question_bank);  update(story_bank_notes)
send(thank_you_note)             # same day; specific, short, real
schedule(next_prep, weighted_by=lowest_grades)

The story bank: policy components, versioned

Behavioral interviews sample from a small space of axes. Hold eight to ten STAR structured stories covering: failure, conflict, leadership without authority, ambiguity, deep technical work, measurable impact, speed under pressure, learning something hard. Each story exists in a two-minute spoken version and a fifteen second compressed one, because interviews demand both. After every use, revise: what landed, what drew skeptical follow ups, what you reached for and didn't have. The stories are versioned policy components — by loop ten they will be unrecognizably better than draft one, if and only if the revision actually happens. One constraint holds absolutely: every story true as told. Refinement means sharper telling of what happened, never embellishment of it polish the lens, not the facts. This is not just ethics, though it is that; embellished stories collapse under the follow up probes of any competent interviewer, and the collapse costs more than a modest true story ever would.

Prep allocation: budget by stage value

Section §03 priced the stages; prep should follow the prices. An onsite is worth four to five times a screen, so it earns four to five times the preparation:

Recruiter screen: ~2 hours. The loop structure (ask! §09), the thirty-second positioning, the "why here" with actual specifics, logistics. Screens filter for coherence and enthusiasm, not depth — over-prepping them is a real and common misallocation.
Technical rounds: 3–5 hours, targeted at the observed distribution from your question bank, not the generic canon.
Onsite: 6–10 hours, spread over days, covering the full loop — technical, behavioral from the story bank, your questions for them (prepared, specific, and real), and energy logistics: sleep, food, and breaks on the day matter more than the tenth practice problem.

On mocks: an hour of live mock with a peer — ideally the §08 ensemble — is worth roughly three hours of solo behavioral prep, because the failure modes of speaking are invisible in silence. Solo drilling remains fine for algorithmic mechanics. Split accordingly.

Sequencing: the curriculum, applied

Per §07: rusty reps are spent where variance is affordable. Order live processes so that genuinely acceptable lower-stakes loops come before the top targets, and never let a dream company be interview number one after a long gap. When scheduling pressure forces a top target early, buy reps synthetically — two or three full-dress mocks in the preceding week, treated with match-day seriousness.

Section notes

Note: Dense reward at last

Outreach pays a bit per week; interviews pay a lot per episode. This inversion is why hours shift toward prep as the pipeline matures — and why the §03 outbound floor exists to stop the shift from going to 100%.

Note: The flattering memory

Un-logged interview memories improve themselves within days — stumbles soften, answers retroactively sharpen. The two-hour rule exists because the data is only honest while it is fresh.

Note: Your questions are also moves

The questions you ask them are read as signal — of seriousness, of taste, of what you'd be like on the team. Prepare them with the same care as your answers. "What does great look like in the first ninety days?" outperforms anything generic.

Note: Rejection debriefs

When rejected after a late stage, ask — politely, once — for any feedback. Most companies decline; the ones that answer hand you off-policy gold. Costs one email.

§ 13 Terminal States: Offers, Negotiation, Deciding

The absorbing state deserves its own policy, because the hours around an offer are the highest-leverage hours of the entire search — and the ones where an unprepared agent's discipline most reliably dissolves into adrenaline. Everything here was set up earlier: the scalarization weights of §05 were written in week one precisely so that this moment would be a comparison against a standard rather than a negotiation with your own excitement.

Five disciplines at the terminal state

Never accept on the call. Express real enthusiasm, thank them warmly, ask for the complete package in writing, and take 48–72 hours minimum. No reputable employer is offended by a day of thought; the ask itself reads as seriousness. Decisions made under adrenaline are policy executed by the wrong subsystem.
Score against the pre committed vector. Pull out the §05 weights and fill them in coldly: compensation, learning slope, sponsorship security, manager quality, mission, optionality. Where the written weights and the felt pull disagree, that disagreement is the interesting data interrogate it in writing before overriding either.
Batch the finals when you can. When one process reaches onsite, tell the other live processes this is standard, expected, and legitimately accelerates most of them. Competing timelines are the strongest negotiating position a candidate ever holds, and they exist only if the §03 outbound floor kept the pipeline alive.
Negotiate, specifically, politely, once. With a written offer in hand at an established company, a justified, specific ask carries material upside and near-zero documented downside. Ground it in something a competing number, market data, the scope of the role make it once, and make it collaborative in tone. This is a decision quality principle, not a script; the point is that silence has a price too, and it compounds over years.
The deadline is also negotiable. An exploding offer's fuse can usually be extended by a polite request for reasonable time. And a company that rescinds over such a request has, at the least, told you something worth knowing before you joined rather than after.

The decision journal: the last buffer entry

Whatever you choose, write the decision down the day you make it: the weights as scored, the runner up and the margin, the assumptions you are consciously making, and most valuable of all what would have to be true in a year for you to regret this. Seal it. This is transfer learning across career decisions: the next fork in the road, years from now, gets to learn from an honest record of this one instead of a memory that will have quietly rewritten itself into inevitability.

Key rule - And then: close the loops

The search's final action is gratitude at scale. Every person who forwarded, referred, advised, or mocked-interviewed gets a short, specific thank you with the news. This is not etiquette or not only. It is the final deposit into the warmth model of §11, made at the exact moment the news gives it maximum value, for the search you cannot yet see coming. The system's last act feeds its next initialization.

Section notes

Note: Why written weights win

An unwritten value function is renegotiated by whichever offer is shiniest. A written one can be revised — but the revision is visible, dated, and has to argue with its predecessor. That friction is the protection.

Note: Multiple offers

Two offers is not twice one offer; it is a different game entirely each is the other's leverage and the other's fallback. This is the compounding return on the unglamorous EV floor of §03, visible at last.

Note: Scope, honestly

Norms here sketch the common case at established employers; startups, agencies, and public sector roles each have their own physics, and none of this is individual financial or legal advice. The durable principles are the ones above: time, writing, and a pre committed standard.

Part V: Robustness - drift, failure, and the first ninety days

§ 14 Non-Stationarity: Seasonality & Regimes

The environment drifts under your policy in two ways: predictably, with the calendar, and unpredictably, with the macro. A learner who cannot tell the difference between the two or between either of them and their own performance will draw exactly the wrong lessons at exactly the wrong moments. This section is the difference.

The predictable drift: hiring seasonality

The hiring calendar: typical rhythm, US tech

Window	Regime	What it means for allocation
Mid-Jan – March	Strong	New budgets unlock; recruiters clear backlogs. Push volume hard.
April – May	Moderate	Steady flow; loops move at normal speed.
June – August	Slow	Vacations stretch loops; interns absorb bandwidth. Expect lag, not silence-as-verdict. Shift hours toward capability and network.
Sept – mid-Nov	Strong	The second wave — teams race to fill before freezes. Push again.
Mid-Nov – Dec	Near-dead	Freezes and holidays. Plant seeds for January; warm the network; build. Do not read December silence as rejection.

Company fiscal years shift these bands locally, and none of it is destiny offers happen in December. But base rates should set expectations, and expectations set morale, and morale is state. A reply rate that halves in July and recovers in September was never about you.

The unpredictable drift: regime shift detection

When your numbers move, run the diagnostic before the self blame:

One channel drops, others hold → a channel or template problem. Yours to fix: rotate the variant, refresh the anatomy (§10), check whether the template has aged into recognizability.
All channels drop together → the market regime changed. Not yours to fix, only to ride: lower volume expectations, lean into the network search (relationship value is far less seasonal than posting volume), and protect morale from a signal that carries no information about you.
Your rates hold but stages slow → process congestion hiring freezes mid loop, reorgs, budget waits. Keep threads warm with the D16 value add rung and widen the funnel.

The sliding window

The mechanical consequence of drift: compute Q-values and stage conversions on the trailing eight weeks, not all time. All time averages are anchors to a market that no longer exists; an eight week window is current enough to track the regime and long enough to hold signal. And keep the graveyard warm — a channel retired under one regime (§07) is a candidate for revival under the next. January resurrects many things that died in November.

Section notes

Note: The market is not your report card

The single most corrosive attribution error in a long search: absorbing a regime shift as personal feedback. The dashboard exists partly to make this error impossible a synchronized drop across channels is exculpatory evidence, timestamped.

Note: Counter-cyclical moves

Slow seasons are when visible work and network deposits are cheapest to make and least crowded. The December post gets read in January by recruiters with fresh budgets.

Note: Change point thinking

Formally this is change point detection on a noisy series with tiny N. Don't over engineer it: the one channel vs all channels test above is the entire useful content of the method at this scale.

§ 15 Failure Modes: An Expanded Taxonomy

Every failure mode below was observed in the wild before it was named here. The original five from the first edition stand, joined by five more that emerge once a system like this actually runs for months. Each carries its recovery.

The original five, condensed

01 · Reward hacking
Optimizing the proxy volume, connections, streaks instead of the objective. Recovery: the one sentence "why this company" test (§05); the −0.5 going through the motions penalty, applied without mercy.

02 · Channel collapse
All capacity into one channel because it produced the last reply. Variance masquerading as signal. Recovery: minimum two active channels always; the exploration budget is not optional (§07).

03 · Premature exploitation
Locking the policy in week two on four data points. Recovery: optimistic initialization, and no kill decisions before N ≥ 20.

04 · Sunk-cost pursuit
Chasing one dream company through months of silence while the pipeline starves. Recovery: the staleness haircut prices the dream honestly; the D30 close (§10) provides the graceful exit ramp.

05 · Burnout spiral
Morale below 2.5 for two weeks; action quality decays; results decay; morale decays further. The loop's one true death spiral. Recovery: the trigger fires automatically from the dashboard cut volume 50% for one week, run Minimum Viable Days, one deliberately good non search thing daily. The system is designed to run at reduced power precisely so it never has to stop.

Five more, from longer runs

06 · Overfitting
Per company resume forks until the core narrative fragments; or overhauling your entire interview style off one loop's feedback — n = 1 is not a gradient. Recovery: one master resume plus two or three role family variants, nothing deeper than bullet reordering per company; interview feedback accumulates in the bank and changes policy only when a pattern repeats.

07 · Variance collapse
The whole pipeline at one maturity stage - all early or all late producing feast famine oscillation and the §03 refill catastrophe. Recovery: the outbound floor (never below 40% of normal volume), and a Friday glance at stage diversity alongside EV.

08 · Novelty churn
Exploration without consolidation: a new channel every week, none ever reaching twenty reps, the whole search a sequence of abandoned pilots. Entertainment wearing exploration's clothes. Recovery: the one-variant-per-week rule cuts both ways new things enter one at a time, and incumbents get their full audition before anything else enters.

09 · Map territory failure
The dashboard is green, the loop hums, and the positioning itself is wrong wrong target market, wrong story, wrong level. The buffer optimizes execution; it cannot detect that the destination is mistaken. Recovery: the monthly zoom-out (Appendix B) exists solely for this: sixty minutes, once a month, asking whether the game is right rather than whether the play is good.

10 · Isolation
Running the entire system alone: sample starved learning, unchecked blind spots, and morale carrying weight it was never meant to carry solo. Recovery: the ensemble (§08) even one peer on a weekly thirty minute sync changes the statistics and the experience of the whole search.

Warning: The meta failure

Abandoning the system in week three because it "isn't working yet." At base rates, weeks one through three are mostly silence by design of the environment median time-to-first-signal on a new thread is two to four weeks. The system's first visible product is not interviews; it is the buffer, the baseline, and the calibrated expectations that make week four survivable. Hold until the data arrives.

Section notes

Note: Failure modes compound

They rarely arrive alone: burnout invites channel collapse (everything into the least painful channel), which invites overfitting to it. The dashboard's job is to catch the first domino.

Note: Kindness as engineering

Half of these recoveries reduce to the same move: lower the stakes of any single unit one application, one interview, one week until no single unit can break you. That is not softness; it is variance management.

Note: Name it to catch it

The taxonomy's practical value is vocabulary. "I think I'm novelty-churning" is a diagnosis and half a cure; an unnamed drift is invisible until it has cost a month.

§ 16 Quickstart: The First Ninety Days

Everything above, as a runnable schedule. The arc: instrument first, baseline second, personalize third by day ninety the defaults in this manual should have been replaced, number by number, with your own.

Week one: instrumentation

Day 1 (45 min): Build the replay buffer from the §08 schema. Add the warmth sheet as a second tab. Do nothing else today; the instrument is the work.
Day 2 (60 min): Run the state audit (§02) in writing positioning sentence, three proof artifacts, honest warm count, binding constraint. Then, while no offer exists to bias you, write the §05 scalarization weights and seal them. This is the day future you gets protected from adrenaline.
Days 3–7: Execute the reference allocation (§06) at whatever your true capacity is. Log every action with its hypothesis. Send at your normal quality this week is a baseline, and a polished baseline is a lie you tell your own instrument.

Weeks two to four: baseline & first updates

Week 2: First Friday review (Appendix B, the five questions). Compute the first dashboard. Introduce the first variant on your strongest-feeling channel. Begin the follow-up ladder on every week one thread the D7 rung is already due.
Weeks 3–4: Enough data exists to compute your first observed stage rates — start replacing the §03 priors. Add the UCB column (Algorithm 3) to the buffer and let it argue with your intuition about next week's allocation. Expect the trough: this is the silence heavy stretch where the meta-failure strikes. The system's answer is the Minimum Viable Day and the base rates you now know.

Month two: the policy becomes yours

Reallocate hours by observed Q, not the reference split. Execute the first kill (something has hit N ≥ 20 by now) and the first double down.
Start the interview curriculum (§12) if screens have arrived: question bank live, story bank drafted, the two hour protocol on every loop.
Recruit the ensemble: one or two peers, a weekly thirty-minute dashboard sync. Month two is when solo searches quietly stall; the ensemble is the countermeasure.

Month three: the personalized system

Move all statistics to the trailing eight week window (§14). Your search now tracks the current regime, not its own history.
Run the first monthly zoom-out if you haven't: is the target market right, is the positioning right, what is being avoided.
Audit against the Numbers Card (Appendix C): every default in this manual should by now be either confirmed by your data or replaced by it. The manual's job was to be outgrown.

The re entry protocol

Every long search contains a lapse — illness, travel, despair, life. The system must be cheaper to re-enter than to abandon, so the protocol is deliberately tiny: write one honest gap row in the buffer ("weeks 5–6: lapsed — moved apartments, morale") and do not backfill anything else. Fabricated continuity poisons the data; an honest gap is just a gap. Then restart with a single Minimum Viable Day. The streak you are protecting was never "days without missing" — it was "always came back."

A closing word. None of this machinery removes the difficulty of a job search. The silences are still silences; the rejections still land. What the system changes is what they mean and what they cost: each one becomes a sample instead of a verdict, logged instead of carried, feeding a policy that gets measurably better while the untracked searcher around you is starting from scratch each Monday on willpower alone. You will not control the market. You will control the learner. Over a long enough search and the searches that matter are always longer than we want the learner wins.

Begin Monday. Forty five minutes, one spreadsheet, the schema in §08. Everything else follows from the loop.

Section notes

Note: Why baseline before optimizing

Week one's only deliverable is truth. Optimizing before measuring guarantees you optimize the wrong thing confidently — the most expensive failure available this early.

Note: The 90-day shape

Instrument → baseline → personalize is the arc of any learning system stood up from nothing. Days 1–7 build the sensor; weeks 2–4 calibrate it; months 2–3 hand the controls to the data.

Note: If the search ends early

An offer in week five doesn't waste the system — the buffer, the warmth sheet, and the story bank transfer whole to the next search, and to the promotion cases and pivots between searches. Nothing here is single-use.

Appendices: Reference matter

Appendix A: Glossary - RL Terms, Translated

The vocabulary, in one place

Term	In the literature	In this search
Agent	The learner and decision-maker	You — specifically, the deliberate version of you running this system
Environment	Everything the agent doesn't control	The market: companies, recruiters, timing, macro
State	The situation description the agent acts from	Pipeline shape, EV, rates, warmth, capacity, morale, runway (§02)
Action	A choice available to the agent	The nine channels, A₁–A₉
Option / macro-action	A temporally extended action	A campaign against one target (§02)
Reward	Scalar feedback signal	The shaped ledger of §05
Return	Cumulative discounted reward	What the whole search delivers, including the network that outlives it
Policy	Mapping from states to actions	Your weekly allocation rule
Q-value	Expected return of an action	A channel's running quality estimate
Value function	Expected return from a state	The stage values of §03; pipeline EV is its sum
Bandit	Stateless action-selection problem	The weekly channel split (§06)
UCB / Thompson	Bandit algorithms balancing explore/exploit	The two spreadsheet columns of Algorithm 3
Epsilon & annealing	Exploration rate, decayed over time	The 25→10% budget schedule against your runway (§07)
Optimistic initialization	High starting estimates to force sampling	Every new channel starts at Q = 0.5
Replay buffer	Stored transitions, revisited for learning	The log — the system's memory and its most valuable artifact (§08)
Off-policy learning	Learning from data another policy generated	Mentor debriefs, peers' dashboards, public post-mortems
Model-based RL	Planning against a learned environment model	Research and simulation before expensive sends (§09)
Eligibility trace	Decaying credit marker on visited states	The warmth score, λ = 0.9 weekly (§11)
Discount factor γ	Weight on future vs. present reward	Why networking is rational: it's a high-γ asset
Credit assignment	Which action caused the reward?	The hypothesis column plus campaign IDs — the only honest answer available
Non-stationarity	Environment changing under the learner	Seasonality and regime shifts; the eight-week window (§14)
Reward hacking	Maximizing the proxy, not the objective	Volume theater; failure mode 01
Curriculum learning	Easy tasks before hard ones	Affordable-variance interviews before dream targets (§07, §12)

Appendix B: Review Liturgies

The Friday five - forty minutes, weekly

What produced the highest reward per hour this fortnight — and does it deserve more hours?
What produced the least and is it approaching its kill threshold, or does it just need its full audition?
What surprised me? Surprise is the fingerprint of a wrong belief; wrong beliefs are the only thing updates are made of.
What am I avoiding, and what is the avoidance costing? The channel unpulled for three weeks is usually the one with the feelings attached.
What one change does next week's policy get? One. A policy that changes everything weekly learns nothing; a policy that changes nothing learns nothing. One deliberate change, logged, is the learning rate.

The monthly zoom out sixty minutes, calendar held

Is the target market still right - the roles, the level, the company shapes I'm pursuing?
Is the positioning sentence still true, and is it what my materials actually transmit?
What would a smart outsider reading my buffer say I am avoiding?
Which belief did I update this month and if none, was I learning or just moving?
Is the pace sustainable for as long as this might actually take?
What is this month's kill-or-double decision the thing that has earned either more of me or none of me?

The weekly review tunes the play; the monthly review questions the game. Failure mode 09 perfect execution of the wrong search — is invisible to the first and obvious to the second, which is the entire reason the second exists.

Section notes

Note: Liturgy, deliberately

The word is chosen for its properties: fixed form, held time, done especially when the mood argues otherwise. Reviews that depend on feeling like reviewing do not survive month two.

Note: Write it down

Answers composed in the head feel complete and aren't. The buffer's Lesson column and a five-line weekly note are the difference between a review and a rumination.

Appendix C: The Numbers Card

Every default in the manual, on one card printed, pinned, and progressively crossed out as your own data replaces it. The right hand column is the whole philosophy: these numbers are scaffolding, not doctrine.

All defaults, consolidated

Parameter	Default	Source	Replace with observed?
Weekly capacity (reference)	15 h	§06	Yes your honest calendar
Exploration budget	25% → 20% → 10% over runway	§07	Keep the schedule; anchor to your clock
Optimistic initial Q	0.5	§07	No, it's a forcing device
Kill threshold	N ≥ 20 and rate < 2%	§07	Keep
Variant policy	One per week, top channel, 2–3 wk audition	§07	Keep
Stage-conversion priors	§03 band table	§03	Yes, first thing to replace
Pipeline EV floor	≥ 1.5	§03	Raise if risk averse or clock bound
Outbound floor (interview weeks)	≥ 40% of normal volume	§03	Keep
Staleness haircut	×0.5 at 21 d; dead at 45 d	§03	Tune to observed loop speeds
Follow-up ladder	D7 · D16 · D30 · quarterly	§10	Keep; tune D16 content to taste
Outreach length	< 120 words email · < 80 LinkedIn	§10	Keep
Send window	Tue–Thu, recipient's mid-morning	§10	Keep free, small edge
Warmth decay λ	0.9 / week	§11	Feel free to tune; keep the decay
Ask threshold	warmth ≥ 0.6	§11	Keep the principle: re warm first
Monthly network minimums	4-6 acquire · 3–5 maintain · 1–2 give · 1 visible	§11	Scale to capacity; keep all four lines > 0
Interview capture window	≤ 2 h post-loop	§12	Keep — memory decays fast
Story bank	8–10 stories · 2-min + 15-sec versions	§12	Keep count; contents are yours
Prep budgets	~2 h screen · 3–5 h technical · 6–10 h onsite	§12	Tune to observed loop difficulty
Offer decision window	48–72 h minimum, in writing	§13	Keep
Statistics window	Trailing 8 weeks	§14	Keep
Morale trigger	< 2.5 avg for 2 weeks → recovery protocol	§15	Keep — it fires so you don't have to
Minimum Viable Day	1 follow-up + 1 log row	§04	Keep — it is the streak
Reviews	Friday 40 min · monthly 60 min	App. B	Keep the cadence; hold the calendar slots
Buffer backup	Monthly	§08	Keep — it's the asset

A Reinforcement Learning Approach to Job Search · The framework is stable by design: markets, titles, and tools will drift; sampling, exploration, memory, and honest review will not · Replace every number with your own; keep every habit · Begin Monday.

DEV Community: Pramod Krishnachari

A Reinforcement Learning Approach to Job Search

Part I: Foundations - the problem, formalized

§ 01 - The Frame

Against the supervised mindset

Three searches run in parallel

One honest caveat

Section notes

§ 02 The MDP: A Full Specification

State s

The state audit — a twenty-minute protocol

Action a

Composite actions: the campaign

Reward r and Transition P(s′ | s, a)

Section notes

§ 03 Funnel Mathematics

Stage conversion bands

Backward induction: what a stage is worth

Pipeline expected value

The refill lag: the classic mid-search catastrophe

Capacity planning: working the math forward

Section notes

Part II: The engine: loops, rewards, and allocation

§ 04 The Algorithm: Daily & Weekly Loops

The daily loop: energy aware scheduling

The lead priority queue

Section notes

§ 05 Reward Shaping: Surviving Sparsity

The reward is a ledger, not a feeling

Multi objective reward: the offer is a vector

Section notes

§ 06 Channels as Bandit Arms

Allocation, formally: UCB and Thompson in a spreadsheet

The reference allocation: a fifteen hour week

Section notes

§ 07 Exploration: Discipline, Decay, Curriculum

Directed exploration: every experiment carries a question

Curriculum learning: sequence the difficulty

Section notes

Part III: Memory & models - the buffer, the research, the craft

§ 08 The Replay Buffer

The dashboard: six numbers, computed weekly

A worked trace: four weeks, compressed

Section notes

§ 09 Model-Based Methods - Research as Simulation

What a target model contains

Modeling the process itself

Section notes

§ 10 The Craft: Anatomy of Outreach

The five components

A compact skeleton, annotated

The forwardable email: the single best mechanic in networking

The follow up ladder

Section notes

Part IV: Long horizons - the network, the interview, the ending

§ 11 Networking: Eligibility Traces & the Giving Portfolio

Warmth as an eligibility trace

The principles, condensed and extended

The monthly minimum

Section notes

§ 12 The Interview Sub MDP

The question bank: experience replay for interviews

The story bank: policy components, versioned

Prep allocation: budget by stage value

Sequencing: the curriculum, applied

Section notes

§ 13 Terminal States: Offers, Negotiation, Deciding

Five disciplines at the terminal state

The decision journal: the last buffer entry

Section notes

Part V: Robustness - drift, failure, and the first ninety days

§ 14 Non-Stationarity: Seasonality & Regimes

The predictable drift: hiring seasonality

The unpredictable drift: regime shift detection

The sliding window

Section notes

§ 15 Failure Modes: An Expanded Taxonomy

The original five, condensed

Five more, from longer runs

Section notes

State `s`

Action `a`

Reward `r` and Transition `P(s′ | s, a)`