yelgabo

yelgabo/job-fraud · system report · 6 November 2026

Job-fraud agentic flow

How a WorkBC job posting becomes a published Low, Medium or High rating. This report covers which steps are plain code, which are single model calls and which are agents, and the rules that stop any model or agent from publishing a claim on its own.

The short version

job-fraud collects WorkBC postings and rates each one on whether it looks like a genuine attempt to hire locally, as opposed to an advert placed to show that no Canadian applied. The rating is published next to a real employer's name, and the whole design follows from that. Models and agents gather evidence. Code turns the evidence into the number.

There are three phases, and they share one Postgres database without calling each other. A scraper collects postings and marks them pending. A judge evaluates pending postings. A read-only Next.js site shows the judged ones. The judge has two front ends. The API judge is one Node process that calls Claude through the Anthropic API. The keyless judge is a Claude Code session that sends helper agents to research employers on the web. Both hand their evidence to the same composer, so the same evidence gets the same score whichever path gathered it. The evidence itself can differ. The API judge's verifier sees an 800-character description excerpt where keyless helpers see 1,800, and the API path records verbatim search blocks where keyless helpers report their own searches.

Where things stand today

These counts come from a read-only query against the production database on 6 November 2026.

Postings
23,751
4,544 employers
Judged
23,021
shown on the site
Low
21,508
93.4% of judged
Medium
1,310
5.7% of judged
High
203
0.9% of judged
Pending
730
waiting for a judge run

The scheduler was scraping but not judging. The newest scrape was from 2 November 2026 at 12:24 UTC, the Monday cron run, while the newest judgment was from 27 October, and 730 postings were waiting. The scheduler service turned out to have no ANTHROPIC_API_KEY, and its script only judges when that key is set. Later on 6 November the backlog was drained with the keyless judge (729 postings scored, one left pending) and the key was added to the scheduler, so the Monday run now judges up to 1,000 postings after scraping.

Most published ratings predate the composer. Of the 23,021 judged postings, 17,033 have scoringVersion null, which means a model chose the number before the composer existed (or the impersonation path wrote it). Another 1,662 are version 1 and 4,326 are version 2. So the rule that no model chooses the number covers about a quarter of what the site shows today. Moving the rest onto the composer needs a fresh Jev call per posting, because those rows store no judgments for npm run recompose to reuse.

Employer verdicts have mixed origins as well. 445 came from Haiku 5.5 alone, 50 from Haiku followed by Opus, and 295 from Opus alone (the re-check script). Another 365 were presumed from an own-ATS match, 963 come from the web check as it ran before the tiers, and 2,426 employers carry neither marker, either because they were never checked or because their verdict predates both fields.

The system in one picture

System overview WorkBC APIs return postings to the scrape CLI, which upserts them into Postgres as pending. A weekly scheduler runs the scrape and, when keyed, the API judge. Either the API judge or the keyless judge reads pending postings from Postgres and writes employer verdicts and scores back. The API judge calls the Anthropic API; both judges call TypeSafe Jev. The web app reads judged rows, and the owner files review requests through its token-gated audit pages. Judge: one path per run, one DB writer WorkBC APIssearch + job detailpublic JSONScrape CLIregex flags, NOC, ATSno model, no keyPostgresEmployer, Job rowsJudgeRequest, search logWeb app (Next.js)public pages, 10 min cachetoken-gated /auditScheduler (cron)Mondays 12:17 UTCscrape x2, judge if keyedOwner (admin)reviews postings on /auditAPI judgenpm run judgeone Node processKeyless judgeClaude Code sessionHaiku + Opus helper agentsAnthropic APIHaiku 5.5, Opus 5.5web_search toolTypeSafe Jevtyped questions aboutthe posting text postingsupsert aspendingjudged rowsflag requestsruns scraperuns judge, if keyedrerun / deeppending postingsverdicts, scoresverify, impersonationJev questions Stacks = several parallel workers or agents. Cylinders = stored state.
The three phases share one database and never call each other. The keyless judge's web research runs inside the Claude Code session, so only the API judge talks to the Anthropic API. Both judges ask Jev the same text questions.

The scraper is deterministic. It reads WorkBC's JSON search and detail APIs, runs regex checks over each description (free email domain, mail-in résumé, crypto payment, banking details or ID before an offer, a fee to apply, WhatsApp or Telegram only), buckets the NOC occupation code into a job category, and classifies the apply link's host against a registry of applicant tracking systems such as Workday and Greenhouse. New postings arrive with scoredAt null, and null is what pending means everywhere else in the system. Re-scraping a posting that is already judged updates its fields and leaves it judged.

The site only shows judged rows. Public pages are cached for 600 seconds, so a judge run is visible within ten minutes, or immediately when REVALIDATE_TOKEN is set and the run can call POST /api/revalidate. There is no staging database. A local judge run writes to production and is live once the cache turns over. The site's only write path is the token-gated /audit mirror, where the owner can leave a note or ask for a posting to be judged again.

Two judge paths, one scorer

One predicate picks the path: is ANTHROPIC_API_KEY non-empty in .env? If it is, npm run judge does everything in one process. If it is not, the keyless judge runs inside a Claude Code session and uses that session's agents for web research. A second predicate, TYPESAFE_API_KEY, decides whether Jev answers the text questions. Without it the regex flags stand in for Jev's route judgment and the row stores judgments = null.

API judgeKeyless judge
Commandnpm run judgejudge-keyless:fetch, then helper agents, then judge-keyless:apply
Who researches employersClaude through the API, with the server-side web_search toolClaude Code helper agents with the WebSearch tool
Escalation to OpusDone in code by verifyEmployerTieredDone by the orchestrating session; apply rejects any verdict that needs escalation and is not marked as Opus-confirmed
Brand-impersonation checkOpus 5.5 API call on each posting whose apply link names a different companyOpus helper agents, one answer per employer and tenant pair; apply holds a posting that needs the check until a result clears it (since 6 November 2026)
Search trail on /auditVerbatim API search blocksThe helper's own list of queries and result URLs, labelled agent-reported
Search limitNone per run; $10 per 1,000 searchesAbout 200 per session, refilling about 100 an hour
Prompt lives inlib/ai/verify-employer-web.tsThe "Agent prompt" section of .claude/skills/judge-postings/SKILL.md
Used forBulk and scheduled runsMachines without the key, and small high-scrutiny sets

What the two share matters more than what differs. Both decide each employer with verificationAction, both enforce the same escalation rule, and both score through buildVerdict in lib/scoring/verdict.ts. The grouping differs in one detail. The keyless fetch groups through planByEmployer, which sends a posting with no employer to the web check on its own, while judge.ts groups inline and gives such a posting no employer check at all.

A failed employer check holds its postings back. When a check fails for any reason other than billing (a refusal, a malformed answer after the retry, a failed Opus re-check), judge.ts logs it and stage 2 skips that employer's postings, which stay pending. Until 6 November 2026 stage 2 scored them anyway on whatever verdict was cached. A placeholder key still causes trouble. It counts as set, every Anthropic call returns 401, and while postings that needed a check stay pending, postings whose employer has a cached or presumed verdict are scored without the impersonation check. On a keyless machine, leave the ANTHROPIC_API_KEY line in .env commented out.

The keyless judge, step by step

This is the multi-agent part of the system. One Claude Code session owns the whole run. It runs the fetch CLI, spawns helper agents, writes their output to files, and runs the apply CLI. The repo's update-postings skill wraps it, and judge-postings holds the procedure and the agent prompt. The skill insists that one session own the run rather than handing batches to separate top-level sessions, because a single owner can restart the run and its progress stays visible.

Keyless judge, one run Inside one orchestrating Claude Code session: the fetch CLI reads pending postings from Postgres and writes batch files, one item per employer, cached.json, and impersonation files, one item per employer and ATS tenant pair. Haiku helper agents each take one batch; verdicts needing escalation go to Opus helpers, whose verdicts replace Haiku's. Opus impersonation helpers answer the impersonation items. The apply CLI writes the impersonation answers first, then validates and stores employer verdicts, and scores postings through Jev and the composer before writing to Postgres. One orchestrating Claude Code session runs every step inside this box Postgrespending rowsEmployer.checks.webjudge-keyless:fetchread-onlygroups by employerbatch-NNN.jsonemployeremployeremployerHaiku helpersone per batch fileWebSearch, returns JSONOpus helpersre-check alarmingverdicts onlyverdicts-N.jsonone web verdict per employerverifiedBy and searchLogcached.jsonpostings whose employerneeds no new checkjudge-keyless:applysingle DB writerimpersonation answers firstrejects bad schema, missing Opus checkthen Jev + composer per postingTypeSafe Jev5 questions per postingimpersonation-NNNpairpairpairOpus impersonationone helper per fileimpersonation-resultsame, affiliate, uncertainor impersonation pending postingswritesfan outneeds escalationkept as haiku-5.5haiku-5.5>opus-5.5employers thatneed no checkcached postingsverdict filesstore verdicts, write scoresposting textapply link namesanother companyfan outanswersapplied first
Helpers only research and return JSON. The orchestrator writes their output to files, and the apply CLI is the one process that touches the database. A verdict that fails the schema or the tier rule is skipped, and a posting whose apply link names another company is held until an impersonation answer clears it; either way those postings stay pending for the next fetch.
Keyless judge sequence Time runs downward. The orchestrator runs fetch, which reads Postgres and returns batch, impersonation and cached files. It sends batches to Haiku helpers in parallel, sends verdicts needing escalation and the impersonation items to Opus helpers, writes the answers to files, runs apply as a dry run and then for real. Apply writes impersonation answers first, stores employer verdicts and search logs, and writes composed scores. The orchestrator repeats until fetch reports zero pending. OrchestratorClaude Code sessionfetch CLIread-onlyHaiku helpersOpus helpersapply CLIsingle writerPostgres 1. run fetch --limit N2. read pending rows, oldest first3. batch + impersonation files4. agent prompt + one batch each, in parallel5. verdicts + searchLog, no scores6. pick verdicts that need escalation7. escalation batch, same prompt8. replacement verdicts9. impersonation items, impersonation prompt10. same / affiliate / impersonation answers11. write verdict and impersonation-result files12. apply --dry-run <dir>13. would score N; tier rejects; held postings14. apply <dir>15. impersonation first16. checks.web + search logs17. write composed scores18. scored N, skipped M19. fetch again until 0 pending
Steps 4, 7 and 9 are the only places agents act, and all three return JSON to the orchestrator. Step 12 is a dry run that validates every answer and lists tier violations and postings held for a missing impersonation answer before anything is written. Jev is called inside step 17 and omitted here.

What a helper receives

The fetch step writes one item per employer that needs a web check, 15 employers to a batch file by default. The item carries the employer's representative posting, chosen as the first posting that gives a mailing address when any does (pickRepresentative). The mailing-address type is the heaviest signal the check produces, and the verifier can only judge an address it has been shown.

{
  "workbcId": "49588691",
  "title": "...", "employer": "...", "location": "...", "salary": "...", "postedAt": "...",
  "atsProvider": "workday", "externalApplyOk": null,
  "flags": [{ "flag": "mail_physical_resume", "evidence": "..." }],
  "descriptionExcerpt": "first 1800 characters of the description",
  "employerPostings": ["49588691", "49588702"]
}

employerPostings is for the apply step, which scores every posting in that list with the employer's verdict. The helper does not need it.

What a helper returns

A JSON array with one verdict per item. The verdict describes the employer. It has no score field.

{
  "workbcId": "49588691",
  "web": {
    "websiteUrl": "https://acme.com",
    "websiteReachable": "yes",          // yes | no | unknown
    "businessMatch": "match",           // match | mismatch | uncertain
    "locationMatch": "match",           // match | mismatch | uncertain
    "hasJobsListing": "yes",            // yes | no | unknown
    "applicationAddressType": "business",
    "confidence": 0.85,
    "summary": "at most 400 characters, shown on the site",
    "verifiedBy": "haiku-5.5"           // set by the orchestrator
  }
}

applicationAddressType is one of business, residential, home_business, po_box, virtual, none or uncertain. home_business needs two sources: the company's own website and at least one independent listing must both give the home as the business address, and otherwise the answer is residential.

What a helper may not do

  • Write to the database. Helpers return JSON to the orchestrator and nothing else. This rests on the prompt and the skill; no tool restriction enforces it.
  • Choose a score. The apply step parses verdicts with a zod schema that keeps only workbcId and web.
  • Return a verdict it did not investigate. If a search fails or the session's search budget runs out, the helper leaves that employer out of its array. An uninvestigated uncertain would be cached on the employer and reused by every later run, while a missing verdict only keeps the postings pending.
  • Call an employer a mismatch because its industry differs from the role. A retailer hiring a developer is a match. Mismatch is reserved for fake, shell, parked or impersonating entities.

The gates in the apply step

judge-keyless:apply is the only place a keyless score is computed. For each verdict, in order:

  1. Parse it with VerdictSchema. A partial or malformed web object fails and the employer's postings stay pending.
  2. Check the tier rule with tierViolation. A verdict that needs escalation must be marked opus-5.5 or haiku-5.5>opus-5.5; anything else is rejected. On the API path code escalates, and here this check is what catches a skipped escalation step.
  3. Store web on the employer once, then ask Jev and compose a score for each pending posting in that employer's employerPostings. A Jev error stops that employer's remaining postings, which stay pending, though the employer verdict is already stored.

Postings in cached.json get one more check. The apply step re-decides each of those employers against all of its current pending postings. If a new posting has arrived since the fetch, for example one with a mailing address the cached verdict never saw, that employer now needs a web check and its postings stay pending for the next fetch. The orchestrator runs apply with --dry-run first, which validates everything and reports tier violations without writing, and then loops fetch and apply until fetch reports zero pending.

The tiered employer check

Both paths check employers with Haiku 5.5 first. Opus 5.5 checks again, and its verdict replaces Haiku's, when needsEscalation in lib/ai/verify-employer-web.ts returns true:

Haiku's verdictWhy it gets a second look
businessMatch: mismatchAdds 25 points. In the model evaluation Haiku found nothing on a real Vancouver payments company and called it a mismatch, which scored 85, a High rating on a real employer.
locationMatch: mismatchAdds 12 points.
Address residential, po_box or virtualOn every posting that mails applications, adds 40 points and floors the score at 70, which is High.
Address home_businessThis label lowers a rating compared with residential, so the cheap model may never give it alone.
Address uncertain when the posting gives a mailing addressOn 1,000 postings Haiku left a house address uncertain where Opus found the residence.

The split rests on docs/verifier-model-eval.md. On the 21 hand-labelled postings, Haiku followed by Opus on alarming verdicts agreed with the labels 20 times out of 21, the same as Opus alone, and escalated 2 of the 21. The estimated cost was about $0.003 per posting against about $0.026 for Opus on everything, roughly an eighth. Opus also gets up to 10 searches per check instead of 5, because at 5 searches 39 of the 584 verdicts stored on 27 October said the search had been cut off.

A cached employer verdict is reused unless one of four things is true. The run is a --rejudge (an API-judge flag); the cached verdict was presumed from an own-ATS match and a new posting does not apply through that tenant; the cached address type is none but a pending posting gives a mailing address; or the cached verdict breaks the tier rule because it was written before the tiers existed. When the re-check script ran over the first 73 such pending-side verdicts, Opus overturned 47 of them.

The API judge

API judge, two stages Stage 1 groups pending postings by employer and decides reuse, presume or web for each. Web-checked employers go to Haiku 5.5; verdicts needing escalation go to Opus 5.5. Presumed and checked verdicts are written to the employer row. Stage 2 runs an apply-host check per posting. A tenant mismatch goes to an Opus impersonation check; a confirmed impersonation is re-attributed and written as score 90. Everything else goes to buildVerdict, which asks Jev and composes the score written to the job row. Stage 1: once per distinct employer, 4 at a timeStage 2: once per posting, 8 at a time Postgrespending rowsGroup by employerverificationAction()web, presume or reuseReuse cached verdictno model callPresume legitimateall apply via own ATSHaiku 5.5 checkweb_search, max 5Opus 5.5 re-checkweb_search, max 10Employer rowchecks.web+ search logApply-host checkATS tenant in apply URLvs. named employerOpus impersonationsame, affiliate, uncertain,or impersonationRe-attribute, score 90skips the composerbuildVerdict()Jev questions + composerJob rowscore, band,reasoningTypeSafe Jev5 typed questions pending postingsreusepresumewebneedsescalationdeterministic verdictOpus verdictno escalation: Haiku verdict kepteach postingtenantmismatchno mismatchimpersonationcleared or uncertainscore, band, signalsposting text
Stage 1 finishes before stage 2 starts, so every posting is scored against its employer's newest verdict. An out-of-credit billing error in either stage stops new work and aborts the run, leaving unscored postings pending.

The API judge is agentic in a narrower sense than the keyless path. Each employer check is one Messages API request that carries two tools: the server-side web_search tool, capped at 5 uses for Haiku and 10 for Opus, and record_web_verification, a client tool whose input schema is the verdict. The model chooses its own search queries and runs them on Anthropic's side. If the server-side loop pauses with pause_turn, the code resends the turn, at most three times. The answer counts only if it arrives as a record_web_verification call that passes zod. Otherwise the request is retried once and then counted as failed. Every search block, including its encrypted result content, goes into EmployerWebSearchLog so the /audit pages can show what the model consulted.

Stage 1 checks four employers at a time and stage 2 scores eight postings at a time, both through p-limit, and the SDK retries rate-limit and overload errors up to five times. Two failure rules keep partial runs safe. An out-of-credit error, which arrives as a 400 and is never retried, sets a flag that turns every queued task into a no-op, and the run throws after the current stage with the completed rows saved. A Jev failure leaves that one posting pending, because composing it without judgments would store a different kind of row under the same scoringVersion.

The weekly job-fraud-scheduler Railway cron service runs this path. On Mondays at 12:17 UTC it runs both scrape passes and then npm run judge -- --limit $JUDGE_LIMIT (1,000 by default) if the service has an Anthropic key. It reaches Postgres over Railway's private network.

The brand-impersonation check

Before scoring each posting, both judges compare the applicant tracking system tenant in the apply link with the employer the posting names. On the keyless path the fetch writes one impersonation-NNN.json item per employer and tenant pair, an Opus helper agent answers it, and apply writes the answer through the same function the API judge uses. Below, the API judge's version. If a posting says it is from one company but its Workday link belongs to another, Opus 5.5 investigates with web search and classifies the relationship as same, affiliate, impersonation or uncertain. The prompt tells it to choose impersonation only when it is confident the two companies are unrelated, so a subsidiary applying through its parent's hiring system is cleared. Results are cached per employer and tenant pair for the rest of the run.

This is the one place the system writes a fixed number without the composer. The write sets scoringVersion to null, and every re-scoring script (backfill-ats, recheck-cached-verdicts, recompose, reverify-mail) skips rows whose reasoning starts with that prefix. Before 6 November the write left scoringVersion unchanged and recompose did not check the prefix, so a re-judged composed row could have lost its 90; neither of the two impersonation rows in the database was affected. Opus runs this check because a smaller model labelled a real construction company's division as an impersonation of its parent group. It moved from Opus 4.8 to Opus 5.5 on 27 October without a side-by-side comparison, and no labelled impersonation cases exist yet. Both October runs logged zero impersonations.

How the score is composed

How the composer turns evidence into a band Four inputs, the employer web verdict, regex flags, NOC category and Jev judgments, feed a route gate, then the weight table, then a sum from baseline 20 with floors, then the band cut-offs. A score scale from 0 to 100 marks Low below 30, Medium 30 to 69, High 70 and up, with three example postings at 16, 40 and 80. Employer web verdictbusiness, location, addressRegex flagsfree email, mail-in, feesNOC categorysoftware roles weigh mail-inJev judgmentsroute fit, pre-hire askRoute gateroute fit under 0.57withholds brand creditsWeight tableeach signal adds-25 to +40 pointsSum and floorsstart at 20, clamp 0-100private address: min 70Bandunder 30 Low30-69 Medium70 and up HighScore, 0 to 100LowMediumHigh03070100baseline 20pub on gmail: 16chain on gmail: 40mails to a house: 80
The route gate decides whether a verified employer counts in the posting's favour. The floors are lower bounds, so no credit can pull a posting that mails applications to a house below 70. The three examples are computed by lib/scoring/compose.ts and listed in the worked-examples table below.

composeScore in lib/scoring/compose.ts is a pure function: the same inputs always give the same output, with no clock, network or model. It works in this order:

  1. Decide whether the application route is disowned. With Jev judgments, it is disowned when routePlausibleForEmployer is below 0.57. Without them, it is disowned when the posting has a free-email or messaging-app-only flag. A disowned route withholds the brand credits (business match, location match, jobs page, business mailing address, no mailing address) while the penalties on the same fields still apply. Confirming that a national chain exists says nothing about a posting that collects replies at a Gmail address.
  2. Apply address signals only to postings that mail applications. The verdict is stored per employer, and a sibling posting that applies online should not inherit one location's street address.
  3. Add a signal for every condition in the weight table that fires. Each condition compares against an explicit value, so a check that returned null, meaning not checked, never counts against a posting.
  4. Add the one weighted Jev judgment, askBeforeHire, worth up to 25 points and nothing at or below probability 0.5.
  5. Sum from a baseline of 20 and clamp to 0 to 100.
  6. Apply the floors. A residential, P.O. box or mail-forwarding address sets a minimum of 70, and so does askBeforeHire above 0.8.
  7. Map to a band. Under 30 is Low, 30 to 69 is Medium, 70 and up is High.

The published reasoning is generated from the composed result by explain.ts, so the prose can never disagree with the number beside it. Before the composer a model wrote that prose without knowing the final score.

Worked examples

Each row below was computed by running composeScore on the stated inputs.

PostingSignals that fireScoreBand
Small local pub, verified, applications to a Gmail address, route fit 0.77baseline 20, free email +20, business match -15, location match -5, applies online -416Low
National chain, verified, applications to a Gmail address, route fit 0.10baseline 20, free email +20; every brand credit withheld because route fit is under 0.5740Medium
Software role, résumés mailed to a house, route fit 0.30baseline 20, private address +40, mail-in for a software role +20; floor 70 not needed80High
Web-verified employer, software role applying through its own Workdaybaseline 20, known ATS -25, business match -15, jobs page -7, location match -5, applies online -40Low

The first two rows are the case the route gate exists for. The regex sees the same free-email flag on both, and Jev's route judgment is what separates a neighbourhood pub on Gmail from a national chain on Gmail.

The weight table

From lib/scoring/weights.ts. Every number except apply_address_home_business (the owner's call on 27 October) was transcribed from the midpoints of the old prompt's prose ranges, and none has been fitted. A reweight is a diff to this file followed by npm run recompose, with no model calls.

SignalPointsFires when
apply_address_private+40The posting mails applications to a residential, P.O. box or mail-forwarding address (floor 70)
apply_address_home_business+25It mails to a home the verified employer publishes as its address (no floor)
business_mismatch+25The web check found a fake, shell, parked or impersonating entity
crypto_payment, banking_info_upfront+25Regex on the description
generic_email_domain+20Applications go to a free mailbox (+5 more if the website is unreachable)
fee_to_apply, id_upfront, whatsapp_telegram_only+20Regex on the description
mail_resume_software_role+20A software or IT role asks for mailed résumés
address_not_geocoded+20A stated address did not resolve, or matched with confidence under 0.5
address_city_mismatch+15The address resolves to a different city
location_mismatch, website_unreachable+12Web check or site probe
pre_hire_ask (Jev)up to +25Probability of a pre-offer money or ID request above 0.5 (floor 70 above 0.8)
apply_address_business, location_match, address_city_match-5Credits, the first two withheld on a disowned route
apply_address_none-4The posting asks for no mailed materials (withheld on a disowned route)
jobs_listing-7The employer's site has a careers page (withheld on a disowned route)
business_match-15The web check found a real company (withheld on a disowned route)
ats_known_provider-25The apply link goes to a recognised tracking system

Jev answers five questions per posting and only one of them is weighted. routePlausibleForEmployer drives the route gate. employerIsOrganisation, brokerRouting and brandProminence are stored on the row and not weighted, because a question earns a weight only after npm run measure-questions shows it separates the bands. Brand prominence is the instructive case. It reads about the same on low and medium postings because the medium band is full of big brands, so weighting it directly would penalise size.

Guardrails

These are the properties a reviewer should check are still true after any change to the judge.

The human loop

There is no public correction route, by design. The README says a rating is an automated screening signal that can be wrong. What exists is an owner-only queue on /audit/<token> pages, the app's only write path. The owner can add a ReviewNote, or file a JudgeRequest of one of two kinds. A rerun clears the posting's scoredAt at once, so the next judge pass re-scores it, and the API judge marks the request resolved when that happens. A deep request keeps the current score and queues the posting for a single-agent investigation with the owner's note passed in as context.

The posting stays on the site with its old score while the agent works. When the agent's verdict names the flagged posting, judge-keyless:apply re-scores it even though it is judged and stamps the request resolved. Before 6 November apply only scored pending postings, so a deep verdict updated the employer but never the posting the owner had flagged.

The update-postings skill runs a full refresh in this order:

  1. Drain the review queue: send each deep request to one fraud-analyst agent with the owner's note and apply its verdict, which re-scores the posting and stamps resolvedAt.
  2. Scrape twice, a term pass for software roles and a city-wide Victoria pass. Both are needed because they collect different postings.
  3. Pick the judge path from the ANTHROPIC_API_KEY predicate.
  4. Judge in waves, applying after every wave so an interrupted session loses nothing, until fetch reports zero pending.
  5. Report counts and cost. Data updates never need a deploy.

docs/self-improving-loop.md describes where this is meant to go: drift reports that file their own review requests, review decisions that become labels, and weight proposals measured against those labels in CI. Two steps stay human on purpose, labelling and merging rubric changes, because every merge changes published claims about named employers.

Cost and throughput

Both rows are API-judge runs, taken from the summaries the judge printed (files under logs/). Cost is the web-check estimate at list price and excludes Jev.

RunPostingsEmployers web-checkedPresumed (own ATS) Escalated to OpusSearchesWeb-check costWall timeBands L / M / H
26 Oct 20261,0002711337846$16.09842 s708 / 250 / 42
27 Oct 2026, backlog4,24321910316585$9.14833 s4,240 / 3 / 0

The backlog run shows why grouping by employer matters. It scored 4,243 postings with 219 web checks, because most of those postings belonged to employers with a reusable or presumed verdict. Jev's text questions cost about $2 for a whole corpus, so employer research is where the money goes.

Known gaps

File map

FileRole
scripts/judge.tsThe API judge: both stages, the billing abort, rerun-request resolution
scripts/judge-keyless-fetch.tsKeyless step 1: read pending, group by employer, write batch files and cached.json
scripts/judge-keyless-apply.tsKeyless step 3: the single writer, schema and tier gates, scoring
.claude/skills/judge-postings/SKILL.mdKeyless procedure, escalation step, search budget, the agent prompt
.claude/skills/update-postings/SKILL.mdThe full refresh: review queue, two scrape passes, path choice, waves
lib/signals/verification-plan.tsverificationAction, planByEmployer, the presumed verdict
lib/ai/verify-employer-web.tsEmployer check prompt and tools, needsEscalation, tierViolation, verifyEmployerTiered
lib/ai/verifier-models.tsModel profiles, search caps and list prices
lib/ai/check-impersonation.ts, resolve-impersonation.tsThe Opus impersonation check and its re-attribution write
lib/ai/jev-judgments.tsThe five Jev questions
lib/scoring/verdict.tsbuildVerdict, the per-posting pipeline both paths call
lib/scoring/compose.ts, weights.ts, explain.tsThe composer, the weight table, the published prose
lib/shared/risk-band.tsBand cut-offs at 30 and 70
scripts/scheduled-refresh.shThe weekly Railway cron: two scrapes, then the judge if keyed
app/audit/[token]/j/[id]/actions.tsOwner review actions: notes, rerun and deep requests