Machine learning · Labor-market signals
Hiring signals from organizational semantic drift
Abstract
A nightly SLURM pipeline embeds each organization's authored public documents — grants, preprints, trial registrations, filings — with Qwen3-Embedding-8B and reduces consecutive documents to a per-√day semantic-shift series, so a two-day pivot in terminology space outranks the same drift spread over eight months. A 2-state Gaussian HMM separates steady voices from teams pivoting hard, and regime-switched Euler-Maruyama Monte Carlo, 4,000 paths of 90 daily steps, yields a next-quarter escalation probability. A Hawkes self-exciting process on posting-arrival times flags an organization hiring hot even when net headcount is flat, and reposts are caught by embedding match (cosine ≥ 0.92) so a re-listed role cannot reset its age. Fifteen-plus public sources feed the corpus; a LangGraph DAG under Qwen3-Next-80B, with a Llama-3.3-70B cross-model critic on four-GPU nights, drafts evidence-cited memos. The "long-open posting means desperate team" boost was killed in red-team: posting age predicts ghost jobs, and interview odds run 2–3× in the first 48 hours, so Day-1 postings lead. No send path exists.
keywords: semantic drift · Gaussian HMM · Hawkes process · Brownian scaling
## The pipeline and its sources
A 16-CPU stage ingests fifteen-plus public sources into a SQLite corpus: NSF, NIH, and USAspending grants; arXiv with OpenAlex affiliation resolution; ClinicalTrials.gov; USAJOBS; eight ATS providers' public APIs including Workday, where big pharma and the FFRDCs hire; SEC Form D; 10-K Item 1A risk factors; USPTO patents; H-1B LCA salary disclosures; and commit-author emails on a target's own public GitHub repositories. A GPU stage serves Qwen3-Next-80B FP8, with a Llama-3.3-70B cross-model critic on four-H100 nights and an A100 pair on the fallback tier, and walks a LangGraph DAG through embedding, regime analysis, a 70B audit, memo synthesis, and proof-of-work brief nomination. The corpus is persistent, so trajectories get denser every night; a one-time backfill of about three years of awards, preprints, and Form D raises makes the regime machinery work on day one rather than day thirty.
## Semantic drift per √day
Each document is chunked, embedded, and mean-pooled to one vector ei; an organization's documents are ordered by date. For consecutive authored documents — grants, papers, and trial descriptions, with job postings and Form D boilerplate excluded because a board's posting variety is cross-sectional, not pivoting — the shift is the cosine distance between successive statements, scaled by the square root of the elapsed days:
The Brownian scaling is the point: a 0.3 jump over two days is violent, the same jump over eight months is drift. NSF and NIH organizations are keyed at the PI level, PI·Institution, so a university's regime measures one lab's voice rather than cross-department heterogeneity. A stable voice takes small steps in a consistent direction; a team fighting a problem beyond its expertise pivots through terminology space in large, erratic steps.
## Regimes and escalation
A K = 2 Gaussian HMM (hmmlearn, full covariance, 200 EM iterations, best of five random restarts by likelihood) is fit to the normalized δ series, yielding per-regime means and variances (μk, σ2k) and a transition matrix A. The high-variance regime is the Hurdle State; an organization is labeled hurdle_state only if its current decoded state is the high-variance one and σ2high > 1.5·σ2low, and the label needs at least five authored documents. Escalation probability is Euler-Maruyama Monte Carlo on the fitted dynamics, 4,000 paths of 90 daily steps at Δt = 1/90, regime switching included:
Escalation is the probability the path mean exceeds μhigh: the chance the organization's semantic volatility runs at Hurdle-State levels over the next quarter. The hurdle probability is 0.6 for being in the hurdle state now plus 0.4 times escalation; thin-history organizations of two to four documents fall back to a volatility proxy capped at 0.5, so none is shortlisted on volatility it cannot yet prove.
## Ranking and its rejected priors
Alignment is half the cosine between an organization's recent semantic state and the pooled profile vector, half the mean of the top-two cosines against facet embeddings, where each expertise-matrix item and each project write-up is embedded on its own. A pooled vector sits between skill clusters and under-scores an organization that hits one skill hard; the facet term fixes that after a pooled-only vector diluted multi-cluster expertise. The prescore is 0.65 alignment plus 0.20 recency (90-day exponential half-life) plus 0.15 hurdle probability; hurdle was demoted to a tiebreaker in red-team because volatility can mean distress and freeze rather than receptivity. Above prescore 0.35 an organization is shortlisted, the top 60 go to the 70B auditor, and audited alignment at or above 0.45 draws a memo, with a fixed fraction of ε-greedy slots from the band below so the weights can be calibrated against reply outcomes. Memo slots are filled by maximal-marginal-relevance so the night's drafts span sectors rather than landing in one. Two priors were rejected on evidence: the long-open-posting boost, because age predicts ghost jobs and interview odds run 2–3× in the first 48 hours; and email automation, because it torches deliverability and converts worse than a few verified memos a day. A Hawkes process on posting-arrival times, intensity λ(t) = μ + nβ∑e−β(t−ti) with β = 1/7 day−1, flags a burst ratio λ(now)/μ ≥ 2 as a hiring machine running hot even when as many roles close as open.
## Structural hallucination control
The auditor sees document excerpts and none of the ranking scores, so it stays an independent second opinion. Every memo claim passes a verification pass that deletes or hedges anything the excerpts do not support, then the cross-model critic's kill mandate, and proof-of-work briefs carry provenance chains of verbatim quotes mechanically checked against sources and marked VERIFIED or UNVERIFIED. The Form D engine caught a $1.1B AI-infrastructure raise two days post-filing; the ghost-role engine recovered synthetic archetypes exactly and its Theil-Sen posting slope to three decimal places. Ranking weights are pre-registered for logistic-regression replacement once roughly 40–50 reply outcomes accrue. No memo has been sent yet: no outreach outcome is claimed. A human reviews and sends everything; no SMTP path exists in the pipeline.
Python, sentence-transformers / Qwen3-Embedding-8B, hmmlearn, NumPy, numba, LangGraph, vLLM (Qwen3-Next-80B FP8, Llama-3.3-70B critic), aiohttp, SQLite. Two-stage SLURM chain on 16 CPU + H100/A100 GPUs, UGA Sapelo2. NSF/NIH/USAspending, arXiv/OpenAlex, SEC EDGAR, USAJOBS, USPTO, DOL LCA — all public.
Liam Kozma · liam@liamkozma.com