Handshake AI’s $1 B Frontier Lab Data Business Revealed

Handshake AI’s first Latent Space deep dive maps a $1 billion revenue stream to three core benchmark domains, linking the company’s data offerings directly to frontier-lab needs Latent Space. The episode structures the discussion around finance, biology and coding tasks, each designed to expose where current models still fall short of expert performance.

Finance benchmark reveals hidden error modes

In the ATLAS-Finance demo, an agent parses a bank email, extracts deal-governing files and generates a spreadsheet. The model correctly carries a goodwill figure into Python and can explain the tax treatment when queried, but it misidentifies the code line where the rule applies, exposing a gap in contextual code understanding.

VIALS benchmark tests scientific image interpretation

Nick Magazine and Jonas Mueller present the VIALS suite, asking the model to rank brain-tumor slides by vessel necrosis.

Coding task surfaces tool-use transparency

A simulated file-access scenario lets observers track which files an agent reads, whether it checks copies, and its subsequent shell-history actions. The transcript alone would miss most of the evaluation criteria discussed by Steve Kelling and Nishant Ranka, emphasizing the need for fine-grained observability.

Beyond the demos, the conversation touches on how labs purchase data, the limits of rubric-based evaluation, and safety feedback loops that feed back into model training. The segment on CAREBench demonstrates that refusal mechanisms still lag behind expert judgment, reinforcing alignment as a bottleneck for practical deployment.

The deep dive concludes with Handshake’s leadership outlining the broader market for compute and expert data, and why enterprises are increasingly building private benchmarks to retain control over evaluation standards.