Aura-V
Aura-V Tech News & Intelligence
Latest AI & Automation MarTech UAE FinTech Modern Tech Business & Society 🎬 Videos & Tutorials 📰 AI Newsroom Agency Solutions

Fable 5.2 and GPT-6.2 Astra fall short of employee-level performance on real-world tasks

Dr. Kai Wei AI
Lead Frontier AI Correspondent • 2026-11-18

Reported by Dr. Kai Wei, Frontier AI Desk

Epoch’s evaluation shows both models excel on benchmarks but miss the quality bar for actual workplace automation.

📖
Original source

«How good are Fable 5.2 and GPT-6.2 Astra at real work?»

Summary and commentary by Aura-V. Facts, figures and opinions belong to the original.

Read the original ↗

Claude Fable 5.2 and GPT-6.2 Astra both hit near-record marks on the Epoch Capabilities Index (ECI), yet Epoch’s internal automation benchmark reveals a stark contrast between synthetic scores and actual work output. The study, published by Epoch AI Compute & Frontier Forecasting, evaluated six models—including Fable 5.2, GPT-6.2 Astra, Opus 5.5, Gemini 4 Argon and two earlier versions—on tasks drawn directly from Epoch’s day-to-day operations. Each output was graded against an internal 100 % standard that reflects the quality expected from Epoch employees Epoch report.

Contractor-level performance on AI data-center analysis

The most notable result is that Fable 5.2 and Opus 5.5 achieved scores of 97.5 % and 87.5 % respectively on the AI data-center analysis task, a level the authors equate with work produced by external contractors. The task required adding a new data center (EcoDataCenter 1) to Epoch’s database, involving news research, regulatory review, satellite-image ordering, and image analysis. Fable 5.2’s only deficiencies were verbose prose and minor image-annotation errors; Opus 5.5 needed additional revisions to equipment classification and more substantial image fixes.

A contractor typically spends about 16 hours per data center. Opus 5.5 completed the work in just under three hours, while Fable 5.2 required roughly 6.5 hours—more time than a contractor but delivering higher fidelity. By contrast, GPT-6.2 Astra scored only 65 % on the same task, omitting critical information and refusing to make reasonable estimates without explicit source confirmation. The authors note that Astra’s “overly cautious” approach led to incomplete conclusions that they would expect a human contractor to avoid.

Persistent weaknesses in experiment design and originality

Across all six models, the evaluation uncovered a systematic inability to design novel experiments. In a “chess variants” test meant to probe out-of-distribution generalization, every model defaulted to known variants rather than inventing truly new rule sets. Fable 5.2 presented “Kingsgate,” which mirrors the existing “Midline Invasion” variant; Opus 5.5 and Fable 5.1 introduced a “camel” piece that is already part of fairy-chess literature; GPT-6.2 Astra merely swapped piece labels without altering gameplay. Even Gemini 4 Argon, while more confident, misapplied assumptions—such as attributing B200 chips to every building—and ignored self-generated evidence that invalidated its own experiment.

The authors attribute this pattern to models’ tendency to recycle existing ideas from their training data, limiting their capacity to address research questions that require genuine novelty. The report emphasizes that without the ability to generate fresh experimental designs, models cannot reliably support higher-level research or automation pipelines.

Mixed progress on writing, graphics, and self-justification

GPT-6.2 Astra showed measurable gains over its predecessor, GPT-6 Astra, narrowing the gap with Gemini 4 Argon in several dimensions. The newer model produced more concise prose, improved graphic design, and delivered a 25-point uplift on the AI data-center research metric. However, a recurring bias emerged: models tended to favor their own generated outputs over higher-quality human references when justifying decisions, rewarding superficial markers such as the number of experiments run rather than substantive rigor.

Overall performance gap

Despite tying for the top ECI score, no model met the 100 % employee standard on any task. The gap between benchmark excellence and workplace readiness underscores the need for more granular evaluation frameworks that reflect real-world constraints.

Our read: Epoch’s findings temper the hype around record ECI scores, showing that current frontier models still require human oversight for complex, multi-step workflows. Their proficiency on curated benchmarks does not guarantee the depth of reasoning, originality, or risk tolerance needed for autonomous B2B deployment.

⚡
Aura-V Editorial Verdict • Reality Check

The models’ headline scores on the Epoch Capabilities Index do not translate into reliable automation for enterprise workflows. Their limitations in research design and conservative judgment raise doubts about immediate deployment for B2B teams.

Critical Analysis Desk Aura-V Intelligence
Aura-V Engineering Advisory

Interested in implementing this technology inside your company?

Our studio designs custom agent meshes, data pipelines, and outbound GTM systems for forward-looking enterprises.

Explore Studio Solutions →