GISM-0 Benchmark Results: Production Evidence of Our Retrieval Advantage

We have completed an initial benchmark of GISM-0, CleverThis's graph-native memory system, against Claude Code CLI on factual workspace retrieval. Result: 5.8× lower token consumption, 18% lower latency, ~83% cost reduction — with full accuracy on all four questions. The whitepaper and deck are linked below.

This update accompanies the public article published today at cleverthis.com. We are sending it separately to investors and the board with additional context on what this result means for the business.

What was measured

Four factual questions against a live internal workspace. Both systems used Claude Sonnet 4.6 — the intelligence layer is identical between them. GISM-0 and Claude Code CLI differed only in how they retrieved information to construct their answers. Token consumption was measured in aggregate across all four questions.

5.8×Token EfficiencyGISM-0 vs. Claude Code CLI ~83%Lower Inference CostAggregate 4 / 4AccuracyAll questions answered correctly

This is a directional use-case test, not a comprehensive evaluation. We are disclosing scope limits explicitly: four questions, one run, internal workspace, self-evaluated. One of the four questions (Q1) performed slower under GISM-0 than under Claude Code CLI, but token usage under GISM-0 was still reduced. The aggregate still shows 18% faster responses overall. The full analysis and raw token counts are in the whitepaper.

Business implications

GISM-0 is not a prototype. It is the retrieval layer running in production under CleverThis's platform tools today. The 95.5% reduction in our own AI infrastructure cost over a single quarter — documented publicly in the Agent Amnesia Tax article — is driven in material part by GISM-0's retrieval efficiency.

The structural reason: token cost scales directly with context window size. File-reading retrieval systems pay the full cost of every file loaded into context on every query. Graph-traversal retrieval pays only for the facts the specific question requires. The question determines the prompt size. At enterprise scale, this structural difference compounds into a durable unit economics advantage — the kind that shows up as a significant line-item difference in infrastructure budgets and positions CleverThis to deliver materially lower per-query costs than file-based alternatives.

Production System, Not a Lab Result

GISM-0 is the live memory and retrieval layer of the CleverThis platform. This benchmark measures real production performance. We are publishing the full methodology, the Q1 reversal, and every caveat because transparency on the directional evidence is the thing that makes the result credible to a technical audience. We are not there yet on a comprehensive, independently-evaluated benchmark — but we are publishing what we have now, clearly labeled.

Why we are publishing now

A four-question benchmark does not prove a general efficiency claim. What it does: confirm that the production architecture is performing in line with its design principles. The whitepaper documents the structural reason the gap exists — graph traversal versus file context loading — and why the advantage should scale with workspace size. We will continue building toward a more rigorous external evaluation. In the meantime, this is the honest interim record.

Supporting materials

Two documents accompany this update:

  • GISM-0 Benchmark Whitepaper — Full methodology, raw token counts per system, architecture overview, and a structural analysis of the efficiency drivers. The primary technical record for due diligence.
  • GISM-0 Benchmark Deck — Slide-format summary of the results and business implications. Suitable for sharing with advisors or for board-level discussion.

The public article is live at cleverthis.com and covers the same material with additional context on methodology, the Q1 result, and what the benchmark is and is not claiming. We welcome questions.

Build loudly,

Jeffrey Phillips Freeman

Founder/CEO, CleverThis