RC
Compare classifier CLS-204 (current production) and CLS-211 (candidate retrain) on the grants-2024-2026 subset. Show precision and recall by ITAR-tier.
Composition example. Real-shape tti-ai-studio session — composer, citations, right-rail corpus + usage + recent sessions, ⌘K palette, plus the AI-elements pieces (context meter, branch nav, inline citations, artifact, follow-up chips, reaction bar) and the Nuxt UI 4 Chat suite (reasoning trace, tool call, shimmer). Refreshed 2026-05-22: app switcher in header, focus-mode overlay on the artifact, entrance animations on the rail.
session · 04ab
Compare CLS-204 vs CLS-211 on grant subset
Started 8 minutes ago by R. Chen. Scoped to
grants-2024-2026. Model: opus-4-7. ITAR-tagged corpus segment
Conversation
I'll need to load grants-2024-2026, filter to ITAR tier 1+, and run both CLS-204 (production) and CLS-211 (candidate) over the same subset. Precision and recall by tier give us the apples-to-apples view — public-tier numbers also matter for regression analysis since the retrain shouldn't sacrifice public-tier accuracy for ITAR gains.
SELECT precision, recall FROM classifier_metrics
WHERE classifier IN ('CLS-204','CLS-211')
AND corpus = 'grants-2024-2026'
AND tier IN ('itar','public')
GROUP BY classifier, tier;
-- 4 rows returned in 312msAcross 12,480 documents[1] in the grants-2024-2026 subset, here's the comparison broken out by ITAR tier[2]:
- CLS-204 precision · ITAR
- 0.872
- CLS-211 precision · ITAR
- 0.918 (+5.3%)
- CLS-204 recall · ITAR
- 0.794
- CLS-211 recall · ITAR
- 0.821 (+3.4%)
- CLS-204 precision · public
- 0.962
- CLS-211 precision · public
- 0.954 (-0.8%)
Citations from the 4 most-discriminating documents:
- [1]
CLS retrain methodology · v3.1.pdf
/grants-2024-2026/retrain/methodology.pdf · cosine 0.91
- [2]
ITAR rubric · §3.2 boundary cases
/policy-briefs-2025/itar-rubric.docx · cosine 0.88
- [3]
DoD-XR contract addendum
/grants-2024-2026/dod-xr/addendum.pdf · cosine 0.84
compare.py
# Reproduces the CLS-204 vs CLS-211 ITAR comparison shown above.
from tti_ai_studio import corpora, models
scope = corpora.load("grants-2024-2026")
itar = scope.filter(tier="itar")
public = scope.filter(tier="public")
for label, subset in [("itar", itar), ("public", public)]:
for cls in ("CLS-204", "CLS-211"):
m = models.classifier(cls)
p, r = m.evaluate(subset)
print(f"{cls} {label}: precision={p:.3f} recall={r:.3f}")
Follow up with
Drafting follow-up response…
grants-2024-2026·opus-4-7