Agent skills
Enterprise includes two coding-agent skills. One runs the A–E workflow. The other groups failed queries and proposes the next test.
The two shipped skills
Both skills call the Rust corec command and inspect its output files. corec validates experiment inputs, calculates metrics, selects stage results, and decides whether the final run passes.
| Skill | Use it when | Files it writes |
|---|---|---|
| run-design-loop | Initialize or resume A–E, execute the active work queue, finalize Stage E, or certify the corpus | Design state, experiment results, decision, and certification manifest |
| analyze-eval-misses | A retrieval or generation run has misses, a gate fails, or quality regresses | Reviewable EvaluationMissAnalysis and one controlled next experiment |
Repository-scoped discovery
The licensed checkout exposes each canonical SKILL.md through the project discovery directory used by Codex and Claude Code. Each environment follows the same reviewed guide.
Codex
$run-design-loop Resume the reference corpus under .corec/enterprise/reference-corpus. Run until human review or external projection work is required.
$analyze-eval-misses Analyze .corec/enterprise/reference-corpus/12-stage-e-retrieval and 13-stage-e-generation. Propose one controlled next experiment if the evidence supports it.
Claude Code
/run-design-loop Resume the reference corpus under .corec/enterprise/reference-corpus.
/analyze-eval-misses Analyze the Stage E retrieval and generation outputs.| Skill | Codex project path | Claude Code project path |
|---|---|---|
| run-design-loop | .agents/skills/run-design-loop/SKILL.md | .claude/skills/run-design-loop/SKILL.md |
| analyze-eval-misses | .agents/skills/analyze-eval-misses/SKILL.md | .claude/skills/analyze-eval-misses/SKILL.md |
run-design-loop · resume the active work
The skill implements the A–E method documented in Retrieval experiments. It resumes design-loop.json, runs the tasks listed in activeExperiments, and calls analyze-eval-misses after a failed query.
It pauses when a person must review converted text, provide credentials, explain a source field, set access, or approve a test question.
run-design-loop · output from the Enterprise tutorial
These values come from the six-record reference fixture used by the Enterprise implementation check. The A–D stage evidence is labeled synthetic_demo; Stage E uses the public Rust evaluators and reviewed JSONL fixtures. Customer certification requires representative customer data and reviewed customer questions.
{
"A": {"experimentId": "A-001", "converter": "canonical:title-content-v1", "hitAt10": 0.96, "latencyMs": 8.0},
"B": {"experimentId": "B-014", "chunkingStrategy": "sentence-aware", "chunkTokens": 160, "overlapTokens": 20},
"C": {"experimentId": "C-001", "embeddingModel": "sentence-transformers/all-MiniLM-L6-v2", "dimensions": 384},
"D": {"experimentId": "D-top_k-001", "searchType": "lexical", "topK": 5, "generationFaithfulness": 0.95, "generationQualifiedAnswerRate": 0.90}
}{
"stageE": {
"retrieval": {"hitAt1": 1.0, "hitAt20": 1.0, "hitAt50": 1.0, "misses": 0, "passed": true},
"generation": {"faithfulness": 0.976, "faithfulnessCoverage": 1.0, "qualifiedAnswerRate": 1.0, "misses": 0, "passed": true}
},
"manifest": {"status": "certified", "readyForPipelines": true, "nextActions": []}
}analyze-eval-misses · inspect failures before changing settings
The skill checks report hashes, configuration IDs, corpus versions, and reviewed-question versions. It then compares each failed query with expected passages, rankings, filters, access rules, and generated answers. The report labels observed facts and possible causes separately. A person approves every change to a reviewed question.
| What you see | Likely cause | Next test |
|---|---|---|
| Source content is absent or conversion loses headings, tables, or columns | Source coverage or conversion | Understand/ingestion review or Stage A |
| Expected evidence is split, truncated, or detached from context | Segmentation | Stage B |
| Relevant evidence remains absent through rank 50 after source and chunk checks | Embedding space | Stage C |
| Another policy ranks better, the right passage appears below selected top_k, or routing is wrong | Policy, depth, routing, or filter | Stage D |
| Required evidence reached the model while facts, citations, faithfulness, or abstention failed | Synthesis | Stage E |
| Accepted evidence, access intent, or source meaning is uncertain | Judgment or policy review | Named human review |
analyze-eval-misses · clean result from the Enterprise tutorial
With zero misses, the skill keeps the selected settings and proposes no new test. With misses, it writes one cause code per query and proposes one setting to change, the settings to hold fixed, the metric to watch, and a stop condition.
{
"kind": "EvaluationMissAnalysis",
"status": "complete",
"reviewStatus": "not_required",
"inputs": {
"configurationFingerprint": "4953c1…7f38bd",
"judgmentsVersion": "4032bf…d4c2a"
},
"summary": {
"retrievalMisses": 0,
"generationMisses": 0,
"primaryCauseCounts": {}
},
"cases": [],
"recommendation": null
}Use the skills safely
- Keep customer inputs immutable and write skill-produced analysis beside derived evaluation artifacts.
- Use one corpus version and reviewed-sample version across comparable candidates.
- Require human review for converter meaning, access intent, and golden-judgment changes.
- Preserve failed and rejected evidence so the next iteration remains explainable.
- Treat a certification manifest with status certified and readyForPipelines true as the promotion decision.