Enterprise tutorial
Follow the Rust workflow used by the Enterprise implementation check: inspect an export, prepare records, test retrieval and answers, approve the selected settings, and route an agent query.
What this tutorial proves
The tutorial uses a six-record synthetic reference-document export so every step runs quickly and every artifact can be inspected. It verifies the complete workflow contract. A customer smoke iteration should use at least 1,000 representative rows; representative-scale design commonly begins around 10,000–50,000 records.
The fixture contains source identity, category, audience, scope, publication date, title, and evidence text. The business goal is an agent that retrieves the governing evidence, respects reader access, cites evidence, and abstains when the corpus lacks support.
0 — install and run the verified path
The demo exits successfully only after canonical ingestion reports succeeded and ready, the exact selected A–D configuration passes full Stage E retrieval and generation, every certification gate passes, and the query-routing artifact is emitted.
git clone <customer-enterprise-repository> corecortex-enterprise
cd corecortex-enterprise
cp .env.example .env
# Set DATABASE_URL in the ignored .env file.
set -a; source .env; set +a
make check
make demo-reference-corpus{"document_id":"DOC-1001","record_type":"reference_document","title":"Define the corpus purpose","content":"Record intended users, representative questions, approved evidence, excluded material, and costly failure modes before changing source data.","category":"design","audience":"analyst","scope":"global","effective_date":"2026-01-01","version":"v3"}0b — substitute a Parquet export
The verified fixture uses JSONL so six rows are easy to inspect. To exercise the Parquet path, replace the input filename in the first two commands. The Enterprise CLI reference owns the complete format matrix.
corec corpus classify \
--input customer-export/documents.parquet \
--output .corec/enterprise/reference-corpus/00-classification
corec corpus explore \
--input customer-export/documents.parquet \
--output .corec/enterprise/reference-corpus/02-profile.json1 — classify the export
Classification identifies candidate dataset boundaries and writes source hashes. Read classification.json before proceeding. In a mixed export, this is where change-records, comments, attachments, users, or other coherent datasets become explicit candidates.
corec corpus classify \
--input examples/reference-corpus/documents.jsonl \
--output .corec/enterprise/reference-corpus/00-classification{
"kind": "CorpusClassification",
"source": {"format": "jsonl", "rows": 6, "sha256": "63b951…39ab7"},
"datasets": [{
"name": "reference-document",
"classification": "deterministic_heuristic",
"shape": "structured",
"shapeConfidence": 1.0,
"rows": 6,
"nextStep": "sample_then_explore"
}],
"structureAdvisory": {"datasetCount": 1, "heterogeneous": false, "sampleRequired": true}
}2 — review samples and explore fields
A source owner confirms that one row represents one versioned reference document and that document_id remains stable across refreshes. Store that approval as a small sample-review artifact. The profile then supplies field types, nulls, cardinality hashes, and text-length distributions while withholding source examples by default.
- Review the longest, shortest, null-heavy, and common-shape records.
- Confirm the business meaning of identity, version, effective dates, and access fields.
- Use --include-examples only in an approved local review context.
corec corpus explore \
--input examples/reference-corpus/documents.jsonl \
--output .corec/enterprise/reference-corpus/02-profile.json{
"status": "approved",
"reviewedBy": "demo:corpus-owner",
"datasets": ["reference-documents"],
"notes": [
"One row represents one versioned reference document.",
"document_id is stable across export refreshes."
]
}{
"kind": "FieldProfile",
"privacy": {"distinctValuesHashed": true, "examplesIncluded": false},
"fields": [
{"path": "document_id", "observed": 6, "distinct": 6, "nullRate": 0.0, "types": {"string": 6}},
{"path": "content", "observed": 6, "nullRate": 0.0, "textLength": {"min": 202, "mean": 223.8, "p95": 258}},
{"path": "scope", "distinct": 2, "nullRate": 0.0, "types": {"string": 6}}
]
}3 — advise, review, and register the schema
Advice accelerates mapping work and remains evidence for review. Registration is the approval boundary: it appends a fingerprinted revision to the registry and records who approved it.
corec schema advise \
--input examples/reference-corpus/documents.jsonl \
--name reference-documents \
--output .corec/enterprise/reference-corpus/03-advice.json
# A source owner reviews title, content, identity, attributes, and access.
corec schema register \
--suggestion .corec/enterprise/reference-corpus/03-advice.json \
--registry .corec/enterprise/reference-corpus/04-schema-registry.jsonl \
--approved-by demo:corpus-owner \
--output .corec/enterprise/reference-corpus/04-registered-schema.json{
"kind": "PipelineAdvice",
"schema": {
"status": "needs_review",
"confidence": 1.0,
"mapping": {
"record_id": "document_id",
"record_type": "record_type",
"title": "title",
"content": "content",
"published_at": "effective_date"
},
"attributeFields": ["audience", "category", "scope", "source_url", "version"]
},
"index": {"lexicalFields": ["title", "content"], "vectorFields": ["content"], "filterFields": ["audience", "category", "scope", "source_url", "version"]}
}{
"kind": "RegisteredSchema",
"dataset": "reference-documents",
"revision": 1,
"approvedBy": "demo:corpus-owner",
"fingerprint": "1d5df967…6d7e7",
"mapping": {"record_id": "document_id", "title": "title", "content": "content"}
}4 — prepare canonical records and an ingestion batch
Preparation maps each row into stable canonical identity, content, typed attributes, explicit access, source provenance, and transformation provenance. The batch command validates duplicates and hashes, then emits backend-neutral request parts of at most 1,000 records.
corec dataset prepare \
--input examples/reference-corpus/documents.jsonl \
--schema .corec/enterprise/reference-corpus/04-registered-schema.json \
--name reference-documents \
--provider "reference-export" \
--observed-at 2026-08-19T00:00:00Z \
--allow demo:reader \
--output .corec/enterprise/reference-corpus/05-dataset
corec dataset chunk \
--input .corec/enterprise/reference-corpus/05-dataset/records.jsonl \
--output .corec/enterprise/reference-corpus/07-source-fragments.jsonl \
--strategy corec.chunk.sentence-aware \
--configuration examples/reference-corpus/chunk-settings.json
corec dataset prepare-batch \
--input .corec/enterprise/reference-corpus/05-dataset/records.jsonl \
--name reference-documents \
--configuration-revision 1 \
--mapping-fingerprint <registered-fingerprint> \
--output .corec/enterprise/reference-corpus/06-ingestion
corec validate-batch .corec/enterprise/reference-corpus/06-ingestion/request.json
corec dataset ingest-demo \
--database-url "$DATABASE_URL" \
--config examples/reference-corpus/corec-demo.yaml \
--batch .corec/enterprise/reference-corpus/06-ingestion/request.json \
--design-configuration examples/reference-corpus/selected-configuration.json \
--output .corec/enterprise/reference-corpus/06-ingestion/applied-receipt.json \
--actor demo:corpus-owner{"record_id":"reference-documents:DOC-1001","record_type":"reference_document","title":"Define the corpus purpose","content":"Record intended users and approved evidence…","attributes":{"audience":"analyst","category":"design","scope":"global","version":"v3"},"access":{"allow":["demo:reader"],"deny":[],"attributes":{}},"source":{"provider":"CoreCortex reference export","source_id":"DOC-1001","published_at":"2026-01-01","observed_at":"2026-08-19T00:00:00Z"},"provenance":{"processor":"corec.enterprise.dataset.prepare","processor_version":"1.0.0","source_artifact_sha256":"63b951…39ab7"}}{
"dataset": "reference-documents",
"configurationFingerprint": "3281ec…5dfc6",
"designConfigurationFingerprint": "4953c1…7f38bd",
"designConfiguration": {"converter":"canonical:title-content-v1","chunkingStrategy":"sentence-aware","chunkTokens":160,"overlapTokens":20,"embeddingModel":{"id":"sentence-transformers/all-MiniLM-L6-v2","dimensions":384},"searchType":"lexical","topK":5},
"fragmentSnapshotSha256": "adf2a8…b33d",
"recordsSeen": 6,
"fragmentsCreated": 6,
"state": "succeeded",
"projectionState": "ready",
"projections": [{"adapter": "io.corec.memory.vector", "index": "reference-documents-demo", "projectionIdentity": "reference_documents_demo__3281ecf48acbd143", "embeddingProvenance": {"provider": "local", "model": "sentence-transformers/all-MiniLM-L6-v2", "revision": "token-hash-v1", "dimensions": 384}, "documentsUpserted": 6, "state": "ready"}]
}5 — generate review candidates and preserve reviewed golden data
Golden generation consumes the exact materialized fragments selected for ingestion. Candidate judgments therefore cite the same stable fragment IDs used by projections and answers. Generated candidates carry unreviewed status. Reviewers promote suitable cases into a separate judgment JSONL and add hard negatives, expected-empty questions, unavailable-source cases, calibration cases, and authorization checks.
corec golden generate \
--fragments .corec/enterprise/reference-corpus/07-source-fragments.jsonl \
--output .corec/enterprise/reference-corpus/07-golden{"fragment_id":"a5a06b04-ec47-51bc-a14b-0fe714d27bb1","record_id":"reference-documents:DOC-1001","ordinal":0,"title":"Define the corpus purpose","content":"Record intended users and approved evidence…","token_count":33,"section":"document","content_sha256":"6760fa…5314","processor":"corec.chunk.sentence-aware","processor_version":"1.0.0","processor_configuration_sha256":"19099a…f0fa"}{"query_id":"reference-q001","query":"How does the corpus preserve source identity?","relevant_fragment_ids":["a5a06b04-ec47-51bc-a14b-0fe714d27bb1"],"expected_empty":false,"review_status":"reviewed"}6 — establish the retrieval and generation controls
Invoke $analyze-eval-misses with the retrieval and generation output directories after evaluation. It verifies configuration and evidence identity, inspects every miss, separates observations from inferred causes, and proposes the earliest affected design stage. This verified fixture reaches Hit@1 1.00, citation precision and recall 1.00, and mean faithfulness 0.976, so the skill correctly proposes no parameter change.
Treat these numbers as wiring verification; production quality requires representative data and reviewed customer questions.
corec eval retrieval \
--fragments .corec/enterprise/reference-corpus/07-golden/fragments.jsonl \
--judgments examples/reference-corpus/retrieval-reviewed.jsonl \
--configuration examples/reference-corpus/selected-configuration.json \
--limit 50 \
--hit-at-1-target 0.55 \
--output .corec/enterprise/reference-corpus/08-retrieval
corec eval generation \
--input examples/reference-corpus/generation-reviewed.jsonl \
--retrieval-report .corec/enterprise/reference-corpus/08-retrieval/report.json \
--configuration examples/reference-corpus/selected-configuration.json \
--chunk-size-tokens-avg 160 \
--model demo/offline-answer-fixture-v1 \
--judge-model demo/deterministic-citation-judge-v1 \
--judge-approved-by demo:corpus-owner \
--checkpoint examples/reference-corpus/generation-checkpoint.json \
--server-health examples/reference-corpus/server-health.json \
--gate-policy answer-quality \
--faithfulness-target 0.80 \
--qualified-answer-target 0.80 \
--output .corec/enterprise/reference-corpus/09-generation{
"kind": "RetrievalEvaluation",
"queries": 11,
"inputs": {"judgmentsVersion": "4032bf…d4c2a", "judgmentsPath": "…/retrieval-reviewed.jsonl"},
"retrievalMode": "bm25",
"retrievalDepth": 50,
"selectedTopK": 5,
"relevance": {"hitAt1": 1.0, "hitAtK": {"5": 1.0, "10": 1.0, "20": 1.0, "50": 1.0}, "mrr": 1.0, "plateau": {"topK": 5}},
"expectedEmpty": {"queries": 1, "accuracy": 1.0},
"misses": {"count": 0},
"gate": {"metric": "hitAt1", "actual": 1.0, "target": 0.55, "passed": true}
}{
"kind": "GenerationEvaluation",
"inputVersion": "c62924…66d3e",
"inputPath": "…/generation-reviewed.jsonl",
"answers": 10,
"citationPrecision": 1.0,
"citationRecall": 1.0,
"faithfulness": 0.976,
"faithfulnessCoverage": 1.0,
"qualifiedAnswerRate": 1.0,
"misses": {"count": 0},
"gate": {"policy": "answer_quality", "passed": true}
}{
"kind": "EvaluationMissAnalysis",
"status": "complete",
"reviewStatus": "not_required",
"summary": {
"retrievalMisses": 0,
"generationMisses": 0,
"primaryCauseCounts": {}
},
"cases": [],
"recommendation": null,
"conclusion": "Preserve the selected configuration and make no parameter change from this run."
}7 — run the A–E design controller
Invoke $run-design-loop with the engagement root to produce this selection and resume later runs from the saved state.
The authored search space names files. Design initialization records each resolved path and computed SHA-256 value in design-loop.json. Evaluators compute their own input identities, and finalization reports both files and both hashes when they differ.
For this fixture, run-design-loop selects title-plus-content conversion, sentence-aware 160-token passages, all-MiniLM-L6-v2 embeddings, lexical search, and top_k 5. The selected object above is the result to inspect before Stage E.
The Retrieval experiments page owns the A–E method. The Agent skills page explains how run-design-loop resumes work and when it pauses for review.
corec design init \
--search-space examples/reference-corpus/search-space.json \
--output .corec/enterprise/reference-corpus/10-design-loop
# For each planned experiment: build its isolated projection, evaluate it,
# audit it, then record the result.
corec design record-result \
--state .corec/enterprise/reference-corpus/10-design-loop/design-loop.json \
--experiment-id <planned-experiment-id> \
--retrieval-report <experiment>/retrieval-report.json \
--ingestion-evidence <experiment>/ingestion-receipt.json \
--results <stage>/results.jsonl
corec design advance \
--state .corec/enterprise/reference-corpus/10-design-loop/design-loop.json \
--results <stage>/results.jsonl \
--output .corec/enterprise/reference-corpus/10-design-loop{
"apiVersion": "corecortex.io/enterprise/v1alpha1",
"kind": "CorpusDesignSearchSpace",
"dataset": "reference-documents",
"collections": ["reference-documents"],
"artifacts": {
"corpus": "documents.jsonl",
"retrievalJudgments": "retrieval-reviewed.jsonl",
"generationJudgments": "generation-reviewed.jsonl",
"miniRetrievalJudgments": "retrieval-reviewed.jsonl",
"miniGenerationJudgments": "generation-reviewed.jsonl"
},
"miniQueries": 11,
"miniGenerationQueries": 10,
"retrievalSweepTopK": 100,
"parameters": {
"converters": ["canonical:title-content-v1", "canonical:content-v1"],
"chunkingStrategies": ["token-window", "sentence-aware"],
"chunkTokens": [80, 160, 240],
"overlapTokens": [0, 20, 40],
"embeddingModels": [
{"id": "sentence-transformers/all-MiniLM-L6-v2", "dimensions": 384, "costPerMillionTokens": 0.0},
{"id": "BAAI/bge-base-en-v1.5", "dimensions": 768, "costPerMillionTokens": 0.0}
],
"searchTypes": ["lexical", "semantic", "hybrid"],
"topK": [5, 10, 20, 50]
},
"anchors": {
"converter": "canonical:title-content-v1",
"chunkingStrategy": "sentence-aware",
"chunkTokens": 160,
"overlapTokens": 20,
"embeddingModel": "sentence-transformers/all-MiniLM-L6-v2",
"searchType": "lexical",
"topK": 10
},
"anchorRationale": "Controls for the first comparison; these are candidates rather than defaults.",
"selection": {"metricTieDelta": 0.01, "plateauDelta": 0.03}
}{
"converter": "canonical:title-content-v1",
"chunkingStrategy": "sentence-aware",
"chunkTokens": 160,
"overlapTokens": 20,
"embeddingModel": {"id": "sentence-transformers/all-MiniLM-L6-v2", "dimensions": 384},
"searchType": "lexical",
"topK": 5
}{
"currentStage": "E",
"status": "ready_for_full_evaluation",
"selected": {
"A": {"experimentId": "A-001", "converter": "canonical:title-content-v1", "hitAt10": 0.96},
"B": {"experimentId": "B-014", "chunkingStrategy": "sentence-aware", "chunkTokens": 160, "overlapTokens": 20},
"C": {"experimentId": "C-001", "embeddingModel": "sentence-transformers/all-MiniLM-L6-v2", "dimensions": 384},
"D": {"experimentId": "D-top_k-001", "searchType": "lexical", "topK": 5, "generationFaithfulness": 0.95, "generationQualifiedAnswerRate": 0.90}
},
"nextAction": "Run full Stage E retrieval and generation with the exact optimum."
}8 — finalize Stage E and certify
The run-design-loop skill performs this finalization and manifest invocation after the complete Stage E evidence is present. Certification binds the exact design configuration, applied projection receipt, receipt-bound retrieval report, retrieval-bound generation report, and reviewed generation-sample version. A missing or failed requirement produces a blocked manifest with specific next actions. Multi-collection corpora also provide a corpus evaluation report covering routing or an explicitly labeled unrouted baseline.
corec design finalize \
--state .corec/enterprise/reference-corpus/10-design-loop/design-loop.json \
--ingestion-evidence .corec/enterprise/reference-corpus/06-ingestion/applied-receipt.json \
--retrieval-report .corec/enterprise/reference-corpus/12-stage-e-retrieval/report.json \
--generation-report .corec/enterprise/reference-corpus/13-stage-e-generation/generation-report.json \
--output .corec/enterprise/reference-corpus/14-design-decision.json
corec manifest emit \
--classification .corec/enterprise/reference-corpus/00-classification/classification.json \
--sample-review examples/reference-corpus/sample-review.json \
--profile .corec/enterprise/reference-corpus/02-profile.json \
--advice .corec/enterprise/reference-corpus/03-advice.json \
--schema-registry .corec/enterprise/reference-corpus/04-schema-registry.jsonl \
--dataset .corec/enterprise/reference-corpus/05-dataset/dataset.json \
--ingestion .corec/enterprise/reference-corpus/06-ingestion/applied-receipt.json \
--judgments examples/reference-corpus/retrieval-reviewed.jsonl \
--design-decision .corec/enterprise/reference-corpus/14-design-decision.json \
--retrieval-report .corec/enterprise/reference-corpus/12-stage-e-retrieval/report.json \
--generation-report .corec/enterprise/reference-corpus/13-stage-e-generation/generation-report.json \
--certified-by demo:corpus-owner \
--retrieval-target 0.55 \
--output .corec/enterprise/reference-corpus/15-certified-manifest.json{
"kind": "CorpusCertificationManifest",
"status": "certified",
"readyForPipelines": true,
"certifiedBy": "demo:corpus-owner",
"gates": [
{"name": "understand_phase", "passed": true},
{"name": "registered_schema", "passed": true, "actual": 1},
{"name": "full_ingestion", "passed": true, "actual": {"state": "succeeded", "projectionState": "ready"}},
{"name": "corpus_design_optimum", "passed": true},
{"name": "full_selected_retrieval", "passed": true, "actual": {"hitAt1": 1.0, "hitAt20": 1.0, "hitAt50": 1.0}},
{"name": "generation_faithfulness", "passed": true, "actual": {"faithfulness": 0.976, "coverage": 1.0}}
],
"nextActions": []
}9 — route an agent request
The routing decision supplies scope, intent, retrieval strategy, evidence plan, filters, limit, confidence, and review requirements. The application adds authenticated tenant and user context before calling retrieval.
corec query route \
"How does the corpus preserve source identity?" \
--dataset reference-documents \
--output .corec/enterprise/reference-corpus/16-query-route.json{
"kind": "QueryRoutingDecision",
"request": {
"query": "How does the corpus preserve source identity?",
"intent": "semantic",
"retrievalPlan": "hybrid",
"scope": "dataset",
"evidencePlan": "single_passage",
"filters": {"dataset": "reference-documents"},
"limit": 5
},
"routing": {"signals": ["dataset_filter", "default_semantic"], "confidence": 0.6, "requiresAgentReview": false}
}Continue with the reference pages
Use the reference pages when you adapt this verified run to another corpus.