SEMANTIC ATLAS
HYPOTHESES BEFORE HEADLINES
Measure the transformation, not the mythology.
Alefba treats efficiency, preservation, uptake, and outcomes as separate research questions. A result becomes a public claim only when the baseline, preprocessing cost, artifacts, and limits travel with it.
A compiler should be evaluated against the raw or established baseline it replaces, with every transformation cost included.
05.1
The paired protocol
Each candidate pack is evaluated beside a declared baseline under the same task, model, sampling, tools, and scoring conditions.
Preprocessing, extraction, retrieval, verification, retries, and review are part of total cost. Excluding them converts an engineering result into a marketing artifact.
Reproducible fixtures and receipts allow a reviewer to trace an observed difference back to the exact context transformation.
- 01BASELINE
- 02APIR PATH
- 03SAME HARNESS
- 04PAIRED RESULTS
- 05ARTIFACTS
05.2
AlefBench measurement families
The benchmark program separates structural coverage from model behavior and task outcomes.
Obligation coverage measures whether required commitments are represented. Semantic recoverability tests what independent evaluators can reconstruct. Model-switch studies test portability across target profiles.
Latency and token metrics are reported with compiler overhead. Safety and policy tests preserve fail-closed cases as failures rather than averaging them away.
- Commitment and obligation coverage
- Omission precision and critical-loss rate
- Semantic recoverability
- Target uptake under controlled probes
- Task outcome and policy compliance
- End-to-end cost and latency
05.3
Adversarial and privacy research
Semantic infrastructure creates new attack surfaces at extraction, provenance, budgeting, rendering, and receipt interpretation.
Research must include source poisoning, commitment smuggling, provenance substitution, cross-tenant leakage, and malicious codec behavior.
Telemetry remains metadata-first. Content-bearing studies require explicit governance because benchmark convenience does not override data ownership.
- 01THREAT MODEL
- 02FIXTURE
- 03ATTACK
- 04FAIL-CLOSED CHECK
- 05DISCLOSURE
05.4
Questions that stay open
The difficult work is not hidden behind a single semantic score.
How should conflicting cultural and legal commitments be represented without flattening them? When can learned extraction outperform deterministic baselines without weakening auditability? Which probes meaningfully test target uptake?
These are research programs, not solved features. Alefba publishes their boundaries so collaborators can challenge the architecture before claims harden around it.