How the score is actually computed
EthosGuard separates judgment from arithmetic. A language model judges ten dimensions of a response; deterministic code computes everything else. The same input and the same judgments always produce the same score, the same flags, and the same audit record.
The evaluation pipeline
Ingest
The AI output arrives with its prompt and any relevant context, through the API or the evaluator.
Judge
A language model scores the response on each of the ten dimensions, from -10 to +10, with written reasoning per dimension. That is the only thing the model decides.
Ground
The evaluation retrieves relevant entries from a curated corpus aligned to the EU AI Act and NIST AI RMF, and ties flags to named sources.
Compute
Deterministic code applies domain weights, safety gates, and normalization to produce the overall score, risk level, flags, and recommendation.
What gets judged
What the code enforces
- ✓Domain weighting. Each industry raises the weight of the dimensions that matter most there: Restraint and Consequences in healthcare, Logic and Restraint in finance, Honesty in education.
- ✓Hard safety gates. A severe failure on Restraint, Truth, or Consequences caps the score and forces the risk level up, no matter how well the response scores elsewhere. A high average can never hide a critical failure.
- ✓Deterministic scoring. The overall score (-100 to +100), the risk level (safe to critical), the flags, and the recommendation are computed by code, not phrased by a model.
- ✓Tiered routing. Routine cases run on an efficient judge; high-risk or ambiguous cases escalate to a stronger model, and an optional second judge scores independently for comparison.
- ✓An audit record on every evaluation. Model, prompt version, corpus version, retrieved sources, and timestamp, stored with a stable audit ID.
How we test it
The engine is exercised against a curated library of 30 expert-designed cases across six industries: realistic prompts, realistic unsafe answers, and realistic good ones. In the current library, every dangerous case is caught and flagged, and every sound response scores positive. The library is available inside the live demo, so you can rerun any case yourself.
Calibration is a living process. Every pilot adds reviewed, human-decided cases to the evidence base, and thresholds are tuned per organization rather than frozen globally.
What it does not do
- •The ten dimension judgments come from a language model and carry model judgment error; the deterministic layer bounds but does not eliminate it.
- •The grounding corpus is curated and finite; a flag citation means the evaluation consulted a source, not that a regulator has endorsed the verdict.
- •Scores describe a single output in context; they are not a certification of the AI system that produced it.
- •EthosGuard supports human governance. It does not replace legal review, organizational accountability, or qualified professional judgment, and it does not itself establish legal or regulatory compliance.
Read a full evaluation, not a summary of one
The clearest proof of a methodology is its output. Open a live Ethos Card and see every score, every flag, and its audit trail.
The full methodology paper, including scoring formulas and the framework's source model, is available on request.