Hospitals and governance committees need a way to check what a clinical-AI tool is doing without putting it near a patient. RocSite compares a model’s output to a locked, published clinical instrument, or a locked intended-use claim, on synthetic, public, or de-identified data, and signs the result. Every evaluation is auditable. This is not clinical decision support, it is not used on real patients, and it is not a third reader in anyone’s worklist.
Three tools, one rule: evaluate claims against a locked reference, sign the result, stay off the patient.
Most institutions do not need another AI tool first, they need a baseline. If your committee is still working out what AI governance should even look like, this is the engagement to start with.
An inventory of what is already deployed: does each tool’s cleared intended use match the problem it was bought to solve?
Which outputs an expert can verify at a glance, and which need a locked, published instrument watching them for drift.
Off-patient evaluations your committee can hand to leadership, an auditor, or a regulator, each one Ed25519-signed and independently checkable.
AI Governor is for CMIOs, AI governance committees, and clinical informatics teams, especially teams without a deep imaging-AI bench. It takes a clinical-AI output and evaluates it against a published, deterministic clinical instrument or a locked intended-use claim, on synthetic, public-dataset, or de-identified inputs. Use it as a bench evaluation before procurement, or as a post-deploy audit. Each evaluation is cryptographically signed with Ed25519 and independently verifiable.
Not a clinical-decision-support device. Not used on real patients. Never inserted into the diagnostic worklist. Inputs are synthetic, public-dataset, or de-identified. The output is an audit record for your governance file, a signed receipt, not a clinical report.
Three of 21 locked instruments, see all instruments below ↓
Where an expert can verify a model’s output at a glance, a radiology read, Governor stays out of the way. Where no one can verify by looking, sepsis risk, deterioration scores, AKI prediction, a locked, published instrument is the only independent reference, and that is where drift hides. That gap is what Governor covers.
SYNTH-AKI-0042
[email protected]
grounded · partial · ungrounded · no clinical claim
ed25519:9f3ab2d1c04e…77c41e, illustrative only; this sample signature is synthetic and will not verify
rocsitediscovery.com/verify/<record-token>, checkable against the public key at /keys/ with no login. This demonstration record is not in the verification registry.
This is what lands in your governance file when you evaluate a deployed model, a signed, independent, off-patient record you can hand to a committee or a regulator.
What decision does it change, and for whom? It gives a procurement or QA committee an independent second opinion on a model’s behavior before or after deployment, without ever entering the diagnostic workflow.
KDIGO 2012
Evaluates AI AKI predictions against KDIGO staging (creatinine ratios + urine output)
NEWS2
Evaluates AI early-warning predictions against NEWS2 (Royal College of Physicians)
APACHE II
Published estimates underestimate elderly ICU mortality by 66–168% across 3 conditions (n=201,905)
SOFA + Sepsis-3 criteria
286,510 ICU cases · pre-registered falsification · published null result
Padua Prediction Score
Evaluates AI VTE predictions against the 11-item Padua Prediction Score
MORSE Fall Scale
Evaluates AI fall-risk predictions against the MORSE Fall Scale (6 weighted items)
Braden Scale
Evaluates AI pressure-ulcer predictions against the Braden Scale (6 subscales)
LACE Index
Evaluates AI 30-day readmission predictions against the LACE Index
BRASS Index
Evaluates AI discharge-planning predictions against the BRASS Index (10 items)
NEWS2-derived
Evaluates AI code-blue triggers against NEWS2-derived deterioration thresholds + rate-of-change
CHA₂DS₂-VASc
Evaluates AI AFib stroke-risk predictions against the 9-point CHA₂DS₂-VASc score
NIHSS
Evaluates AI stroke-severity predictions against the 15-item NIH Stroke Scale
ESI v4
Evaluates AI ED-triage predictions against the 5-level Emergency Severity Index (ESI v4)
Cockcroft-Gault
Cross-checks AI renal-dosing recommendations against Cockcroft-Gault clearance
CLSI / CAP criticality thresholds
Evaluates AI critical-value flags against CLSI / CAP-published criticality thresholds
Charlson Comorbidity Index
Evaluates AI risk-stratification against the Charlson Comorbidity Index (validated 40+ years)
USPSTF Grade A/B
Evaluates AI preventive-care gap predictions against USPSTF Grade A/B recommendations
C-SSRS
Evaluates AI psychiatric-risk predictions against C-SSRS lethality scoring
PHQ-9
Evaluates AI depression-screen predictions against PHQ-9 nine-item scoring
Independent multi-pathology imaging baseline
Bench disagreement review for procurement and QA, not a PACS-integrated reader
Locked RocSite ICH Model V1
Offline evaluation against the locked, SHA-256-verified RocSite ICH reference. Inputs de-identified or public only
Demo: on a de-identified disagreement case, the model under evaluation reads “normal”, the human
read is right lower lobe consolidation, early pneumonia, the bench evaluation records what each source said,
which named criteria sets the reasoning referenced, and whether the reasoning was grounded
(grounded · partial · ungrounded · no_clinical_claim),
with named verdict-provenance (primary · text_override · scope_gap_kept).
The result goes to a governance file, not to the next patient. ~45 s end-to-end.
A deliberately engineered subtle epidural hemorrhage where the locked RocSite ICH model under-calls (probability 4.09%, below the 11.52% threshold) while another model catches it. The audit trail records the discordance even when our own specialist is the one that is wrong, same Ed25519 signature, same evidence hash, same provenance chain. Governance includes governance of ourselves.
Engineered case SYNTH_SUBTLE_EDH_MISS. Synthetic input; not a production modality; not counted in the modality total.
Adam will show you the methodology, the verification flow, and what a bench evaluation looks like at your institution. Sometimes the signed answer is that a tool does not fit your institution, or should be pulled back, that record protects the committee too. Engagements are scoped to the institution, contact us.
Request a walkthrough Start 30-day trialThe Discovery Engine is the research arm. It surfaces contradictions and patterns across published clinical mortality data, with pre-registered methodology and a public evidence chain. Two engagement paths: a licensed engine in your environment with monthly contradiction reports, or single-paper integrity scoring.
Cross-dataset replication for mortality, sepsis, and risk-stratification claims. Pre-registered methodology, OSF evidence chain.
Ed25519-signed evidence hashes for every finding. Public methodology. Confirmatory replication is pre-registered before clinical claims are made.
Dedicated engine instance, your data stays in your environment, monthly contradiction reports. Or single-paper Evidence Integrity Score on request.
Whether it is a license, a single-paper integrity scan, or a partnership conversation, Adam responds personally within 24 hours. Engagements are scoped to the institution.
Start 30-day Trial Contact AdamCaliper takes a drug-condition-population hypothesis and runs it through a 6-tier validation framework on credentialed ICU cohorts. Propensity-score matching with 40 covariates. Cross-dataset replication on MIMIC-IV 3.1 and eICU-CRD 2.0. COVID-era stratification. Every finding is signed with Ed25519 and externally verifiable via the public /keys/ page. Findings that don't survive PSM are reclassified to CONFOUNDED_INVERTED and surfaced in the corpus, not hidden. The framework catches its own errors.
Engine runs cohort cuts, PSM, era stratification, and signs the result.
ROBUST_PUBLISHABLE → ROBUST → CANDIDATE → HYPOTHESIS_GENERATING → CONFOUNDED_INVERTED → PANDEMIC_ARTIFACT.
Ed25519 signature, public verify URL, no authentication required to validate.
Encrypted at rest, scoped to your session, never crosses to the credentialed cohort path.
40 covariates, 1:1 nearest-neighbor matching, caliper 0.2·SD, era stratification all surfaced per finding.
MIMIC-IV 3.1 (94,458 ICU stays per PhysioNet release; Caliper filters to a credentialed adult subset) and eICU-CRD 2.0 (200,859 unit stays across 208 hospitals). Patient-level data never leaves the credentialed path.
For real-world evidence teams, pharmacoepidemiology groups, signal-detection scientists, clinical-pharmacology directors, and biotech R&D teams. Every result cryptographically signed. Trial limits are set during credential review; engagements are scoped to the institution.
Start 30-day Trial Contact AdamEvery paper here is approved by Adam personally. No auto-indexing. Each preprint is shared for pre-submission peer commentary and carries the full evidence chain, datasets, code, OSF preregistration, signed hashes.
Across 286,510 ICU encounters in MIMIC-IV and eICU-CRD, reported benchmark accuracy depends on which sepsis definition generates the labels. The pooled clinical claim did not survive the pre-registered falsification protocol, and the null was published rather than shelved.
Read the full paper →Sepsis-AI tools trained on academic-medical-center data systematically misfire in community hospitals. The miscalibration is structural, not statistical, and pre-registered replication confirms it.
Read the full paper →Published mortality estimates for elderly ICU patients underestimate true risk by 66–168% across MI, sepsis, and respiratory-failure cohorts. Cross-validated on 201,905 encounters.
Read the full paper →The full library, including abstracts, OSF links, and download bundles, lives at /research.
Tell Adam which paper or which question, and he'll reply directly.