Medicine
Medical research, evaluated on what was pre-specified
Medicine earns trust by saying in advance what it will measure, then reporting what happened. The draft rubric below scores that discipline, and the founding cohort will hold it to the field's own standards.
What counts as evidence of medical research skill
Medical research leaves an unusually public paper trail: trial registrations on ClinicalTrials.gov, systematic-review protocols on PROSPERO, reporting standards like CONSORT and PRISMA. Skill is legible in the distance between registration and publication — endpoints that held, deviations that were declared, analyses that were pre-specified rather than discovered.
Much of the field's strongest methodological work happens outside universities: trial design at sponsors and CROs, regulatory submissions that survive scrutiny, real-world evidence studies built on imperfect data with their limitations priced in. The people doing it are researchers by any honest definition, and almost none of it is citable.
Evidence synthesis is its own craft. A systematic review with a defensible search strategy, explicit inclusion criteria, and heterogeneity examined rather than averaged away tells an evaluator more about research judgment than many first-author originals do.
Critical appraisal has a public footprint in medicine that most fields lack: journal-club critiques, letters that identified a real flaw in a major trial, contributions to living guidelines, post-publication review of registered analyses. Evaluators here will weigh that record directly, because appraising someone else's evidence under a stated standard is, precisely, the job.
The medical evaluation rubric, first draft
Medical research earns trust through pre-specification, so this rubric scores what was promised before the data arrived against what was reported after.
- Protocol fidelity
- Registered endpoints match reported ones, deviations are declared and justified, and outcome switching is absent or owned.
- Bias control
- Randomization, blinding, and confounding strategy fit the design, and where randomization is impossible the causal caveats travel with the result, not behind it.
- Synthesis quality
- For reviews and meta-analyses: the search is reproducible, inclusion criteria are explicit, and heterogeneity is examined rather than averaged into false precision.
- Clinical relevance
- Outcomes matter to patients, surrogate endpoints are labeled as surrogates, and effect sizes are framed in terms a clinical decision could actually use.
Founding medical evaluators will test this draft against registered trials and published reviews before it scores anyone.
What founding medical evaluators will do
Stress the rubric where medicine is hardest to judge: single-arm trials, real-world evidence, diagnostics, and studies where blinding is impossible.
Run calibration rounds on public artifacts — registrations paired with publications, systematic reviews, guideline evidence tables — and chart where expert readers part ways.
Establish the calibration records that will weight medical evaluation when the community opens beyond its founding cohort — so that in this discipline, of all disciplines, the weighting of judgment is itself evidence-based from the first day.
Who this is for
The cohort needs people who practice evidence appraisal as a discipline, not a chore:
- Clinician-researchers and methodologists who read the registration before the abstract.
- Trialists, biostatisticians, and epidemiologists at sponsors, CROs, and public agencies.
- Systematic reviewers and guideline developers who practice evidence appraisal as a discipline.
- Early-career researchers in medicine whose methods training exceeds their publication list.
Who this is not for
The exclusions matter as much as the invitations here:
- Anyone seeking endorsement of their own trial, product, or practice — evaluation here reads other people's work.
- Researchers who treat reporting standards as bureaucracy rather than substance.
- Anyone unwilling to carry a visible record of their evaluation accuracy.
- Anyone who needs a live platform now; the mechanics are in design, and this page is deliberate about that.
Apply to evaluate medicine
Medicine is pre-selected on the application. Link an ORCID profile, registered trials, or published reviews we can read.