Cross-cohort results
Project overview · Source counts · Summary generator · Narrative category index
Counts are a dated snapshot: as of 2026-10-29 (commit ac95c69686). The scope contains 282 distinct protein records, including 242 prediction targets and 40 paired human reference records. Overlapping selections are counted once in the combined totals. These purposive, retrospective cohorts test informative biological distinctions; their proportions do not estimate proteome-wide accuracy.
186 of the 242 prediction targets have been assessed at the time of this build: 162 have a GO prediction-review YAML (including 17 reviewed empty GO outputs) and 24 have only a narrative function review. The remaining 56 are selected but not yet reviewed and contribute nothing to any count below.
GO-term assessments
Each row counted here is one emitted GO term. Narrative functions, protein names and SL localization outputs are excluded. Zero GO PLI or REP judgments does not imply the absence of narrative errors.
| Category | GO claims |
|---|---|
| COR | 35 |
| CNN | 48 |
| LSP | 83 |
| UNC | 78 |
| NPI | 42 |
| PLI | 2 |
| REP | 0 |
| Total | 288 |
GO assessments by cohort
COR here means "biologically supported and absent from the target's cached GOA/UniProt record", not novelty with respect to the model's training data (the VDCL sense). COR is only available when the target record lacks the term, so a cohort's COR rate reflects its records' existing annotation, which terms its predictions emit, and how its reviewers drew the LSP/COR line, as much as the model. Model-organism cohorts, where a supported prediction is usually already present (CNN) or less specific than an existing annotation (LSP), have almost no COR. Annotation density alone does not explain the split (compare the median GOA column), so per-cohort COR rates should not be read as a property of the model. Cohort rows overlap, so they are not summed. The median cached-GOA column counts non-NOT rows in each GO-assessed target's *-goa.tsv.
Latest annotation date in each target's cached GOA file (a lower bound on when the file was fetched): 2006: 2, 2022: 1, 2023: 1, 2026: 134, no annotations: 24. ProtNLM2 was trained on UniProt 2023_04, so COR/CNN are judged against a GOA state that post-dates training, and a COR term may still have been learnable from annotated orthologs in the training release.
| Cohort | GO claims | COR | CNN | LSP | UNC | NPI | PLI | REP | COR share | Median cached GOA rows per target |
|---|---|---|---|---|---|---|---|---|---|---|
| ARGO50 | 77 | 16 | 10 | 10 | 25 | 15 | 1 | 0 | 21% | 4 |
| HORSE40 | 89 | 16 | 21 | 10 | 28 | 13 | 1 | 0 | 18% | 6 |
| FLY41 | 50 | 2 | 7 | 19 | 14 | 8 | 0 | 0 | 4% | 9 |
| POMBE20 | 32 | 1 | 3 | 25 | 3 | 0 | 0 | 0 | 3% | 12.5 |
| NEUROSPORA20 | 21 | 0 | 6 | 14 | 1 | 0 | 0 | 0 | 0% | 8.5 |
| MOD_EVOLUTION20 | 19 | 0 | 1 | 5 | 7 | 6 | 0 | 0 | 0% | 1 |
A mechanical is_a/part_of check of all 35 COR calls against each target's cached GOA (cor_goa_entailment.py, TSV) finds DESCENDANT: 4, NONE: 31. 0 COR calls are entailed by an existing target annotation and are candidates for LSP/CNN re-review. Entailment by an existing MF through relations outside go-basic, or by an InterPro2GO domain mapping, is not checked.
Reviewed records with no GO predictions
The 162 prediction-review YAML files include 17 records with zero emitted GO predictions and 145 records with GO assessments. Completed predictions: [] records with a summary evaluation document reviewed output absence; a missing review file is not counted as zero output. Their descriptions record the available evidence for potential missed functions. No VDCL category or confidence score is assigned to an absent prediction, and these records do not enter the emitted GO-claim denominator. Zero output alone does not establish a biological false negative or a recall estimate.
Narrative function reviews
57 gene/accession review records assess emitted FUNCTION text. A record may contain multiple paragraphs and multiple claim categories. Each category below counts records with at least one such judgment, once per record; categories overlap and must not be summed or pooled with GO counts. These are neither atomic-claim counts nor one verdict per whole paragraph. Name and localization assessments remain in the gene notes and are outside both denominators.
| Category present in function review | Review records |
|---|---|
| COR | 4 |
| CNN | 24 |
| LSP | 2 |
| UNC | 26 |
| NPI | 16 |
| PLI | 13 |
| REP | 0 |
| SUPPORTED | 1 |
SUPPORTED records contain an explicitly supported claim whose review does not assign a novelty-specific COR/CNN/LSP category. They are preserved as such. References to separate GO judgments, hypothetical corrections, and claims explicitly not emitted are excluded.
Individual narrative records
Cohort scope
The cohort rows retain overlapping selections, including six fly targets selected twice. Paired reference records provide evidence and contribute no extra prediction assessments. Use the deduplicated totals above for the combined corpus.
| Cohort | Records | Records with GO assessments | Reviewed zero GO output | Narrative review only | Not yet reviewed | GO claims | Narrative review records |
|---|---|---|---|---|---|---|---|
| ARGO50 | 50 | 41 | 9 | 0 | 0 | 77 | 0 |
| HORSE40 | 40 | 27 | 0 | 13 | 0 | 89 | 17 |
| HORSE40_HUMAN_PAIR | 40 | 0 | 0 | 0 | 0 | 0 | 0 |
| FLY41 | 41 | 33 | 8 | 0 | 0 | 50 | 13 |
| FLY_LOCATION_KEYWORD | 29 | 0 | 0 | 0 | 29 | 0 | 0 |
| FLY_NEXT20 | 20 | 0 | 0 | 0 | 20 | 0 | 0 |
| POMBE20 | 20 | 18 | 0 | 2 | 0 | 32 | 10 |
| POMBE_REMAINING8 | 8 | 0 | 0 | 0 | 8 | 0 | 0 |
| NEUROSPORA20 | 20 | 16 | 0 | 0 | 4 | 21 | 3 |
| MOD_EVOLUTION20 | 20 | 10 | 0 | 9 | 1 | 19 | 14 |
Reference-pair rows (HORSE40_HUMAN_PAIR) are evidence records, not prediction targets, so their "not yet reviewed" count is zero by construction.
Source metadata
In the JSON summary, prediction_review_files counts all review YAML files, including explicit empty records. go_review_files is retained as a legacy alias for that same total; records_with_go_assessments counts only records with emitted GO claims.
source_method: ProtNLM2 names the model. source_version identifies the release, XML artifact or dated API snapshot. API retrieval timestamps are observation times, not model training dates. The pilot, exploratory XML and later API snapshots remain distinct; frozen responses and source references retain their original provenance.