# AI instructions for ai-gene-review ## Project Overview This project is for managing AI reviews of gene annotations. The overall goal is to both review existing GO annotations using strict defined criteria, and to summarize the resulting high quality annotations. ## Local parallel work and storage - Keep at most six unfinished workstreams, including the coordinator. An idle agent or an open PR still occupies a slot until its work is delivered and its workspace is retired. At the limit, finish existing PRs before starting genes. - Keep campaign working storage around 30 GB total, including the coordinator, child checkouts, temporary files, environments, and caches. Count shared stores once. Use fewer or sparse checkouts if six do not fit. When over budget, stop intake and reduce storage before resuming research. - Use Orca-managed worktrees when running in Orca; otherwise use standard registered Git worktrees or isolation explicitly provided by the environment. Verify the checkout's top level and Git directory before writing. Never copy a repository into `tmp/` to simulate isolation: Git discovery there may resolve to the parent checkout and share its index. Resolve metadata access through the environment's supported worktree interface; do not work around a Git permission error by making a nested repository copy. - Assign one writer, branch, and PR to each checkout. Read-only helpers may share it; concurrent writers must have distinct registered worktrees. Share existing runtimes and immutable source caches rather than copying them per task. Identify retained cache versions by source revision or content hash; never overwrite a keeper that an earlier cleanup receipt or active review depends on. Workers must use project code from their own registered checkout and must not reinstall or mutate a shared environment concurrently. Check editable installs and shared cache references before retiring the checkout they point into. Calling the shared Python interpreter does not constrain nested `uv run` subprocesses. Prefer direct validation helpers; set `UV_PROJECT_ENVIRONMENT` to the retained runtime and set `UV_NO_SYNC=1`, `UV_FROZEN=1`, `UV_OFFLINE=1`, and `PYTHONDONTWRITEBYTECODE=1` before validation. Verify that neither a local `.venv` nor `uv.lock` changed unexpectedly after running a wrapper. - At delivery, verify that the PR and retained Git history contain the intended work, preserve unpublished changes and unique evidence, then retire completed worktrees through their owning tool. Remove redundant temporary copies against explicit retained copies or recoverable Git objects, with a compact receipt. Treat evidence archives, keeper catalogs, retention registries, and recovery refs as retained data even when they live under `tmp/`. Verify and record any replacement before removing them; a missing original path is not proof of loss or permission to discard its recovery archive. Do not retain full source snapshots, API payloads, or environments per review. - Report measured campaign storage and unfinished workstreams before starting a new batch. File allocation and APFS physical free-space changes are different measurements; do not claim one proves the other. ## Repository layout src/ ai_gene_review/ schema/ gene_review.yaml <-- linkml schema for review docs genes/ human/ CFAP300/ CFAP300-notes.md <-- keep your notes here CFAP300-ai-review.yaml <-- the output of your AI review CFAP300-notes.md <-- your own notes, edit this, use it like a journal and keep appending. CFAP300-uniprot.txt <-- the uniprot record downloaded from the UniProt website (do not edit this file) CFAP300-goa.tsv <-- the GO annotation downloaded from the QuickGO API (do not edit this file, no need to view it) CFAP300-deep-research.md <-- the deep research for the gene review (do not edit this file, it is automatically generated for you) CFAP300-citations.md <-- the citations for the gene review (do not edit this file) CFAP300-bioinformatics/ <-- the folder for any bioinformatics analyses (create this if necessary) CFAP300-predictions-review.yaml <-- review of computational predictions NOT in GOA (optional) / ... worm/ lrx-1/ SCHPO/ <-- other orgs use uniprot species codes ... publications/ PMID_123456.md <-- cached publication text (this should be available for you, DO NOT EDIT) reactome/ R-HSA-NNNNNN.md <- cached reactome entry gocams/ MODEL/ <-- one folder per production GO-CAM model (bare local id) MODEL-src.yaml <- cached gocam-py model (activities/annotons; DO NOT EDIT) MODEL-review.yaml <- optional reviewer assessment index.tsv <- gene_product -> GO-CAM activity (annoton) index; join key to reviews/modules history/ genes/// <- append-only curation session records (see docs/history.md) modules//, gocams//, projects//, schema/, other/ ## History records `history/` holds append-only curation session provenance, one YAML per session per target, kept outside the curated files themselves (mechanism ported from dismech). When a PR creates or edits curated content (a gene review, module, GO-CAM review, or project page), add a matching record — scaffold it with `just new-history` (never hand-write the filename/session id), edit the emitted `details`, then check it with `just validate-history `. Records are append-only: never rewrite an existing record's `target.slug`/`target.path`; use `target.superseded_by` for renames. See `docs/history.md` for the format and `just backfill-history` for retrospectively generating records from PRs. You can regenerate the derived files by running commands like: ``` just fetch-gene human CFAP300 ``` This makes the uniprot, goa, and publications files. It will also create a stub for the ai-review.yaml if it doesn't exist. After making edits to the ai-review.yaml file, you can validate the file by running: ``` just validate human CFAP300 ``` This ensures that the yaml file is consistent with the schema in `src/ai_gene_review/schema/gene_review.yaml`, and also that it implements best practices. To validate all files, run: ``` just validate-all ``` files will be written to reports/validation-all.tsv * All gene symbols follow the standard nomenclature for that species. For human, should be HGNC symbols. ## Gene Review Guidelines Your job is to review existing annotations for a gene, following GO guidelines, and synthesize these with what is known in the literature, and what can be inferred bioinformatically. The general structure of the review is as follows: ```yaml id: gene_symbol: taxon: id: label: description: references: existing_annotations: core_functions: proposed_new_terms: suggested_questions: suggested_experiments: ``` See the schema and existing files for more details. ### Description field quality The top-level `description` field is for a project-independent biological summary that a biologist can read without knowing the current curation project. It should describe what the gene product is, where it acts, and its major biological roles. Do not put project, workflow, or curation commentary in `description`, including phrases such as `Proteostasis Network`, `PN`, `this review`, `curation`, `GOA correctly captures`, or `should/should not be annotated`. Put that material in the gene notes file (`GENE-notes.md`), in `review.reason` for a specific annotation, or in `core_functions`/`suggested_questions` as appropriate. When creating a new review, always make sure files are in place: `just fetch-gene ` This creates the uniprot, goa files, and a stub for the gene review. The next thing is for you to do deep research on that gene and make a GENE-notes.md file, with citations inline (e.g [PMID:12345 title, ""]). In some cases this will have been done ahead of time for you. ## Reviewing existing annotations You should start with the ai-review.yaml file, where stubs are created for you (from `just fetch-gene `). Any PMID cited should be in the publications folder, e.g. `publications/PMID_123456.md`. Review these. A deep research file should already be present in the gene folder, e.g. `genes/SPECIES/GENE/GENE-deep-research-PROVIDER.md` (where PROVIDER is openai, falcon, perplexity, etc.). This should contain a summary of the literature, and any other information you found useful for the review. There may be irrelevant information in the citations, that is OK. If this is not present, you can generate one using: ```bash just deep-research SPECIES GENE --provider perplexity ``` This will create a file like `GENE-deep-research-perplexity.md`. You can also make your own notes in `genes/SPECIES/GENE/GENE-notes.md`. NEVER write your own content and name it `-deep-research-{provider}.md`. If deep research fails, document in `-notes.md` or `-deep-research-manual.md` instead. For worm, fly and yeast genes, also run `just mod-description-deep-research SPECIES GENE`. It writes the model organism database's hand-written gene description verbatim as `GENE-deep-research-.md` (`wormbase`, `flybase`, `sgd`). Other organisms are skipped: MGI's note summarises mutant phenotypes, and RGD, ZFIN and Xenbase have no hand-written text (see `projects/MOD_DESCRIPTIONS.md`). This is provider content, like an Affinage file, not your own. Treat it as an uncited, often out-of-date source: check its claims, record your verdict in that reference's `reference_review`, and record errors in its text as `source_defects` (see `docs/source_defects.md`). A review judges claims about the gene, not evidence codes. Rows that share a term (e.g. the same term from IBA, IEA and NAS) reinforce each other; their actions must be consistent, and REMOVE is never the way to deduplicate them or to drop a true-but-broad term. (The one sanctioned exception is bare `GO:0005515` protein binding, which the protein-binding policy removes as uninformative; see `src/ai_gene_review/validation/protein_binding_policy.py`.) Consistency rules live in `src/ai_gene_review/schema/action_consistency.yaml` and are explained in `docs/action_consistency.md`. Be aware that many GO terms are over-annotations. You should also not take existing annotations as gospel, whether experimental or bioinformatic. always use a holistic synthesized picture of the gene function that you have obtained from your research. ```yaml ActionEnum: permissible_values: ACCEPT: description: Accept the existing annotation as-is, no modifications, and retain as representing the core function of the gene. For an ND root-term row, ACCEPT means the annotation follows the evidence-code guideline (see "ND annotations" below) KEEP_AS_NON_CORE: description: Keep the existing annotation as-is, but mark it as non-core. For pleiotropic genes, this may be the developmental processes, or other processes that are not the core function of the gene. REMOVE: description: Remove the existing annotation, as it is unlikely to be correct based on combined evidence MODIFY: description: The essence of the annotation is sound, but there are better terms to use (use in combination with proposed_replacement_terms). if the term is too general, then MODIFY should be used, with a proposed replacement term for the correct specific function. sometimes terms can also be overly specific and contorted, so in some cases you might want to generalize MARK_AS_OVER_ANNOTATED: description: The term is not entirely wrong, but likely represents an over-annotation of the gene TOO_GENERAL: description: The term is true of the gene but uninformative, because a more specific term asserted elsewhere in this review already entails it. Judges the term, not this row's evidence; never means the claim is false. Use sparingly; not for GO:0005515 protein binding. UNDECIDED: description: The annotation is not clear, and the reviewer is not sure what to do with it. ALWAYS USE THIS IF YOU ARE UNABLE TO ACCESS RELEVANT PUBLICATIONS NEW: description: This is a proposed annotation, not one that exists in the existing GO annotations. Use this to propose a new annotation not covered by the existing GO annotations. Use this conservatively, do not over-annotate, especially for biological process. Do not use for indirect or pleiotropic effects. Be sure you have good evidence, this can be from multiple sources. ``` **Correct but general terms are `MODIFY`, not `KEEP_AS_NON_CORE`.** When a term is right but less specific than the evidence supports (e.g. `membrane` vs `plasma membrane`, `peptidase activity` vs `serine-type endopeptidase activity`), use `MODIFY` with `proposed_replacement_terms` set to the specific supported term (an `is_a`/`part_of` descendant, preferably one the gene already carries; say so in `reason`). `KEEP_AS_NON_CORE` is only for terms that are as precise as warranted but peripheral to the core function. If the general term is already as precise as the evidence allows, keep it (`ACCEPT` if core, otherwise `KEEP_AS_NON_CORE`). The action must match the reason: never `ACCEPT` or `KEEP_AS_NON_CORE` while arguing that the term is too general. When several specific descendants are supported, as for a pleiotropic gene, list all of them rather than picking one. This rule is about refinement. Lateral or sign corrections, such as `positive` → `negative regulation of X`, are a separate, long-standing use of `MODIFY` whose replacement is not a descendant. **Measured activity is not automatically function.** For an enzyme activity that was assayed but is weak or promiscuous, decide whether the protein evolved to catalyse the reaction: a genuine secondary activity is `KEEP_AS_NON_CORE`; real in vitro activity of unknown relevance is `MARK_AS_OVER_ANNOTATED`; noise-level activity of a paralog, when a dedicated enzyme for the same reaction exists in the organism, is `REMOVE`, even on an IDA row. There is no numeric cutoff; state the judgement in `reason`. See the promiscuous-activity tiers in `projects/ENZYME_SPECIFICITY.md`. **Decomposing precomposed "X involved in Y" terms.** When you MODIFY such a term to its genus X, say where Y went in `review.decomposed_into`: `gocam_associations` or `module_associations` (Y is a mechanism with a model), `in_core_functions` (Y is central to the gene), or `context_dropped: true` (Y is peripheral context or was itself the over-annotation). Do not encode Y as an annotation extension (X plus `part_of(Y)` restates the retired class), and do not split the term into two unrelated annotations. Modules model reusable mechanisms, not tissue or developmental contexts: do not create a module per Y. GO's `consider` is curator advice: check each half against the evidence before using it. Its `replaced_by` is an automated mapping, so if it is wrong for your gene, record the correct term and flag the `replaced_by` upstream. See `projects/INVOLVED_IN_PRECOMPOSITION.md`. ### ND annotations: reward adherence to the evidence-code guideline An ND (No biological Data available, `GO_REF:0000015`) row says a curator searched the literature for one aspect and found nothing to annotate. **ACCEPT it when no literature on this gene product, in this species, identifies a function in that aspect.** IBA, ISO, ISS or IEA rows in the same aspect do not make it stale: they are not literature on the gene, so never REMOVE an ND "because an IBA now covers the aspect". Only when such literature exists is the ND stale: REMOVE it (or MODIFY, when no fitting term existed) with `nd_case: STALE` or `ONTOLOGY_BLOCKED`, and quote the paper. Never use `KEEP_AS_NON_CORE`, `MARK_AS_OVER_ANNOTATED` or `TOO_GENERAL` on an ND, and never propose a `NEW` ND (record a `knowledge_gaps` entry instead). Record why there is no data in `review.nd_case`; `TRUE_GAP` and `EXISTENCE` need a BIOLOGY knowledge gap whose `dark_aspect` matches the ND aspect. What counts as literature on the gene: a bare protein-binding IPI does not (it identifies no function, so it does not make an MF ND stale); a paper on an ortholog does not, however strong; an `NAS` or Reactome `TAS` row on the gene itself does. An accepted ND must cite the search: the publications checked (PMID, DOI, PMC), or a `supported_by` `supporting_text` quote from the gene's own deep-research or notes file where it records the queries and their result; the UniProt record and MOD gene descriptions are not a search. The `nd_policy` check warns on deviations, and the compliance score credits an accepted ND that cites the literature search. See `projects/NO_DATA.md` for the case taxonomy. ### Root organelle terms: keep the organelle, add the sub-compartment Cellular-component rows on a root organelle term (`GO:0005739` mitochondrion, `GO:0005783` endoplasmic reticulum, `GO:0005794` Golgi apparatus, `GO:0005777` peroxisome, `GO:0005764` lysosome) follow the policy in `projects/ORGANELLES.md`. The nucleus, the series organelles (endosome, vacuole, autophagosome) and plastids are not covered yet. For these five terms the policy is an exception to the "correct but general is `MODIFY`" rule above: - **The organelle term is informative.** Never use `TOO_GENERAL` for it, and never `MODIFY` it to a sub-compartment. Knowing a protein is in the mitochondrion is a real claim even when its membrane or lumen is also known. (A generic term such as `GO:0016020` membrane is not a root organelle term, so the rules above still apply to it.) - **Direct evidence** (IDA, EXP, HDA, HTP, ...) from an assay that cannot resolve the sub-compartment (organelle proteomics, whole-organelle co-staining, GFP fusion) is `ACCEPT`, or `KEEP_AS_NON_CORE` if the location is peripheral to the function. - **Existing synthesis rows** (IBA, IEA, ISS, ISO, IC, ...) are a red flag for granularity, not soundness: examine what the source rests on (the PAINT IBD seeds, the UniProt keyword, the ortholog's own annotation) to see why the row stops at the organelle and whether the organelle claim holds. For IBA this is the phylogenetic reading described below, not a donor count. If the organelle claim holds, keep the row and record the check in `reason`; if it does not, `MARK_AS_OVER_ANNOTATED`, or `REMOVE` for a demonstrably wrong electronic mapping or over-propagation. - **Be bold about the sub-compartment, but add it, don't substitute.** When the gene has no sub-compartment row and at least two independent lines agree (a TM segment, a cleavable presequence, a sub-organelle assay, a function that only makes sense on one side), add the matrix, membrane or lumen term as a `NEW` row (usually IC, an inference you author and justify) beside the root rows. With one line, or lines that disagree, stay at the root and say why. - **`core_functions.locations`** names the sub-compartment only where the core activity is shown to happen; a well-supported sub-compartment location is not by itself where the function acts (see the MMADHC note in the project). - **Check crude fractions for ER.** A "mitochondrial" fraction that was not checked for an ER marker cannot separate mitochondrion from mitochondria-associated ER membranes; transient recruitment to the organelle surface (signalling, innate immunity) is not an outer-membrane location. - **ER: transit is not residence.** A secreted or plasma-membrane protein's ER row is `KEEP_AS_NON_CORE` only with direct evidence on the gene or the endogenous protein in its native cell; a synthesis row resting only on transit or on retention of an overexpressed or mutant protein is `MARK_AS_OVER_ANNOTATED`. Never add a topology-forced ER membrane row for a protein that only passes through. A `NEW` ER membrane (`GO:0005789`) row needs a TM or tail anchor plus an ER-specific assay on the endogenous protein; a lumen (`GO:0005788`) row needs a soluble signal-peptide protein shown to be retained. A soluble, cytosolically made protein (no signal peptide or targeting sequence) shown at the organelle and stably tethered there by a named integral partner (losing the partner releases or destabilises it) gets the cytoplasmic-side term, `GO:0098554` for the ER, which is the more specific child of `GO:0005789`; brief or regulated recruitment stays at the root. See rule 7 in `projects/ORGANELLES.md`. ### Do not overrule curators from incomplete evidence PomBase, and the GO consortium databases generally, are highly reliable. Curators who make an experimental annotation (IDA, IMP, IPI, IGI, etc.) have read the **full text**, which is often NOT in our `publications/PMID_*.md` cache — many cached entries are **abstract-only** (check the `full_text_available:` field). A paper's **title or abstract frequently foregrounds one gene (or a paralog) while the full text also assays the gene being annotated** — this is normal, not a curation error. Therefore: - **Never use `REMOVE` (or assert "wrong gene / paralog mis-attribution / name confusion") for an experimental annotation just because the cached title/abstract is about a different gene, paralog, or organism.** You cannot see what the curator saw. If you genuinely cannot verify the supporting evidence, use `UNDECIDED` (per the enum) or, if the function is clearly correct for the gene, `ACCEPT` and defer to the curator. - Before claiming an annotation is mis-attributed, **verify the actual GO term definition** (via the OLS MCP / QuickGO) and the **organism in the paper** (read the abstract; for abstract-only papers, try WebSearch or the article MCP for full text). E.g. GO:0003925 "G protein activity" includes small monomeric GTPases; a title naming a paralog does not mean the gene was not also assayed. - `REMOVE` is appropriate for: genuinely contradicted functions, EC/IEA mappings that are demonstrably wrong, or over-propagated electronic (IEA/IBA) inferences you can argue against on biological grounds — **not** for second-guessing an experimental annotation whose full text you have not read. - Reserve confident "this reference is about organism/gene X" caveats for cases where the **cached abstract explicitly states it** (e.g. "in nontransformed mammalian cells"). Do not infer the organism or assay details that the abstract does not state. ### Do not add what curators deliberately declined to add The mirror image of the rule above, and the one that governs `NEW`. Everything else in this document is an audit of assertions that exist; `NEW` is the one action that manufactures an assertion, and it needs its own bar. **A gene product is `involved_in` a process only if the product itself does some of the work of that process — catalysing a step, or contributing the structure or cofactor activity that a step depends on.** Being consumed by the process, being required for it, or being the thing it acts on is not participation. The distinction matters most where the evidence is strongest: knockout abolishes the outcome, rescue restores it, and human loss-of-function is lethal — all of which establish that the gene product is **necessary**, which is exactly what being a substrate means. Necessity evidence and participation evidence are indistinguishable in a GAF row, so ask explicitly *which entity performs the step* before proposing a process term, and do not let a mountain of perturbation data stand in for an answer. **"Every other participant has this term and my gene does not" is not evidence of a gap.** It is usually evidence that the term means something your gene does not do. Before proposing `NEW` on that reasoning, run the comparator check: name two or three other gene products standing in the **same role** relative to the same kind of process, and see whether they carry the term. This converts "the curators overlooked it" from an assumption into a prediction you can falsify in one QuickGO query, and a systematic absence across species and MODs should be read as a convention you have not yet identified, not as a twenty-year oversight. A worked example, from a review where this went wrong (`genes/human/AGT`, PR #2972). `GO:0002003 angiotensin maturation` is annotated to every protease that acts on angiotensinogen — REN, ACE, ACE2, ENPEP, MME and a dozen more — and to no angiotensinogen in human, mouse or rat. That was read as a pathway-completeness gap. The comparator check settles it the other way in a single query: INS is absent from `GO:0030070` insulin processing, whose annotations are the convertases; POMC, GCG, PENK and NPPA are absent from `GO:0016486` peptide hormone processing, on which CORIN sits but its substrate pro-ANP does not; APP is absent from `GO:0034205` amyloid-beta formation among 182 annotations. The reason is structural: `GO:0002003 is_a GO:0016486` peptide hormone processing, under protein processing and proteolysis — the term describes the cleaving, and the substrate does none of it. The convention is not absolute, and the exceptions tell you where the line is. GO *does* annotate thyroglobulin to `GO:0006590` thyroid hormone generation, fibrinogen to `GO:0042730` fibrinolysis, and C3 to `GO:0006956` complement activation. Each of those substrates does some of the work, and the three span the range of what that can mean. Thyroglobulin is the **scaffold** case: TPO catalyses the iodination and the coupling, but thyroglobulin supplies the tyrosyl residues and holds the donor and acceptor pair in position — and `GO:0006590` carries both, the peroxidase and the scaffold, each by IDA. Fibrin is the **cofactor** case: it polymerises, then accelerates its own lysis by acting as the template for tPA-mediated plasminogen activation. C3 is the **chemistry** case: it carries an internal thioester that cleavage exposes, which C3b then uses to form its own covalent bond to the target surface. Angiotensinogen is none of the three — it supplies no residue to the product beyond the bond that is cut, positions nothing, and catalyses nothing; renin performs every step of the conversion. So the test is never "is my gene the substrate" but "does my gene do any of the work", and a fibrinogen-shaped case can legitimately carry the term. Therefore, before proposing a `NEW` process term: - **Name the entity that performs the step.** If the answer is another gene product, the term belongs to that one. Necessity evidence does not answer this question. - **Run the comparator check** above, and treat a systematic absence as a convention to identify rather than a gap to fill. - **Read the term's parents.** They usually say what kind of thing carries the term. A process under `GO:0006508 proteolysis` names whatever does the cleaving — which is the substrate itself in the autoprocessing case (`GO:0016540`), and otherwise is not. - **Check `gocams/index.tsv` and the cached models.** They often contain the gene already, in the role GO intends for it — the renin-angiotensin model (`gocams/6246724f00000549/`) has AGT twice over, as the hormone-activity node and as the proteases' input molecule. A curator who modelled the pathway, included your gene, and gave it a different term made a decision to argue with explicitly, not an absence to fill in. - **Reject a term that is an ancestor or descendant** of another you are proposing, or of one the gene already carries. That is redundancy, not added coverage. Where a substrate relationship genuinely needs to be machine-readable, the GO mechanism for it lives on the enzyme (`has input`), not on the substrate. Raise it as a `suggested_questions` entry rather than asserting it as an annotation. ### What an IBA asserts: read the phylogeny, not the donor count An IBA is **not** a pairwise similarity transfer. Behind every IBA is a PAINT curator's **IBD** (Inferred from Biological aspect of Descendant): the curator inspected the family tree and the MSA, read the experimental annotations of all extant members, judged at which node the function arose — sometimes recent, sometimes as deep as LUCA — and placed the assertion there. IBA rows follow mechanically from descent from that node. So an IBA carries a considered phylogenetic judgment, and challenging one means arguing with the node placement (is the target inside the clade that inherited the function? is there target-specific evidence of loss or divergence?), not with a similarity score. Two consequences that are easy to get backwards: - **A short `WITH/FROM` donor list is not weak support.** A node seeded by a single well-characterized MOD or human gene can be entirely sound. Do not use donor count as a proxy for evidential strength. - **The target appearing in its own `WITH/FROM` is correct and expected — not circular.** When a gene has its own experimental annotation for the term, that annotation is one of the descendant evidences the curator used to place the IBD, so the gene legitimately appears among the sources of the IBA it later receives. It is a marker that experimental grounding exists *on the target itself*, and the IBA then says something additional: that the function is inherited rather than lineage-specific. **Never** mark such a source `CIRCULAR_OR_REDUNDANT` or describe it as inflating support. Reserve `CIRCULAR_OR_REDUNDANT` for genuine circularity: a propagation whose source is itself a propagated annotation with no experimental grounding anywhere in the chain, or a source that adds nothing because the target already has stronger direct evidence for the same claim. See `projects/IBA_REVIEW.md` for the full propagation taxonomy and catalogued failure patterns. In a family review (`interpro/panther//-review.yaml`), a node assessment that disputes a PAINT row records the fix in `proposed_action` as well as in prose: `paint_action` (MOVE_IBD, REMOVE_IBD, ADD_IRD, ADD_IKR, REMOVE_LOSS, MOVE_LOSS) and, for the moves and added losses, `target_node_id`. `family_residue_validator` checks that the action fits the verdict and that the target sits correctly in the family's PANTHER tree (below the node for TOO_DEEP, NEEDS_PRUNING and LOSS_TOO_BROAD). PAINT prunes per annotation: a loss of a general term does not block a more specific term asserted at an ancestral node, so request each change separately. ### Term-id validation: GOA ids are trusted, your `core_functions` ids are checked Validation deliberately treats the two sources of GO term ids differently: - **`existing_annotations[].term.id` is NOT hard-validated.** These ids come from GOA, not from you, so there is no hallucination risk — and GOA legitimately lags or leads the ontology version used for validation. `linkml-term-validator` therefore does **not** reject unresolvable or obsolete ids here (e.g. a non-existent `GO:9999999` passes); label correctness is instead checked against GOA in the best-practices rules. **Do not rewrite an existing-annotation id just to satisfy validation.** - **`core_functions` term ids ARE strictly validated.** These you author, so the `molecular_function`, `contributes_to_molecular_function`, `locations`, and `in_complex` slots are bound to dynamic enums (branch-reachability). A wrong-branch term — e.g. a cellular-component term placed in `molecular_function` — is a blocking `❌ ERROR`. Rule of thumb: machine-sourced ids are trusted (other deterministic steps guarantee they are real GOA terms); author-supplied ids are checked hard. When a core activity has **no GO term yet** (e.g. in-situ holdases after GO:0051082 was obsoleted without a replacement), do not put an obsolete or ill-fitting id in `core_functions.molecular_function`. Instead set `proposed_molecular_function` to the `proposed_name` of a top-level `proposed_new_terms` entry, and leave `molecular_function` unset. Validation errors if no entry matches, if both are set, or if neither the core function nor the proposed term has `supported_by` (a proposed activity can never trace to an existing annotation). In a MODIFY, the matching replacement is `proposed_replacement_terms: [{id: NTR, label: ...}]` whose label starts with that same `proposed_name` (a warning otherwise). When GO creates the term, swap its id into `molecular_function`. ### PANTHER ids: never write a family label from memory PANTHER family/subfamily ids (`PANTHER:PTHR12345`, `PANTHER:PTHR12345:SF7`) used in modules are now hard-validated against `interpro/panther/panther.obo`, built from PANTHER's own HMM classifications. Two rules follow: - **`term.label` must be PANTHER's official name, verbatim.** It is not a place for your description of the protein. Writing a plausible-sounding label is exactly how a wrong family id stays hidden — the id is real, so nothing else catches it. Put your readable description in `preferred_term`, which is free text and is not label-checked. Look the name up: `grep -A1 "^id: PANTHER:PTHR12345$" interpro/panther/panther.obo` - **The declared family must contain its own `representative_members`.** This is checked against the PANTHER member index and is a blocking error. If it fires, the representative protein is usually right and the family id is wrong — look up the member's real family rather than deleting the member. The index is a build artifact, not committed: it lives in the git-ignored `.cache/panther/panther-members-.tsv`, and `just validate-modules` / `just validate-families` build it automatically (incrementally, from release-pinned PANTHER classifications plus UniProt). Do not commit it or cite it as a `file:` source; cite PANTHER's classification files or UniProt instead. Accessions the index cannot resolve only warn. Where PANTHER's own files and UniProt's PANTHER cross-reference disagree (different families, or different subfamilies), the index keeps both and a member matching either passes with a warning; do not "fix" such a family id to the other source without checking which placement is right. Where a curator has decided which assignment is right, add a reasoned row to the committed `interpro/panther/panther-members-overrides.tsv`; overrides are applied after both sources on every build. `just refresh-panther-members --rebuild` regenerates the index from scratch. - **If a label mismatch names a *different protein*, fix the ID, not the label.** A wildly-wrong label is weak evidence of a typo and strong evidence that the id was guessed. An id invented at random is still a hallucination when it happens to resolve to a real family, and rewriting its label to the official name converts a visible error into an invisible one. `just fix-panther-labels` therefore refuses to touch these and reports them for review (`--allow-divergent` overrides, only after you have checked the id). Beware the inverse too: PANTHER family names are often dominated by one member, so a correct id can legitimately carry a surprising name (`PTHR24322` = "PKSB" really does contain DHRS3). The representative-member check is what tells the two cases apart. PTN ancestral nodes (`PANTHER:PTN...`) are checked separately, against PAINT data — see `validate_paint_ptns`. Their label is conventionally just the bare id. **If you cannot be sure, assert no id.** A family descriptor's `term` is optional. When you cannot establish that a PANTHER family corresponds to what the descriptor means, omit `term` entirely and keep `preferred_term` (free-text intent) and `representative_members` (the proteins themselves). An omitted id says "not established"; a wrong id says something false, and says it in a machine-readable field other tooling will believe. Never invent a plausible id to fill the slot, and never guess a replacement for one that failed validation -- re-pointing a family is a judgement about evolutionary placement, and doing it mechanically has previously broken real PAINT links. The same rule applies to evolutionary claims generally: if the PAINT evidence for a step is not clear, say nothing about it rather than asserting an inference the data does not support. ## Curated annotation rules `rules/curated/CR-nnnn.yaml` holds annotation rules this project owns (`CuratedRule`). A rule couples sequence-feature conditions with taxon or module-context conditions and names the InterPro2GO mapping or UniProt rule it replaces. Use one when a mapping is right for some members and wrong for others and a condition can separate them; when a term is wrong for every member, record a removal instead. Module conditions are exclusion-only. See `rules/curated/README.md`; check with `just validate-curated-rules`. ### Reviewing rules: every verdict needs receipts A review of an ARBA or UniRule rule (`rules/**/-review.yaml`) is a verdict on someone else's rule, and an audit in November 2026 found most earlier verdicts did not follow from their evidence. `just rules-validate` now checks receipts as well as schema (`src/ai_gene_review/validation/rule_review_receipts.py`); read `docs/rule_review_receipts.md` first. In short: - **Record what you examined.** `rule.condition_sets` from the fetched rule, with `retrieved_date` and `source_condition_set_count`. A verdict on a rule whose sets are not recorded is an error; a rule you only sampled is `UNDECIDED`. - **Never name an identifier from memory.** Write `IPR000999 (Ribonuclease III domain)`; the label is the checksum of the id, and it is checked. - **Judge the conjunction, not the signature.** A condition set is everything it requires together, taxon included. Alternative sets are meant to be disjoint. - **Size, generality, redundancy and low recall are not errors.** The "limit of 12 condition sets" is the analysis tool's default, not a standard. A true but general term is `MODIFY` or `UNDECIDED`, never `DEPRECATE`. - **`DEPRECATE` names a wrongly annotated protein** (accessions in `annotation_assessments[].incorrect_examples`), an obsolete consequent or a withdrawn rule. Before listing a protein as a false positive, look up its experimental annotations to the term and its descendants; `--online` does this. - **Your notes and deep-research files are not literature.** Cite the paper or record they point to, or omit `literature_support`. - **`confidence` is confidence in the verdict**; below 0.5 the verdict is `UNDECIDED`. - **Withdraw wrong claims in `corrections`**, with the evidence that refuted them, and delete them from the prose. ## Reviewing references Entries in the top-level `references:` list can carry an optional `reference_review` object recording your **manual** assessment of each reference. The `id` and `title` are machine-fetched (title comes from the cached `publications/PMID_xxxx.md`); `reference_review` is reviewer-supplied judgment. Use it especially to flag citation problems that format validation cannot catch (e.g. a well-formed PMID that points to the wrong paper). The reference validator already catches the *mechanical* citation failures automatically: it verifies each cited reference's `title` matches the fetched record (a transposed/wrong PMID whose title no longer matches fails) and that every `supporting_text` is a **verbatim substring** of the cached publication (a quote from the wrong paper, or a paraphrased/invented quote, fails). A `supporting_text` value must contain actual verbatim source text. Editorial square brackets may clarify part of a real quotation; an empty string or a value consisting only of editorial commentary is not a quotation. Omit this optional field when no quotation is being asserted, and keep method commentary in the finding statement, review reason, or notes. Scientific notation copied from the source (such as `[4Fe-4S]`) is literal text, not editorial commentary. Quotes citing `file:` or `Reactome:` sources are checked the same way, against the local file (resolved under `genes/`, then the repository root) or the cached `reactome/R-*.md` entry; a UniProt flat-file quote may run across the file's line wraps. Failures that predate this check are listed in `conf/local_quote_baseline.yaml` and only warn; fix them and run `just local-quote-baseline --prune`, never add to it. If `just fetch-gene` or a re-run deep-research file changes a source so that a quote no longer matches, re-quote the new text in the same change; if a review file is renamed, move its key in the baseline unchanged. The same checks apply to every other YAML kind that carries quotes: modules, family reviews, GO-CAM reviews, rules and projects. `just validate-quotes` (`src/ai_gene_review/validation/quote_check.py`) finds every `supporting_text`, whether its source is named by `source_id`, `reference_id`, `reference` or the enclosing `references[].id`. It runs on every PR, not only when the quoting file changes, because a quote breaks when its *source* changes: editing a gene review can break a family review or module that quotes it, so re-run it after rewriting any file that others cite. A quote whose source has no readable text (`url`, `GO:`, `PMCID:`, no reference id) is reported as unchecked, never as verified. Failures that predate the check are in `conf/quote_check_baseline.yaml`, which only shrinks (`just validate-quotes --prune-baseline`). **Never cite a GOA file as support.** `GENE-goa.tsv` lists the annotations under review: it says what is annotated, never why it is true, so a `supported_by` entry pointing at it cites an annotation as evidence for itself. Another gene's GOA file is no better: that a paralog or partner carries a term is the state of its annotation, not evidence. This is a blocking error whatever the action (`supporting_text_goa_file`, `src/ai_gene_review/validation/goa_support.py`). Quote the publication or curated UniProt statement behind the annotation, or your own analysis; if nothing independent supports the row, use `UNDECIDED`. A comparator argument ("paralog X carries this term") belongs in `reason`. The same holds in prediction reviews (`validate-predictions`): `supported_by` shows the predicted term is true or false, and that it is already in GOA (CNN, LSP) is said in `summary`. **Quote UniProt for evidence, not for labels or unvetted predictions.** These rules apply to every UniProt quote, whatever it supports. - **Good support:** experimental and curated statements (kinetics, binding or active sites, curated free text, best of all text tied to a publication: evidence `ECO:0000269` with a PubMed id; syncing such curated notes is part of our job). Family and domain classification is also good support for membership, and you may use membership to argue for a function: NCBIfam, InterPro/Pfam/PROSITE signature matches, the curated UniRule systems (HAMAP, PIRSR, PIRNR, RuleBase), "Belongs to the X family" notes (unless their only evidence is ARBA) and family cross-references. Structure-derived automatic evidence (`ECO:0007…`) counts too, and so do sequence-analysis features (`ECO:0000256|SAM:...`: Phobius transmembrane helices, SignalP signal peptides, MobiDB-lite disordered regions, Coils coiled coils), which are valid computational evidence. - **Not support, unvetted predictions:** a statement whose only evidence is an ARBA association rule (`{ECO:0000256|ARBA:...}`), annotation imported unchecked from the submitter (`ECO:0000313`), a name generated by a machine-learning model such as ProtNLM (`ECO:0008006`), or an electronic GO line (`IEA:`). Citing one repeats a prediction; citing ARBA text for an ARBA annotation is circular. - **Not support, labels:** protein names and EC numbers (`DE`), keywords, and the record's own `DR GO` lines restate an annotation rather than support it. - **Protein existence (`PE`) is not evidence of function.** Quote it to argue that an annotation rests on prediction (e.g. `MARK_AS_OVER_ANNOTATED`, `KEEP_AS_NON_CORE`, `MODIFY`, `UNDECIDED`), never to support an `ACCEPT`, a `NEW` or a core function. An accepted `ND` annotation needs evidence that the literature was searched, not the existence level. Validation warns on all three (`uniprot_quote_unvetted_prediction`, `uniprot_quote_restating_field`, `uniprot_quote_existence_as_support`; `src/ai_gene_review/validation/uniprot_quote_source.py`). **Record which section a UniProt quote comes from.** A UniProt quote's `supporting_text` item carries `record_section`, the section(s) of the record it quotes: `CC:CAUTION`, `CC:FUNCTION`, `FT:BINDING`, `DR:Pfam`, `DE` and so on. This is the database-record counterpart of `reference_section_type` (which is for manuscript sections such as RESULTS) and makes questions like "which UniProt cautions does this review use?" answerable by query. Do not write it by hand: run `just backfill-record-sections genes//` after editing, which derives it from the cited file and changes nothing else. Validation warns (`record_section_mismatch`) when a stated value disagrees with the cited file. `reference_review` is for what those checks *cannot* see — chiefly whether an internally-consistent citation actually **supports** the claim, or whether a well-formed id+title points to a paper that is simply the wrong choice for this gene. ```yaml references: - id: PMID:24268103 title: "The KCTD family of proteins: structure, function, disease relevance" reference_review: relevance: HIGH # how relevant the reference is to THIS gene's function/review correctness: VERIFIED # citation correctness + scientific soundness (most salient issue) review_notes: "PubMed-verified; supports the family-level 'mostly unknown' framing" ``` - **`relevance`** (`ReferenceRelevanceEnum`): `HIGH` (directly establishes/strongly informs the gene's function) · `MEDIUM` (supporting/corroborating) · `LOW` (background/contextual or passing mention) · `NONE` (not relevant to this gene; candidate for removal). - **`correctness`** (`ReferenceCorrectnessEnum`, single-valued — record the most salient issue, detail in `review_notes`): `VERIFIED` (identifier resolves to the intended, supporting paper, no soundness concerns) · `UNVERIFIED` (not yet checked) · `WRONG_IDENTIFIER` (resolves to a different paper than intended) · `MISCITED` (right paper, but it does not support the claim) · `DISPUTED` (correctly cited but the claim is contested) · `LOW_QUALITY` (correctly cited but methodologically weak/preliminary). - **`review_notes`**: free text explaining the judgment (e.g. what you verified, or why it is wrong). This complements (does not replace) the existing `is_invalid` (retracted/replaced) and `full_text_unavailable` flags. All fields are optional; an absent `reference_review` simply means the reference has not been manually adjudicated. **Verify, don't trust**: confirm a PMID against PubMed (or anchor a claim to a checkable fact such as the GOA evidence code) before marking it `VERIFIED` — an LLM-generated deep-research summary asserting a citation is not sufficient. **Deleted duplicate PMIDs and conflicting findings:** Keep GOA's `original_reference_id`. Record a verified canonical identifier in `reference_review.replacement` with `reference_id`, `reason`, and verification notes; fetch failure alone does not establish a remapping or retraction. Quote the canonical paper using its own `reference_id` in `review.supported_by`. For a statement from P1 contradicted by P2, use P1's `findings[].finding_review` with `finding_status`, `superseded_by`, and `supported_by` containing P2's exact snippet. See [Reference Curation](docs/reference_curation.md) for examples and validation rules. **Errors that live in the source record** (an annotation or GO-CAM activity on the wrong gene, a mis-keyed reference, a wrong evidence code) go in `review.source_defects` (or `activity_reviews[].source_defects` for GO-CAMs), with `asserted_value` → `corrected_value`, the responsible `resource` and a `confidence`. That is how "that gene, not this gene" is recorded; `just export-source-defects` turns them into per-resource upstream reports. See `docs/source_defects.md`. ## Tools Use the OLS MCP to find relevant ontology terms, if the terms you need are not in existing_annotations. ## Curation guidelines Avoid the term `protein binding`, this doesn't tell us anything about the actual function. Instead find a more informative MF term (e.g for adapter function) For the ClinGen Mendelian project, follow its standing [curation instructions](projects/CLINGEN_MENDELIAN.md#curation-instructions): supported generic binding may remain `KEEP_AS_NON_CORE` when no evidence-backed specific function is available. Inaccessible or unadjudicated evidence remains `UNDECIDED`. The generic validator still emits `protein_binding_policy` warnings for these non-core binding rows; retain and report the resulting computed status. This project-specific instruction does not change validation. ### A curation gap is not a knowledge gap `knowledge_gaps` (on gene reviews, core functions and modules) are for things **nobody knows**: an unknown activity, substrate, partner, mechanism or role that could only be resolved by **new wet-lab experiments**. A good knowledge gap reads like the motivation for an experiment, and usually pairs with a `suggested_experiments` entry. Work that is merely **not done yet** is not a knowledge gap, even though the schema's `KnowledgeGapKindEnum` offers `CURATION` and `ONTOLOGY`: - **Curation gaps** — a member gene not yet reviewed, an annotation that exists in the literature but not in GOA, a module part not yet modelled. Record these in the project page's plan, a module's `notes`, or the gene's notes file. - **Ontology gaps** — a missing or ill-fitting GO term. Record these as `proposed_new_terms`, in `notes`, or as a `suggested_questions` entry. Do not create a `knowledge_gaps` entry whose only `gap_kind` is `CURATION` and/or `ONTOLOGY`. Those values may appear only alongside `BIOLOGY`, on a gap whose core is a genuine biological unknown. Test before writing one: *would a wet-lab experiment close this gap?* If the answer is "no, a curator could close it by reading or annotating", it belongs in the plan or notes instead. ## Isoform and Negation Tracking The system tracks isoform-specific GO annotations and NOT (negated) annotations: ```yaml existing_annotations: # Isoform-specific annotation - term: id: GO:0045892 label: negative regulation of DNA-templated transcription evidence_type: IDA original_reference_id: PMID:9815658 isoform: P19544-1 # UniProt isoform ID - only present for isoform-specific annotations review: summary: ... # NOT annotation (negated) - term: id: GO:0045893 label: positive regulation of DNA-templated transcription evidence_type: IDA original_reference_id: PMID:9815658 negated: true # This is a NOT annotation isoform: P19544-1 review: summary: WT1 does NOT activate transcription... ``` - **`isoform`**: UniProt isoform ID (e.g., `P19544-1`) - indicates what isoform was tested - **`negated`**: Set to `true` for NOT annotations (e.g., `NOT|involved_in` qualifier in GOA) **Important**: Just because an annotation was made on a specific isoform doesn't mean the function is isoform-specific. The `isoform` field indicates what was tested, not necessarily what is unique to that isoform. To backfill isoform info on existing reviews: ```bash just backfill-isoforms human WT1 just backfill-isoforms-organism human # all genes in organism ``` See `docs/isoform_tracking.md` for full documentation and `projects/ISOFORMS.md` for genes with notable isoform-specific functions. ### When UniProt's isoforms and the literature disagree Two kinds of mismatch are recorded in structured form, not prose: - **An isoform the literature describes is missing from the reviewed entry** (SYNGAP1: the alpha1 PDZ-ligand C-terminus is not among Q96PV0's isoforms). Add a `functional_isoforms` entry, map it to the sequences that do carry it (`REFSEQ_PROTEIN`, `UNIPROTKB_UNREVIEWED`), and record a `source_defects` item with `defect_type: MISSING_ISOFORM`, `resource: UNIPROT`, `requested_change: ADD`. `just export-source-defects` includes it in the report for UniProt. - **A paper numbers residues on a non-displayed isoform** (RASAL2 "S351" is Ser351 of Q9UJF2-2 and Ser203 of the displayed Q9UJF2, where 351 is Asp). UniProt is not wrong here, so this is not a source defect. Add `residue_mappings` to the finding: `as_reported` is the residue in the numbering the paper used, `canonical` the same residue in the review's own displayed sequence (the CURIE without an isoform suffix, `UniProtKB:Q9UJF2`, never `-1`: the displayed isoform is not always isoform 1). Pin `sequence_version` on both sides, as for any `ResiduePosition`. Each side is checked against the real sequence (`python -m ai_gene_review.validation.finding_residue_mappings`, run by `just validate-families`), so a number read on the wrong sequence fails; a side that is not a UniProtKB sequence (a RefSeq-only isoform) is reported unresolved rather than checked. Never state a residue number in a finding without knowing which sequence it refers to. ### Alternative-ORF peptides: one folder per UniProt accession Isoforms and polyprotein cleavage products share the host's UniProt accession, so they stay in the host folder: use `isoform:` on annotations and `functional_isoforms` (`SPLICE_VARIANT`, or `CLEAVAGE_PRODUCT` mapped to `PRO_` chains, as in `POMC`). A peptide from an alternative ORF, a uORF or an overlapping frame is a **separate UniProt entry** with its own GOA rows. UniProt nevertheless often files it under the host gene's symbol (for example, L0R8F8 AltMIEF1 is gene `MIEF1`). Such a peptide gets its own folder: - It has its own HGNC symbol (`ASDURF`, `MLDHR`, `NBDY`): use the symbol as usual. - It shares the host symbol: use `genes//__/` with `id: ` and `gene_symbol: `, fetched with `just fetch-gene human --alias __` (the stub then takes `gene_symbol` from UniProt). The double underscore follows the `genes/PSEPK/aroE__Q88K85` paralog precedent and never collides with hyphenated symbols. - It has no gene symbol at all: use the accession as the folder name. Never model such a peptide as a `functional_isoform` of its host. See `projects/MICROPROTEINS.md`. ### Gene model problems When the entry's sequence is not a single real gene product (two neighbouring genes fused into one model, a split or truncated model, a wrong start site), record it in the top-level `gene_model_issues` list rather than only in prose: `issue_type`, `status` (`SUSPECTED` or `CONFIRMED`), a `description` of how it affects the entry's name and annotations, optional `components` and `reviewed_segment` for fused models, `evidence_types`, and `supported_by` quotes. A genuine biological fusion is not a gene model issue, and an entry UniProt already flags as a fragment (`DE Flags: Fragment`) needs an entry only when the review finds more than the flag records. See `projects/GENE_MODEL_ERRORS.md`. ## Bioinformatics analyses In some cases, it may be useful to do additional bioinformatics analyses. To validate gene function. Here are some guidelines: For a gene-specific analysis, the folder is `-bioinformatics/`. You should make this be reproducible. You can do a `uv init` here and `uv add` for any packages you need. Never hardcode the results of the script, it's OK for a script to return inconclusive results. Never ever make up results. **IMPORTANT**: DO NOT write scripts that just tell users to go to a website. If you cannot programmatically call a tool (e.g., requires API keys, browser interaction, or authentication you don't have), then: 1. Be honest about this limitation upfront 2. Write scripts that can parse/process the output from the tool if the user runs it manually 3. Never write misleading code that pretends to do something it doesn't actually do You can summarize the results of the bioinformatics analyses in the `-bioinformatics/RESULTS.md` file. This can be referenced as any other reference as e.g `file://bioinformatics/RESULTS.md`. ## Reviewing Computational Predictions The `GENE-predictions-review.yaml` file reviews computational/ML predictions that are NOT in the curated GOA/UniProt annotations. This is separate from `existing_annotations` in the main review file. Use the `PredictionReview` class (validated with `-C PredictionReview`): ```bash uv run linkml-validate -s src/ai_gene_review/schema/gene_review.yaml -C PredictionReview genes/ECOLI/yciO/yciO-det-predictions-review.yaml ``` ### Prediction sources Track predictions from: DeepECTF, PANTHER/IBA, InterPro2GO, CLEAN, GloEC, MAPred, or any other computational method that produces GO or EC predictions. ### Assessment categories (from de Crécy-Lagard et al. 2025, PMID:40703034) | Code | CS | Description | |------|-----|-------------| | COR | 2 | Correct novel prediction | | CNN | 2 | Correct but Not Novel (already in training data) | | LSP | 2 | Less Precise than existing annotation | | UNC | 1 | Uncertain - cannot validate or refute | | PLI | 0 | Paralog Incorrect - wrong paralog subfamily | | NPI | 0 | Nonparalog Incorrect - refuted by evidence | | REP | 0 | Repetition - frequency-biased prediction | ### Error types - `PARALOG_OVERANNOTATION` - Wrong paralog subfamily (Type 6) - `PATHWAY_CONTEXT_IGNORED` - Predicted pathway absent from organism - `FREQUENCY_BIAS` - Model defaults to high-frequency training labels - `IN_VITRO_NOT_IN_VIVO` - In vitro activity doesn't match biological function - `TRAINING_DATA_CONTAMINATION` - Prediction already in training data - `PSEUDOENZYME_OVERANNOTATION` - Ancestral catalytic activity assigned to a fold-retaining pseudoenzyme (lost catalytic residues) - `LOCALIZATION_DEFAULT` - Defaults to cytosol/cytoplasm when no TM/signal features, mislocalizing secreted/organellar/membrane proteins - `TAXON_CONSTRAINT_VIOLATION` - Term valid only in another lineage/kingdom (e.g. animal terms for a plant protein) - `WRONG_INPUT_SEQUENCE` - Pipeline fed the wrong protein sequence (data error, not model error) - `DOMAIN_ARCHITECTURE_MISMATCH` - Predicted activity needs a domain or catalytic region the selected protein lacks - `COMPLEX_ACTIVITY_TRANSFER` - Catalytic activity of a complex assigned to a noncatalytic accessory subunit - See schema for full list ## Page rendering and deployment With `PAGES_ARTIFACT_DEPLOY_ENABLED=true`, the site is built from `main` and deployed as an Actions artifact via GitHub Pages to https://ai4curation.io/ai-gene-review/. In this mode, generated files in Git are not the live site. The Pages source setting must separately be `GitHub Actions`. ### Gene review HTML ```bash just render # render one gene just render-all # render all genes ``` Output: `genes///-ai-review.html` This HTML is gitignored and must not be committed (not even with `git add -f`): the Pages workflow renders every review from `main`, and CI rejects a PR that adds one. Use `just render` only to preview a page locally. Older copies are still tracked while they are untracked in batches (#6221); do not stage re-rendered versions of them. `deploy-browser` and the prediction evals only link to a review page when its HTML exists on disk, so run `just render-all` first if you need those links in a local build. The propagation browser links whenever the review YAML exists. ### Project pages Project docs live in `projects/`. Render with: ```bash just render-projects # render all ai-gene-review render-projects projects/FOO.md -o pages/projects # render one ``` Output: `pages/projects/.html` **Folder structure convention.** Each project is a top-level markdown file `projects/FOO.md`, optionally accompanied by a same-named folder `projects/FOO/` holding its supporting material (sub-pages, data, scripts, reports): ``` projects/ FOO.md # the project page FOO/ # supporting material for FOO (optional) FOO-pathway.md # sub-pages, notes, data, scripts, ... ... ``` Put a project's pathway summaries, curation-recommendation docs, sub-analyses, data files, and scripts inside `FOO/` rather than as loose `FOO-*.md` siblings at the top level. `render-projects` renders the whole `projects/` tree recursively and **mirrors the subfolder structure** into `pages/projects/` (so `FOO/bar.md` renders to `pages/projects/FOO/bar.html`). Use ordinary relative links between a project page and its sub-pages (`[x](FOO/bar.md)` from `FOO.md`; `[parent](../FOO.md)` from inside `FOO/`); the renderer rewrites `.md`→`.html` and preserves the path. **Important:** The project index page (`pages/projects/index.html`) is **manually maintained**. When adding a new project, you must manually add a `
` entry to the index HTML. The `render-projects` command does NOT update the index. **Collections.** Related projects are grouped under an index page by listing a collection key in frontmatter, e.g. `collections: [HOMOLOGY_PROPAGATION]`. Keys are registered in `projects/collections.yaml` (title + index page slug); the index page gets an auto-generated member table, members get a link back to it, and the all-projects table gains a Collection filter. Current collections: `FUNCTION_PREDICTION` (index `FUNCTION_PREDICTION_EVALUATION`) and `HOMOLOGY_PROPAGATION` (index `HOMOLOGY_PROPAGATION`, with the propagation browser at `app/propagation/`). **Manual reviews.** A project page may record reviewer sign-offs in frontmatter under `manual_reviews` (a list). Each entry needs a `reviewed_by`; `status` (if given) must be `READY` or `CHANGES_REQUESTED`; `date` is `YYYY-MM-DD`; `notes` is free text and `todos` is a list. Unknown keys are rejected by validation. ```yaml manual_reviews: - reviewed_by: cjm date: 2024-06-01 status: CHANGES_REQUESTED notes: needs stronger evidence for the X claim todos: - cite PMID:12345 - drop the speculative pathway diagram ``` Reviews render as a block on the project page, and the **latest** review's status (most recent `date`) surfaces as a filterable "Review" column in the all-projects table. **Manifest (slides, briefs).** A project page lists its companion resources in frontmatter under `manifest`, a mapping of typed lists. Allowed lists are `slides` and `artifacts` (defined once in `MANIFEST_KINDS` in `src/ai_gene_review/render_projects.py`; adding e.g. `data` is one entry there). Each entry needs `href` and may carry `title` and `description`; unknown keys are rejected. A `slides` href is either an `https://` URL or a path **relative to `projects/`** to a rendered deck `.html` whose Marp `.md` source sits beside it (e.g. `FOO/slides/FOO-slides.html`); an `artifacts` href must be `https://`. ```yaml manifest: slides: - href: UNFOLDED_PROTEIN_BINDING/slides/UPB-slides.html title: Project deck # optional; default label "Slides" description: AI generated # optional; shown on the pill and as its tooltip artifacts: - href: https://claude.ai/artifact/XXXX title: Project brief # optional; default label "Brief" ``` Entries render as a pill bar directly under the page title (a `description` appears as small text after the pill label, e.g. "Slides" then "AI generated"; artifacts open in a new tab) and as Slides/Brief columns in the all-projects table. Machine-made decks carry `description: AI generated`. The same `validate_manifest()` backs the pytest check and the renderer: at render time an invalid entry is left out and reported as a page warning, so one bad page never stops the site render. Manifest-linked decks are published with their images, so do **not** also add an in-body `## Slides` section with the deck link. `scripts/populate_project_manifest.py` fills `manifest` from the deck folders plus a `stemurl[title]` brief list, editing only the `manifest` block of the frontmatter. **Gene-symbol auto-linking.** Project pages auto-link prose gene symbols to their review pages — never hardcode `genes/...` URLs. Linking is convention + metadata driven: - **Bare symbols** (`GPX4`, `CASPL4C1`) link automatically when the symbol maps to exactly one species directory. When a symbol exists in several species, list the project's species in frontmatter; `species:` is treated as a **priority-ordered** preference, so the first listed species that has the review wins (this also fills the all-projects table). Add `genes: [...]` to populate the table's gene count. - **`CODE/symbol`** (e.g. `POPTR/CASPL4C1`, `human/TP53`) is the inline convention for explicit, per-mention disambiguation in genuinely multi-species prose. `CODE` is a 5-character uppercase UniProt mnemonic (e.g. `ARATH`, `9INFA`) or one of the lowercase model-organism dirs (`human`, `mouse`, `rat`, `worm`, `yeast`). It links only when `genes/CODE/symbol/` exists; a known species with a missing symbol warns. - **`label`** tags are for cases where the visible text differs from the symbol (e.g. `E/P00720`). - Set `autolink_gene_symbols: false` in frontmatter for paper-like pages where prose symbols should not become links. - List symbols that are ordinary words on the page under `autolink_exclude: [...]` (e.g. `RCA`, the evidence code, vs the Arabidopsis gene *RCA*); the rest still link. ### Browser app ```bash just deploy-browser # update data.js + index.html for the interactive browser just deploy-browser pages # disposable artifact build, without Git's blob cap ``` Output: `app/` ### CI automation The `generate-pages` workflow runs daily at 08:23 UTC, with manual runs available through GitHub Actions. With `PAGES_ARTIFACT_DEPLOY_ENABLED=true`, it renders, stages, compresses, checks and uploads the site, then deploys the artifact. It skips generated-file commits and PRs entirely; publication needs no App token, review or merge. Disabling that variable retains the legacy regeneration PR code path, but it cannot build a browser payload at or above Git's 100 MiB blob limit. The current corpus already exceeds that limit: flag-off is not a working rollback. It also does not change the repository's Pages source setting. Its publication schedule is exempt from agent cron profiles. Gene reviews are validated in PR CI and by the weekly full validation workflow. `just build-pages` builds the disposable artifact locally. All three browser builders accept the `pages` target (or `BROWSER_TARGET=pages`); their default `git` target retains GitHub's 100 MiB Git blob limit. The artifact target has no per-file Git limit: staging compresses the main annotation browser data and checks total site and tar sizes. Prediction and propagation browser payloads are currently staged uncompressed; these checks do not guarantee browser memory or load-time performance. Never commit generated output just because the artifact build passed. GitHub Pages officially supports a 1 GB site. Our existing temporary policy allows larger deployments below the 10 GB absolute artifact cutoff; that is not a hosting-capacity guarantee. A completed upload can be redeployed without rendering via `deploy-existing-pages.yaml`, while the artifact is retained. ## General guidelines * NEVER guess identifiers for terms, genes, publications. Always use the relevant tools or MCPS, or look them up in derived files. * Use YAML, not TSV, for any structured data file you author (project tables, proposal lists, curated records). TSVs produced by deterministic pipelines (e.g. `GENE-goa.tsv`, PAINT `*-paint.tsv`) are inputs and stay as they are. * For files `-notes.md`, use literature deep search, and always record provenance for assertions, e.g `[PMID:12345 ""]` ## Support code There is also support code in `src/ai_gene_review`, see the CLAUDE.md file in that directory for more details on best practices for working with the code.