Galaxy Notebooks manuscript — improvement ideas

What this is: a prioritized list of ideas for improving the manuscript, generated by an Opus review subagent that read manuscript.md, manuscript-draft-2.md, the supporting paper files (HISTORY_MARKDOWN_ARCHITECTURE.md), and the three use-case debriefs. The paper itself was not modified. Line references are to manuscript.md.

The 5 highest-leverage changes

1. Re-pitch the thesis around the one surprising, falsifiable claim. The paper’s strongest, most defensible idea is “what a notebook displays is what seeds reuse” — display is not decoration, it is the provenance handle. UC1’s byte-identical regeneration loop and UC3’s off-graph→on-graph 0-tools→5-steps contrast are the evidence. Lead with this instead of the generic “notebooks + reproducibility” framing; it’s the claim a skeptic can try to break and the work survives.

2. De-memo the manuscript. Delete the “Evaluation Plan” section and every self-addressed “this should be regenerated / once the vignette is captured” hedge. These read as notes-to-self and undercut otherwise-strong, already-demonstrated evidence. The vignettes exist now — state them as implemented behavior with real numbers, not as plans.

3. Fix the unforced factual errors (each is individually citable by a hostile reviewer):

   • The content_editor claim contradicts the architecture doc (§6 correction) — fix to match how editing actually works.
   • The “byte-identical” passage contradicts itself elsewhere in the draft — reconcile to one consistent claim (UC1 is byte-identical; say it once, correctly).
   • Table 1 has column-category errors (rows placed under the wrong heading) — re-audit the table against the actual feature set.

4. Decide and disclose the agent-built provenance, then turn it into an evidence layer. The vignettes were constructed largely by an AI agent. Rather than hide or hand-wave this, make it the agent-authorship story: an agent could build, document, and extract these analyses because the surface is machine-legible. That converts a potential credibility liability into a distinctive contribution — but only if disclosed deliberately and consistently.

5. Replace promissory Availability with real handles. The Availability/Methods sections are currently all promises (“will be available”, “code is being prepared”). Use the actual PR (#22860), branch, commit IDs, and tool IDs/versions now captured in the recipes. A reproducibility paper whose own availability section is aspirational undercuts its thesis.

Biggest weakness / biggest strength

• Weakness: the unfinished, self-addressed state (eval plan, “should be regenerated”, empty vignette slots) undercuts genuinely strong evidence — the work is more done than the prose admits.
• Strength: the off-graph→on-graph mechanism plus UC1’s byte-identical regeneration loop is surprising, falsifiable, and already demonstrated — rare for a tools/reproducibility paper. Build the paper around it.

Full idea list by activity

Scientific story / framing

• Make “display seeds reuse” the spine; every section earns its place by supporting it.
• Position the three use cases as a deliberate triad: UC1 happy-path (clean 14-step, byte-identical), UC2 robustness/limits (the 2-way seam + the _original_hda fix), UC3 display-anti-pattern (0-tools→5-steps). Name the division of labor explicitly so none is over-claimed.
• State the negative result plainly: a fully tool-driven analysis can still seed nothing if it displays off-graph artifacts. The negative control is the most persuasive evidence the paper has.

Writing itself

• Cut self-referential hedging and future tense throughout; convert plans to results.
• One claim, one place: the byte-identical and reuse claims are currently stated inconsistently across sections.
• Tighten the abstract to the falsifiable thesis; it currently diffuses across several weaker claims.

Implementation / evidence & rigor

• Fold in the _original_hda collection-op closure fix (UC2) as maturation evidence (unit + Selenium tests) — shows the extraction machinery is robust and improvable, not just asserted.
• Lean on the no-core-change figure story for UC3: PDF figures stay on-graph by converting each to PNG with an in-graph tool (graphicsmagick_image_convert → __EXTRACT_DATASET__), so no pymupdf dependency or renderer change is added — a reproducibility paper that needs zero new core surface for figures is the stronger position.
• Use real numbers everywhere a placeholder sits: 14 steps / 0 dangling (UC1); 34 steps single vs 5+29 split (UC2); 13 steps / 6 outputs (UC3); 45,620 significant peaks etc.

Background / literature review

• Add the missing Electronic Lab Notebook related work (Jupyter, RMarkdown/Quarto, observable, ELN systems) and position against it: Galaxy Notebooks bind narrative to server-side provenance, not a re-executed kernel.
• Cite Galaxy’s own prior machinery honestly: Pages, Galaxy-Flavored Markdown, and the workflow-report system the notebook surface reuses. The “we reuse existing rendering” claim is stronger when the prior art is named, and weaker (looks like reinvention) when it isn’t.

Figures

• Add the UC3 before/after extraction figure (degenerate 2-input/0-tool vs full 5-step DAG of the same analysis) — the single most persuasive visual available.
• Add a polished rendered-notebook screenshot (UC1 heatmaps or UC3 PCA+volcano), captioned honestly re live-vs-baked rendering.
• Re-audit Table 1 categories (see fix #3).

Structure / reproducibility

• Replace the Evaluation Plan with an Evaluation/Results section grounded in the three captured vignettes.
• Add a Reproducibility/Availability section pointing at the recipes (tool IDs, versions, parameters, input provenance) and the real PR/commit handles.
• Cross-check internal cross-references after de-memoing (several sections reference the eval plan that should be removed/renamed).

Cross-references for consistency after edits

Abstract renderer-reuse claim (~line 5); Design Goals “reference artifacts, not just describe them” (~line 29); Extraction “references are not natural-language guesses” (~line 84); Test Coverage (~lines 132–136); Evaluation Plan (~line 144); Discussion limits (~line 176); Methods “reuse Galaxy markdown rendering utilities” (~line 182). These are the spots the three UC paper-integration proposals also touch — keep them mutually consistent.