Galaxy Brain

TEST_JOB_VALIDATION_TS_FOLLOWUP_PLAN

Cross-check Plan (inputs/outputs match workflow)

Follow-up to TEST_SCHEMA_TS_PLAN.md. Goal: land data-level cross-check in the current wf_test_schema PR; leave galaxy-workflows-vscode with the smallest possible shim to adopt later.

Shared surface in @galaxy-tool-util/schema/test-format/

Expose canonical DTOs + pure extractors + comparator. No AST, no ranges (plugin keeps those).

export interface WorkflowInput  { name, doc?, type, default?, optional? }
export interface WorkflowOutput { name, doc?, uuid? }

extractWorkflowInputs(parsed, format: "native"|"format2"): WorkflowInput[]
extractWorkflowOutputs(parsed, format: "native"|"format2"): WorkflowOutput[]

isCompatibleType(declaredType, valueType): boolean   // port from plugin utils

checkTestsAgainstWorkflow(
  testsDoc: unknown,
  workflow: { inputs: WorkflowInput[]; outputs: WorkflowOutput[] },
): TestFormatDiagnostic[]   // json-pointer paths like /0/job/foo

Diagnostics reuse existing TestFormatDiagnostic shape. New keyword values: workflow_input_undefined, workflow_input_required, workflow_input_type, workflow_output_undefined.

Notes from implementation review:

Work items

  1. Extractor ports. extract-native.ts, extract-format2.ts. Mirror plugin’s NativeWorkflowDocument.getWorkflowInputs/Outputs and GxFormat2WorkflowDocument AST walks, but operate on already-parsed JSON/YAML dicts. Native reads tool_state (string-encoded JSON) for default/optional — reuse existing schema-package utilities rather than re-parsing.

  2. Reuse resolveFormat. Already lives in @galaxy-tool-util/schema (workflow/serialize.ts), CLI just re-exports. Cross-check imports it directly; no new format-detection helper.

  3. isCompatibleType port — small pure function, mirror plugin tests.

  4. checkTestsAgainstWorkflow — walks each test entry’s job + outputs, emits diagnostics with json-pointer paths (same shape Ajv uses).

  5. CLI flags. validate-tests gets --workflow <path> for single-file pairing. validate-tests-tree gets --auto-workflow only (opt-in convention-based sibling discovery: foo.gxwf-tests.yml ↔ foo.gxwf.yml/foo.ga, bar-tests.yml ↔ bar.yml/bar.ga; silent no-op when no sibling found). --workflow on tree doesn’t fit — tree walks many files.

  6. Fixtures. Under packages/schema/test/fixtures/test-format/ add workflows/ (small native + format2 pair) and cross-check/:

    • positive/match_inputs_outputs-tests.yml + the workflow pair
    • negative/input_not_in_workflow-tests.yml, missing_required_input-tests.yml, input_type_mismatch-tests.yml, output_not_in_workflow-tests.yml Layout that plugin tests can vendor directly.

  7. Tests. Schema unit tests: extractors over native+format2 fixtures; integration test asserts each negative fixture yields the expected diagnostic keyword. CLI test: validate-tests --workflow snapshot.

  8. Export surface. Re-export WorkflowInput, WorkflowOutput, extractWorkflowInputs, extractWorkflowOutputs, isCompatibleType, checkTestsAgainstWorkflow from @galaxy-tool-util/schema root. Keep internals under test-format/ for cohesion.

  9. Changeset. Minor on @galaxy-tool-util/schema (new public API) + note on @galaxy-tool-util/cli.

What this buys the VS Code plugin (separate follow-up PR, out of scope)

Plugin refactor reduces to three drop-ins:

Net effect: two extractors + three rule methods collapse to thin adapters; test-document AST walking stays plugin-local (it’s what the plugin is for).

Risks / boundaries

Test strategy (red-to-green)

  1. Land DTOs + empty extractors + failing extractor tests against native and format2 fixtures.
  2. Fill in format2 extractor (simpler); tests green.
  3. Fill in native extractor (incl. tool_state decode); tests green.
  4. Add isCompatibleType + its unit tests (red-to-green).
  5. Add checkTestsAgainstWorkflow + negative-fixture tests.
  6. Wire CLI --workflow flag + snapshot test.
  7. Wire --auto-workflow in tree command + discovery test.

Unresolved questions