Galaxy Workflow Invocation Failure Reference

This note describes workflow-level failure surfaces in Galaxy. It is separate from galaxy-tool-job-failure-reference because invocation state answers whether Galaxy could schedule and drive the workflow, while job state answers whether individual tool jobs succeeded.

Invocation Versus Job Failure

Important distinction:

Invocation state says whether Galaxy scheduled, cancelled, failed, or completed the workflow invocation.
Job state says whether jobs produced by invocation steps succeeded or failed.
Invocation messages explain scheduler/evaluation/cancellation problems.
Step states usually describe scheduling progress, not actual job success, unless a legacy serialization mode substitutes job state.

A robust workflow test reference should inspect both invocation APIs and job APIs.

Invocation States

Galaxy invocation states are defined in ~/projects/repositories/galaxy/lib/galaxy/schema/invocation.py and related model code.

State	Meaning
`new`	Newly created invocation.
`requires_materialization`	Deferred or undeferred inputs must materialize before scheduling can continue.
`ready`	Ready for another scheduler iteration.
`scheduled`	Workflow has been scheduled. Older clients may treat this as successful terminal state.
`cancelling`	Scheduler will cancel jobs and subworkflow invocations.
`cancelled`	Cancellation terminal state.
`failed`	Scheduler/evaluation/materialization failure terminal state.
`completed`	Newer completion state after all jobs reach terminal states.

Version caveat: completed may not be uniformly available across all Galaxy/Planemo combinations. Planemo accounts for both scheduled and completed as success-like invocation states.

Invocation Message Reasons

Structured invocation messages are the main durable surface for workflow-level failures.

Reason	Typical meaning
`dataset_failed`	Dataset input or upstream dataset was failed/unusable.
`collection_failed`	Collection was failed, incompletely populated, or unusable.
`job_failed`	Upstream dependent job failed.
`output_not_found`	Expected step/subworkflow output could not be resolved.
`expression_evaluation_failed`	Workflow expression or condition evaluation failed.
`when_not_boolean`	Conditional `when` expression did not return a boolean.
`unexpected_failure`	Scheduler or evaluation failure without a more specific safe reason.
`workflow_parameter_invalid`	Workflow parameter validation failed.
`step_input_deleted`	Step input dataset/collection was deleted.
`history_deleted`	Invocation history was deleted, causing cancellation.
`user_request`	User/API cancellation.
`cancelled_on_review`	Pause/review step rejected continuation.
`workflow_output_not_found`	Warning: declared workflow output was not found.

Message records can include affected workflow step id, dependent step id, job id, HDA/HDCA ids, output name, and nested workflow step index path. These fields matter more than free-text messages for reference-quality diagnosis.

Common Workflow-Level Failure Surfaces

Surface	Reference signal
Request-time validation	API returns an error before useful invocation state exists.
Input materialization	Invocation enters `requires_materialization`, then may fail with `dataset_failed`.
Conditional evaluation	Invocation messages include `expression_evaluation_failed` or `when_not_boolean`.
Missing output graph edge	Invocation messages include `output_not_found`.
Deleted/purged inputs	Invocation messages include `step_input_deleted` or `dataset_failed`.
Upstream job failure	Invocation messages or summaries point to `job_failed`; job API has details.
Collection population failure	Invocation messages include `collection_failed`; inspect HDCA and mapped jobs.
Pause/review rejection	Cancellation reason `cancelled_on_review`.
User/history cancellation	Cancellation messages `user_request` or `history_deleted`.
Warning-only missing output	`workflow_output_not_found` warning; may later become Planemo assertion failure.

API Surfaces

Useful invocation APIs from ~/projects/repositories/galaxy/lib/galaxy/webapps/galaxy/api/workflows.py:

API	Use
`GET /api/invocations/{invocation_id}`	Invocation state, messages, inputs, outputs, and steps.
`GET /api/invocations/{invocation_id}?step_details=true`	Richer step detail, jobs, outputs, and output collections.
`GET /api/invocations/steps/{step_id}`	Detail for one invocation step.
`GET /api/invocations/{invocation_id}/steps/{step_id}`	Step detail scoped to invocation.
`GET /api/invocations/{invocation_id}/jobs_summary`	Job state summary across the invocation.
`GET /api/invocations/{invocation_id}/step_jobs_summary`	Job state summary per workflow invocation step.
`GET /api/invocations/{invocation_id}/completion`	Completion record, if available.
`DELETE /api/invocations/{invocation_id}`	Cancel invocation.
`PUT /api/invocations/{invocation_id}/steps/{step_id}`	Continue or cancel pause/review step.
`GET /api/invocations/{invocation_id}/report`	Invocation report surface.

Workflow aliases under /api/workflows/{workflow_id}/invocations/... also exist.

Planemo Caveats

Planemo polls invocation and job state, but its user-facing output may not expose all structured invocation messages. Treat Planemo terminal output as a summary; use raw invocation and job APIs when the failure class matters.

Likely noisy boundaries:

Planemo may show failed job details but omit structured invocation messages.
step_jobs_summary is useful in Galaxy but not necessarily surfaced directly by Planemo output.
Subworkflow paths can be represented in Galaxy message fields but may not be displayed by Planemo.
A workflow can be completed or scheduled while one or more jobs failed; job summaries must still be inspected.

Durable Reference Use

Use this note when a workflow run fails before or around job scheduling, when outputs are missing without an obvious tool stderr cause, or when Planemo only reports a generic invocation failure.

The goal is not to prescribe a single repair loop. The goal is to preserve which Galaxy API surface proves the failure mode so later Molds can decide whether the defect belongs to data-flow, template, concrete step implementation, workflow test assertions, or runtime environment.

Verification Gaps

Actual runs should verify:

Whether Planemo surfaces invocation messages for each reason.
Whether completed and /completion are populated in the target Galaxy used by tests.
How mapped collections and subworkflow failures appear in step_jobs_summary.
How warning-only workflow_output_not_found interacts with Planemo assertions.

Galaxy workflow invocation failure reference