Home Pattern

Collection: flatten after fan-out

Use FLATTEN to collapse nested collection outputs to a flat list once the outer axis no longer matters.

draft pattern

Revised: 2026-05-03
Rev: 2
Patterns: collection-build-named-bundle
harmonize-by-sortlist-from-identifiers
Molds: implement-galaxy-tool-step
Related: iwc-transformations-survey
galaxy-apply-rules-dsl

Pattern health

warn

IWC exemplar anchors
3 abstract workflow anchors declared.
Foundry verification fixture
No structural verification fixture yet.
Pattern map coverage
No pattern map links to this leaf yet.
Metadata contract
Pattern frontmatter matches the site contract.

Collection: flatten after fan-out

Tool

Use Galaxy built-in __FLATTEN__.

The corpus-attested operation is simple: collapse a nested collection such as list:list or list:paired into a flat list when the outer grouping axis has served its purpose.

When to reach for it

Use this after a tool fans out within each sample/group and downstream no longer needs that grouping.

Typical shape: a domain tool maps over a collection and produces nested outputs, the outer axis becomes only organizational, and __FLATTEN__ produces one flat list for pooling, relabeling, reporting, or MultiQC-style consumption.

Do not flatten if downstream needs to know which outer sample/group produced each element.

Parameters

tool_id: __FLATTEN__
tool_state:
  input: { __class__: ConnectedValue }

The useful authoring decision is input collection type and whether the outer axis is truly no longer meaningful.

Pitfalls

Flattening discards structure. Element identifiers may retain hints, but the Galaxy collection type no longer encodes the outer axis.
Flatten only after the outer axis is done.
If flattened identifiers collide or become unreadable, add an explicit relabel step.
Do not use Apply Rules for plain flattening; the survey found __FLATTEN__ dominates simple cases.

See also

iwc-transformations-survey — Recipe H and candidate boundary.
collection-build-named-bundle — assemble named outputs into a collection.
harmonize-by-sortlist-from-identifiers — order/intersect sibling collections by identifier file.

IWC exemplars3 anchors

IWC Exemplars

microbiome/mags-building/MAGs-generationhigh

Flattens a list:list of bins from all samples for pool-level processing.

transcriptomics/rnaseq-pe/rnaseq-pehigh

Flattens a paired collection to a flat list for MultiQC.

microbiome/metagenomic-raw-reads-amr-analysis/metagenomic-raw-reads-amr-analysishigh

Flattens list:list output from sylph_profile before relabeling.

Incoming References (10)

Collection: build named bundlerelated pattern— Use BUILD_LIST to assemble named outputs into a collection bundle for publishing or downstream fan-in.
Fan-in bundle, consume, and flattenrelated pattern— Bundle parallel outputs into a collection consumer, then flatten nested results for pooled downstream processing.
Collection: harmonize by sortlist from identifiersrelated pattern— Use SORTLIST with sort_type:file to reorder one collection by another collection's identifiers.
Manifest to mapped collection lifecyclerelated pattern— Use a manifest or table to build a collection, map a tool per row, then relabel or reshape outputs.
Reshape, relabel, and remap by collection axisrelated pattern— Use Apply Rules and deterministic relabeling when domain fan-out creates the wrong map-over axis.
Galaxy Apply Rules DSLrelated note— Reference for Galaxy's Apply Rules DSL: rule operations, mapping operations, composition patterns, pitfalls.
Iwc Map Over Lifecycle Surveyrelated pattern— Survey of IWC map-over lifecycle recipes, with a Nextflow-to-Galaxy crosswalk for collection construction, cleanup, reshape, reduce, and publish phases.
Iwc Transformations Surveyrelated note— Corpus survey of collection-shape transformations across IWC: built-in collection ops, toolshed transformers, and the multi-step recipes that bracket map-over.
Nextflow: grouped channel to regrouped Galaxy collectionimplemented_by_patterns— Route Nextflow groupTuple, transpose, and grouped tuple payloads to Galaxy collection reshape patterns when the key is a real axis.
Nextflow: mix and collect to report aggregationimplemented_by_patterns— Route Nextflow mix, collect, toList, and collectFile report fan-in idioms to Galaxy aggregation and bundle patterns.