Home Pattern

Fan-in bundle, consume, and flatten

Bundle parallel outputs into a collection consumer, then flatten nested results for pooled downstream processing.

Revised
2026-05-04
Rev
1

Pattern health

warn
  • IWC exemplar anchors

    1 abstract workflow anchor declared.

  • Foundry verification fixture

    No structural verification fixture yet.

  • Pattern map coverage

    1 pattern map link here.

  • Metadata contract

    Pattern frontmatter matches the site contract.

Fan-in bundle, consume, and flatten

Use this recipe when several parallel outputs should become one named collection for a downstream collection-aware tool, then nested results must be flattened before optional pooled aggregation.

This contrasts with manifest-to-mapped-collection-lifecycle: collection construction happens mid-workflow as fan-in, not just at the workflow boundary.

Recipe

  1. Identify parallel outputs that represent comparable alternatives, bins, reports, or method results.
  2. Build a named collection bundle from those individual outputs.
  3. Feed the bundle into the downstream tool that expects a collection input.
  4. Flatten nested collection output if the downstream tool emits one subcollection per input element.
  5. Aggregate tabular/report outputs when the final consumer needs one file rather than a collection.

Reach For This When

  • Multiple tools produce equivalent artifacts and a later tool compares, refines, or pools them.
  • A workflow bundles related outputs because a later collection-aware tool consumes them, not merely for publication.
  • A downstream collection consumer emits nested outputs that need pooled processing.

Operation Handoffs

Bundle-only publication is related but narrower; use collection-build-named-bundle when there is no downstream collection consumer.

Pitfalls

  • Do not use collection bundling when the real operation is file-content concatenation.
  • Do not use positional identifiers for bundles that will become test-facing outputs.
  • Do not flatten away an axis that still encodes method, sample, or replicate identity needed downstream.

See Also

IWC exemplars1 anchor

IWC Exemplars

microbiome/mags-building/MAGs-generationhigh

Builds a collection from multiple binner outputs for binette, then flattens nested bins for pooled downstream processing.

Incoming References (4)

  • Galaxy: collection patternsrelated pattern— Use this MOC to choose corpus-grounded Galaxy collection transformation patterns.
  • Iwc Conditionals Surveyrelated note— Corpus survey of Galaxy conditional step usage in IWC, covering when-gates, boolean shims, and routed output selection.
  • Iwc Map Over Lifecycle Surveyrelated note— Survey of IWC map-over lifecycle recipes, with a Nextflow-to-Galaxy crosswalk for collection construction, cleanup, reshape, reduce, and publish phases.
  • Nextflow: mix and collect to report aggregationimplemented_by_patterns— Route Nextflow mix, collect, toList, and collectFile report fan-in idioms to Galaxy aggregation and bundle patterns.