Plan: Complete paired_or_unpaired Subtyping at Any Rank
Problem Statement
paired is treated as a subtype of paired_or_unpaired only at the deepest (innermost) rank:
list:paired->list:paired_or_unpaired— WORKS todaypaired:list->paired_or_unpaired:list— BROKEN, should worklist:paired:list->list:paired_or_unpaired:list— BROKEN, should work
Affected Code: 3 Parallel Implementations
Layer A: TypeScript (collectionTypeDescription.ts)
Three methods all use endsWith(":paired_or_unpaired") — never checking middle positions:
canMatch(lines 87-109) — determines if output type can satisfy input typecanMapOver(lines 110-146) — determines if collection can be mapped over inputeffectiveMapOver(lines 147-194) — computes effective map-over type after consuming input
Layer B: Python (type_description.py)
can_match_type(lines 106-124) — Python equivalent ofcanMatch, sameendswithlimitationhas_subcollections_of_type(lines 76-99) — only handlespaired_or_unpairedas entire subcollection type
Layer C: Python (subcollections.py)
_is_a_subcollection_type(lines 28-34) — usesendswithcheck_split_dataset_collection(line 52) — uses exact equalitychild_collection.collection_type == collection_type
Layer D: Python (basic.py line 2668)
Tool parameter presentation only handles terminal paired_or_unpaired.
Core Algorithm: Rank-by-Rank Comparison
Subtyping should be applied rank-by-rank. New helper function (needed in both TS and Python):
canMatchRankByRank(thisType, otherType):
thisRanks = thisType.split(":")
otherRanks = otherType.split(":")
if len(thisRanks) != len(otherRanks): return false
for (thisRank, otherRank) in zip(thisRanks, otherRanks):
if thisRank == otherRank: continue
if thisRank == "paired_or_unpaired" and otherRank == "paired": continue
return false
return true
Same comparison needed for suffix matching during map-over calculations.
Detailed Changes
Step 1: TS collectionTypeDescription.ts — Add _canMatchRanks helper
Step 2: TS canMatch — Replace endsWith logic with rank-by-rank helper
Step 3: TS canMapOver — Update suffix matching to be subtype-aware
Step 4: TS effectiveMapOver — Correctly compute map-over for non-terminal positions
Step 5: Python type_description.py can_match_type — Same rank-by-rank logic
Step 6: Python type_description.py has_subcollections_of_type — Handle compound subcollection types
Step 7: Python type_description.py effective_collection_type — Fix string length subtraction
Step 8: Python subcollections.py _is_a_subcollection_type — Rank-by-rank suffix matching
Step 9: Python subcollections.py _split_dataset_collection — Subtype-aware element comparison
Step 10: Python basic.py — Handle non-terminal paired_or_unpaired in tool parameter presentation
Test Plan (Red-to-Green)
A: TS Unit Tests (terminals.test.ts)
Add to parameter_steps.json:
paired_or_unpaired:list collection inputpaired:list input
New test cases:
"accepts paired:list -> paired_or_unpaired:list connection"— direct match at non-terminal rank"rejects paired_or_unpaired:list -> paired:list connection"— inverse must NOT work"accepts list:paired:list -> paired_or_unpaired:list connection"— map-over with non-terminal subtyping"accepts list:paired:list -> list:paired_or_unpaired:list connection"— outer match + inner subtyping"accepts paired:paired -> paired_or_unpaired:paired connection"— first rank subtyping
B: Standalone collectionTypeDescription.test.ts
Focused unit tests:
canMatch("paired_or_unpaired:list", "paired:list")-> truecanMatch("paired:list", "paired_or_unpaired:list")-> falsecanMatch("paired_or_unpaired:paired", "paired:paired")-> truecanMatch("list:paired_or_unpaired:list", "list:paired:list")-> truecanMapOverandeffectiveMapOvertests for non-terminal positions
C: Python Unit Tests
can_match_type("paired:list")onCollectionTypeDescription("paired_or_unpaired:list")-> Truehas_subcollections_of_type("paired_or_unpaired:list")onCollectionTypeDescription("list:paired:list")-> True- Splitting tests for
_split_dataset_collection
D: API Integration Tests
In test_tool_execute.py:
- Tool with
paired_or_unpaired:listinput usingpaired:listcollection - Map-over:
list:paired:list->paired_or_unpaired:listinput
E: Spec Updates
Add examples to collection_semantics.yml, remove limitation disclaimer (lines 556-563).
Implementation Order
- Write failing tests (TypeScript + Python)
- Implement TypeScript changes (add helper, update 3 methods)
- Implement Python changes (add helper, update 5 locations)
- API integration tests (end-to-end)
- Update spec
Edge Cases
- Multiple
paired_or_unpairedranks:paired:pairedmatchingpaired_or_unpaired:paired_or_unpaired— rank-by-rank handles naturally single_datasetssemantics at non-terminal position: Shouldlist:listmatchpaired_or_unpaired:list? This is separate frompaired<:paired_or_unpaired— potentially defer- Runtime adapter wrapping:
PromoteCollectionElementToCollectionAdaptermay need to wrap at appropriate level during_split_dataset_collection - No existing tools use
paired_or_unpairedat non-terminal rank — test tool XML may be needed
Critical Files
| File | Change |
|---|---|
client/src/components/Workflow/Editor/modules/collectionTypeDescription.ts | Core TS matching: canMatch, canMapOver, effectiveMapOver |
lib/galaxy/model/dataset_collections/type_description.py | Core Python matching: can_match_type, has_subcollections_of_type, effective_collection_type |
lib/galaxy/model/dataset_collections/subcollections.py | Runtime splitting: _is_a_subcollection_type, _split_dataset_collection |
lib/galaxy/tools/parameters/basic.py | Tool parameter presentation (line 2668) |
client/src/components/Workflow/Editor/modules/terminals.test.ts | TS test cases |
lib/galaxy/model/dataset_collections/types/collection_semantics.yml | Update spec |
Unresolved Questions
- Should “single_datasets” semantics (
listmatchingpaired_or_unpaired) extend to non-terminal positions? Or defer? - Any existing Galaxy tools/workflows use
paired_or_unpairedat non-terminal rank? Need test tool XML? effective_collection_typeuses string slicing — when input type haspaired_or_unpairedbut actual collection haspaired, lengths differ. Compute from actual or input type string?- Coordinate with tool authors who might want
paired_or_unpaired:listinputs, or purely infrastructure?