Foundry
Home Schema

Galaxy workflow test format

JSON Schema for the planemo workflow test format (`<workflow>-tests.yml`), vendored from `@galaxy-tool-util/schema`.

draft schematarget/galaxy
Revised
2026-05-05
Rev
4
Related
schema tests-format @galaxy-foundry/foundry @ 0.0.0 upstream ↗

GalaxyWorkflowTests

Galaxy workflow tests file — a YAML list of test entries asserting the expected inputs and outputs of a workflow run.

110 definitions. Anchor links per definition (e.g. #has_text_model) are stable.

assertion_dict

assertion_dict
field type req description
attribute_is → base_attribute_is_model | null
attribute_matches → base_attribute_matches_model | null
element_text → base_element_text_model | null
element_text_is → base_element_text_is_model | null
element_text_matches → base_element_text_matches_model | null
has_archive_member → base_has_archive_member_model | null
has_element_with_path → base_has_element_with_path_model | null
has_h5_attribute → base_has_h5_attribute_model | null
has_h5_keys → base_has_h5_keys_model | null
has_image_center_of_mass → base_has_image_center_of_mass_model | null
has_image_channels → base_has_image_channels_model | null
has_image_depth → base_has_image_depth_model | null
has_image_frames → base_has_image_frames_model | null
has_image_height → base_has_image_height_model | null
has_image_mean_intensity → base_has_image_mean_intensity_model | null
has_image_mean_object_size → base_has_image_mean_object_size_model | null
has_image_n_labels → base_has_image_n_labels_model | null
has_image_width → base_has_image_width_model | null
has_json_property_with_text → base_has_json_property_with_text_model | null
has_json_property_with_value → base_has_json_property_with_value_model | null
has_line → base_has_line_model | null
has_line_matching → base_has_line_matching_model | null
has_n_columns → base_has_n_columns_model | null
has_n_elements_with_path → base_has_n_elements_with_path_model | null
has_n_lines → base_has_n_lines_model | null
has_size → base_has_size_model | null
has_text → base_has_text_model | null
has_text_matching → base_has_text_matching_model | null
is_valid_xml → base_is_valid_xml_model | null
not_has_text → base_not_has_text_model | null
xml_element → base_xml_element_model | null

assertion_list

assertion_list

Assert Attribute Is

attribute_is_model

Asserts the XML ``attribute`` for the element (or tag) with the specified XPath-like ``path`` is the specified ``text``. For example: ```xml <attribute_is path="outerElement/innerElement1" attribute="foo" text="bar" /> ``` The assertion implicitly also asserts that an element matching ``path`` exists. With ``negate`` the result of the assertion (on the equality) can be inverted (the implicit assertion on the existence of the path is not affected).

field type req description
that = "attribute_is" const "attribute_is"
attribute string ✓ The XML attribute name to test against from the target XML element.
path string ✓ The Python xpath-like expression to find the target element.
text string ✓ The expected attribute value to test against on the target XML element
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Attribute Is (Nested)

attribute_is_model_nested

Nested version of this assertion model.

field type req description
attribute_is → base_attribute_is_model ✓

Assert Attribute Matches

attribute_matches_model

Asserts the XML ``attribute`` for the element (or tag) with the specified XPath-like ``path`` matches the regular expression specified by ``expression``. For example: ```xml <attribute_matches path="outerElement/innerElement2" attribute="foo2" expression="bar\d+" /> ``` The assertion implicitly also asserts that an element matching ``path`` exists. With ``negate`` the result of the assertion (on the matching) can be inverted (the implicit assertion on the existence of the path is not affected).

field type req description
that = "attribute_matches" const "attribute_matches"
attribute string ✓ The XML attribute name to test against from the target XML element.
expression string ✓ The regular expressions to apply against the named attribute on the target XML element.
path string ✓ The Python xpath-like expression to find the target element.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Attribute Matches (Nested)

attribute_matches_model_nested

Nested version of this assertion model.

field type req description
attribute_matches → base_attribute_matches_model ✓

base_attribute_is_model

base_attribute_is_model

base model for attribute_is describing attributes.

field type req description
attribute string ✓ The XML attribute name to test against from the target XML element.
path string ✓ The Python xpath-like expression to find the target element.
text string ✓ The expected attribute value to test against on the target XML element
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_attribute_matches_model

base_attribute_matches_model

base model for attribute_matches describing attributes.

field type req description
attribute string ✓ The XML attribute name to test against from the target XML element.
expression string ✓ The regular expressions to apply against the named attribute on the target XML element.
path string ✓ The Python xpath-like expression to find the target element.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_element_text_is_model

base_element_text_is_model

base model for element_text_is describing attributes.

field type req description
path string ✓ The Python xpath-like expression to find the target element.
text string ✓ The expected element text (body of the XML tag) to test against on the target XML element
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_element_text_matches_model

base_element_text_matches_model

base model for element_text_matches describing attributes.

field type req description
expression string ✓ The regular expressions to apply against the target element.
path string ✓ The Python xpath-like expression to find the target element.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_element_text_model

base_element_text_model

base model for element_text describing attributes.

field type req description
path string ✓ The Python xpath-like expression to find the target element.
asserts → assertion_list | null
children → assertion_list | null
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_archive_member_model

base_has_archive_member_model

base model for has_archive_member describing attributes.

field type req description
path string ✓ The regular expression specifying the archive member.
all boolean | string Check the sub-assertions for all paths matching the path. Default: false, i.e. only the first default: false
asserts → assertion_list | null
children → assertion_list | null
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_element_with_path_model

base_has_element_with_path_model

base model for has_element_with_path describing attributes.

field type req description
path string ✓ The Python xpath-like expression to find the target element.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_h5_attribute_model

base_has_h5_attribute_model

base model for has_h5_attribute describing attributes.

field type req description
key string ✓ HDF5 attribute to check value of.
value string ✓ Expected value of HDF5 attribute to check.

base_has_h5_keys_model

base_has_h5_keys_model

base model for has_h5_keys describing attributes.

field type req description
keys string ✓ HDF5 attributes to check value of as a comma-separated string.

base_has_image_center_of_mass_model

base_has_image_center_of_mass_model

base model for has_image_center_of_mass describing attributes.

field type req description
center_of_mass string ✓ The required center of mass of the image intensities (horizontal and vertical coordinate, separated by a comma).
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
eps integer | number The maximum allowed Euclidean distance to the required center of mass (defaults to ``0.01``). default: 0.01
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

base_has_image_channels_model

base_has_image_channels_model

base model for has_image_channels describing attributes.

field type req description
channels integer | null Expected number of channels of the image.
delta integer Maximum allowed difference of the number of channels (default is 0). The observed number of channels has to be in the range ``value +- delta``. default: 0
max integer | null Maximum allowed number of channels.
min integer | null Minimum allowed number of channels.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_image_depth_model

base_has_image_depth_model

base model for has_image_depth describing attributes.

field type req description
delta integer Maximum allowed difference of the image depth (number of slices, default is 0). The observed depth has to be in the range ``value +- delta``. default: 0
depth integer | null Expected depth of the image (number of slices).
max integer | null Maximum allowed depth of the image (number of slices).
min integer | null Minimum allowed depth of the image (number of slices).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_image_frames_model

base_has_image_frames_model

base model for has_image_frames describing attributes.

field type req description
delta integer Maximum allowed difference of the number of frames in the image sequence (number of time steps, default is 0). The observed number of frames has to be in the range ``value +- delta``. default: 0
frames integer | null Expected number of frames in the image sequence (number of time steps).
max integer | null Maximum allowed number of frames in the image sequence (number of time steps).
min integer | null Minimum allowed number of frames in the image sequence (number of time steps).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_image_height_model

base_has_image_height_model

base model for has_image_height describing attributes.

field type req description
delta integer Maximum allowed difference of the image height (in pixels, default is 0). The observed height has to be in the range ``value +- delta``. default: 0
height integer | null Expected height of the image (in pixels).
max integer | null Maximum allowed height of the image (in pixels).
min integer | null Minimum allowed height of the image (in pixels).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_image_mean_intensity_model

base_has_image_mean_intensity_model

base model for has_image_mean_intensity describing attributes.

field type req description
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
eps integer | number The absolute tolerance to be used for ``value`` (defaults to ``0.01``). The observed mean value of the image intensities has to be in the range ``value +- eps``. default: 0.01
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
max integer | number | null An upper bound of the required mean value of the image intensities.
mean_intensity integer | number | null The required mean value of the image intensities.
min integer | number | null A lower bound of the required mean value of the image intensities.
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

base_has_image_mean_object_size_model

base_has_image_mean_object_size_model

base model for has_image_mean_object_size describing attributes.

field type req description
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
eps integer | number The absolute tolerance to be used for ``value`` (defaults to ``0.01``). The observed mean size of the uniquely labeled objects has to be in the range ``value +- eps``. default: 0.01
exclude_labels integer | number[] | null List of labels to be excluded from consideration, separated by a comma. The primary usage of this attribute is to exclude the background of a label image. Cannot be used in combination with ``labels``.
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
labels integer | number[] | null List of labels, separated by a comma. Labels *not* on this list will be excluded from consideration. Cannot be used in combination with ``exclude_labels``.
max integer | number | null An upper bound of the required mean size of the uniquely labeled objects.
mean_object_size integer | number | null The required mean size of the uniquely labeled objects.
min integer | number | null A lower bound of the required mean size of the uniquely labeled objects.
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

base_has_image_n_labels_model

base_has_image_n_labels_model

base model for has_image_n_labels describing attributes.

field type req description
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
delta integer Maximum allowed difference of the number of labels (default is 0). The observed number of labels has to be in the range ``value +- delta``. default: 0
exclude_labels integer | number[] | null List of labels to be excluded from consideration, separated by a comma. The primary usage of this attribute is to exclude the background of a label image. Cannot be used in combination with ``labels``.
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
labels integer | number[] | null List of labels, separated by a comma. Labels *not* on this list will be excluded from consideration. Cannot be used in combination with ``exclude_labels``.
max integer | null Maximum allowed number of labels.
min integer | null Minimum allowed number of labels.
n integer | null Expected number of labels.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

base_has_image_width_model

base_has_image_width_model

base model for has_image_width describing attributes.

field type req description
delta integer Maximum allowed difference of the image width (in pixels, default is 0). The observed width has to be in the range ``value +- delta``. default: 0
max integer | null Maximum allowed width of the image (in pixels).
min integer | null Minimum allowed width of the image (in pixels).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
width integer | null Expected width of the image (in pixels).

base_has_json_property_with_text_model

base_has_json_property_with_text_model

base model for has_json_property_with_text describing attributes.

field type req description
property string ✓ The property name to search the JSON document for.
text string ✓ The expected text value of the target JSON attribute.

base_has_json_property_with_value_model

base_has_json_property_with_value_model

base model for has_json_property_with_value describing attributes.

field type req description
property string ✓ The property name to search the JSON document for.
value string ✓ The expected JSON value of the target JSON attribute (as a JSON encoded string).

base_has_line_matching_model

base_has_line_matching_model

base model for has_line_matching describing attributes.

field type req description
expression string ✓ The regular expressions to attempt match in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_line_model

base_has_line_model

base model for has_line describing attributes.

field type req description
line string ✓ The full line of text to search for in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_n_columns_model

base_has_n_columns_model

base model for has_n_columns describing attributes.

field type req description
comment string Comment character(s) used to skip comment lines (which should not be used for counting columns) default: ""
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
sep string Separator defining columns, default: tab default: "\t"

base_has_n_elements_with_path_model

base_has_n_elements_with_path_model

base model for has_n_elements_with_path describing attributes.

field type req description
path string ✓ The Python xpath-like expression to find the target element.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_n_lines_model

base_has_n_lines_model

base model for has_n_lines describing attributes.

field type req description
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_size_model

base_has_size_model

base model for has_size describing attributes.

field type req description
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
size string | integer | null Desired size of the output (in bytes), can be suffixed by ``(k|M|G|T|P|E)i?``
value string | integer | null Deprecated alias for `size`

base_has_text_matching_model

base_has_text_matching_model

base model for has_text_matching describing attributes.

field type req description
expression string ✓ The regular expressions to attempt match in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_has_text_model

base_has_text_model

base model for has_text describing attributes.

field type req description
text string ✓ The text to search for in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

base_is_valid_xml_model

base_is_valid_xml_model

base model for is_valid_xml describing attributes.

base_not_has_text_model

base_not_has_text_model

base model for not_has_text describing attributes.

field type req description
text string ✓ The text to search for in the output.

base_xml_element_model

base_xml_element_model

base model for xml_element describing attributes.

field type req description
path string ✓ The Python xpath-like expression to find the target element.
all boolean | string Check the sub-assertions for all paths matching the path. Default: false, i.e. only the first default: false
asserts → assertion_list | null
attribute string | null The XML attribute name to test against from the target XML element.
children → assertion_list | null
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Collection

Collection
field type req description
class = "Collection" const "Collection" ✓
collection_type string | null
elements → LocationFile | → PathFile | → ContentsFile | → CompositeDataFile | → Collection[] | null
identifier string | null
name string | null
rows object | null

CollectionAttributes

CollectionAttributes
field type req description
collection_type string | null

CompositeDataFile

CompositeDataFile
field type req description
class = "File" const "File" ✓
composite_data string[] ✓
contents string | null
dbkey string | null
decompress boolean | null
deferred boolean | null
filetype string | null
hashes → HashEntry[] | null
identifier string | null
info string | null
location string | null
name string | null
path string | null
space_to_tab boolean | null
tags string[] | null
to_posix_lines boolean | null

ContentsFile

ContentsFile

CWL File literal — content inlined as a string.

field type req description
class = "File" const "File" ✓
contents string ✓
composite_data string[] | null
dbkey string | null
decompress boolean | null
deferred boolean | null
filetype string | null
hashes → HashEntry[] | null
identifier string | null
info string | null
location string | null
name string | null
path string | null
space_to_tab boolean | null
tags string[] | null
to_posix_lines boolean | null

Directory

Directory

CWL-style directory input. Supported by stage_inputs for directory-typed datasets (e.g. bwa_mem2_index test fixtures). Rare in workflow tests; IWC does not use it.

field type req description
class = "Directory" const "Directory" ✓
filetype string | null
location string | null
name string | null
path string | null

Assert Element Text Is

element_text_is_model

Asserts the text of the XML element with the specified XPath-like ``path`` is the specified ``text``. For example: ```xml <element_text_is path="BlastOutput_program" text="blastp" /> ``` The assertion implicitly also asserts that an element matching ``path`` exists. With ``negate`` the result of the assertion (on the equality) can be inverted (the implicit assertion on the existence of the path is not affected).

field type req description
that = "element_text_is" const "element_text_is"
path string ✓ The Python xpath-like expression to find the target element.
text string ✓ The expected element text (body of the XML tag) to test against on the target XML element
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Element Text Is (Nested)

element_text_is_model_nested

Nested version of this assertion model.

field type req description
element_text_is → base_element_text_is_model ✓

Assert Element Text Matches

element_text_matches_model

Asserts the text of the XML element with the specified XPath-like ``path`` matches the regular expression defined by ``expression``. For example: ```xml <element_text_matches path="BlastOutput_version" expression="BLASTP\s+2\.2.*"/> ``` The assertion implicitly also asserts that an element matching ``path`` exists. With ``negate`` the result of the assertion (on the matching) can be inverted (the implicit assertion on the existence of the path is not affected).

field type req description
that = "element_text_matches" const "element_text_matches"
expression string ✓ The regular expressions to apply against the target element.
path string ✓ The Python xpath-like expression to find the target element.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Element Text Matches (Nested)

element_text_matches_model_nested

Nested version of this assertion model.

field type req description
element_text_matches → base_element_text_matches_model ✓

Assert Element Text

element_text_model

This tag allows the developer to recurisively specify additional assertions as child elements about just the text contained in the element specified by the XPath-like ``path``, e.g. ```xml <element_text path="BlastOutput_iterations/Iteration/Iteration_hits/Hit/Hit_def"> <not_has_text text="EDK72998.1" /> </element_text> ``` The assertion implicitly also asserts that an element matching ``path`` exists. With ``negate`` the result of the implicit assertions can be inverted. The sub-assertions, which have their own ``negate`` attribute, are not affected by ``negate``.

field type req description
that = "element_text" const "element_text"
path string ✓ The Python xpath-like expression to find the target element.
asserts → assertion_list | null
children → assertion_list | null
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Element Text (Nested)

element_text_model_nested

Nested version of this assertion model.

field type req description
element_text → base_element_text_model ✓

Assert Has Archive Member

has_archive_member_model

This tag allows to check if ``path`` is contained in a compressed file. The path is a regular expression that is matched against the full paths of the objects in the compressed file (remember that "matching" means it is checked if a prefix of the full path of an archive member is described by the regular expression). Valid archive formats include ``.zip``, ``.tar``, and ``.tar.gz``. Note that depending on the archive creation method: - full paths of the members may be prefixed with ``./`` - directories may be treated as empty files ```xml <has_archive_member path="./path/to/my-file.txt"/> ``` With ``n`` and ``delta`` (or ``min`` and ``max``) assertions on the number of archive members matching ``path`` can be expressed. The following could be used, e.g., to assert an archive containing n&plusmn;1 elements out of which at least 4 need to have a ``txt`` extension. ```xml <has_archive_member path=".*" n="10" delta="1"/> <has_archive_member path=".*\.txt" min="4"/> ``` In addition the tag can contain additional assertions as child elements about the first member in the archive matching the regular expression ``path``. For instance ```xml <has_archive_member path=".*/my-file.txt"> <not_has_text text="EDK72998.1"/> </has_archive_member> ``` If the ``all`` attribute is set to ``true`` then all archive members are subject to the assertions. Note that, archive members matching the ``path`` are sorted alphabetically. The ``negate`` attribute of the ``has_archive_member`` assertion only affects the asserts on the presence and number of matching archive members, but not any sub-assertions (which can offer the ``negate`` attribute on their own). The check if the file is an archive at all, which is also done by the function, is not affected.

field type req description
that = "has_archive_member" const "has_archive_member"
path string ✓ The regular expression specifying the archive member.
all boolean | string Check the sub-assertions for all paths matching the path. Default: false, i.e. only the first default: false
asserts → assertion_list | null
children → assertion_list | null
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Archive Member (Nested)

has_archive_member_model_nested

Nested version of this assertion model.

field type req description
has_archive_member → base_has_archive_member_model ✓

Assert Has Element With Path

has_element_with_path_model

Asserts the XML output contains at least one element (or tag) with the specified XPath-like ``path``, e.g. ```xml <has_element_with_path path="BlastOutput_param/Parameters/Parameters_matrix" /> ``` With ``negate`` the result of the assertion can be inverted.

field type req description
that = "has_element_with_path" const "has_element_with_path"
path string ✓ The Python xpath-like expression to find the target element.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Element With Path (Nested)

has_element_with_path_model_nested

Nested version of this assertion model.

field type req description
has_element_with_path → base_has_element_with_path_model ✓

Assert Has H5 Attribute

has_h5_attribute_model

Asserts HDF5 output contains the specified ``value`` for an attribute (``key``), e.g. ```xml <has_h5_attribute key="nchroms" value="15" /> ```

field type req description
that = "has_h5_attribute" const "has_h5_attribute"
key string ✓ HDF5 attribute to check value of.
value string ✓ Expected value of HDF5 attribute to check.

Assert Has H5 Attribute (Nested)

has_h5_attribute_model_nested

Nested version of this assertion model.

field type req description
has_h5_attribute → base_has_h5_attribute_model ✓

Assert Has H5 Keys

has_h5_keys_model

Asserts the specified HDF5 output has the given keys.

field type req description
that = "has_h5_keys" const "has_h5_keys"
keys string ✓ HDF5 attributes to check value of as a comma-separated string.

Assert Has H5 Keys (Nested)

has_h5_keys_model_nested

Nested version of this assertion model.

field type req description
has_h5_keys → base_has_h5_keys_model ✓

Assert Has Image Center Of Mass

has_image_center_of_mass_model

Asserts the specified output is an image and has the specified center of mass. Asserts the output is an image and has a specific center of mass, or has an Euclidean distance of ``eps`` or less to that point (e.g., ``<has_image_center_of_mass center_of_mass="511.07, 223.34" />``).

field type req description
that = "has_image_center_of_mass" const "has_image_center_of_mass"
center_of_mass string ✓ The required center of mass of the image intensities (horizontal and vertical coordinate, separated by a comma).
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
eps integer | number The maximum allowed Euclidean distance to the required center of mass (defaults to ``0.01``). default: 0.01
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

Assert Has Image Center Of Mass (Nested)

has_image_center_of_mass_model_nested

Nested version of this assertion model.

field type req description
has_image_center_of_mass → base_has_image_center_of_mass_model ✓

Assert Has Image Channels

has_image_channels_model

Asserts the output is an image and has a specific number of channels. The number of channels is plus/minus ``delta`` (e.g., ``<has_image_channels channels="3" />``). Alternatively the range of the expected number of channels can be specified by ``min`` and/or ``max``.

field type req description
that = "has_image_channels" const "has_image_channels"
channels integer | null Expected number of channels of the image.
delta integer Maximum allowed difference of the number of channels (default is 0). The observed number of channels has to be in the range ``value +- delta``. default: 0
max integer | null Maximum allowed number of channels.
min integer | null Minimum allowed number of channels.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Image Channels (Nested)

has_image_channels_model_nested

Nested version of this assertion model.

field type req description
has_image_channels → base_has_image_channels_model ✓

Assert Has Image Depth

has_image_depth_model

Asserts the output is an image and has a specific depth (number of slices). The depth is plus/minus ``delta`` (e.g., ``<has_image_depth depth="512" delta="2" />``). Alternatively the range of the expected depth can be specified by ``min`` and/or ``max``.

field type req description
that = "has_image_depth" const "has_image_depth"
delta integer Maximum allowed difference of the image depth (number of slices, default is 0). The observed depth has to be in the range ``value +- delta``. default: 0
depth integer | null Expected depth of the image (number of slices).
max integer | null Maximum allowed depth of the image (number of slices).
min integer | null Minimum allowed depth of the image (number of slices).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Image Depth (Nested)

has_image_depth_model_nested

Nested version of this assertion model.

field type req description
has_image_depth → base_has_image_depth_model ✓

Assert Has Image Frames

has_image_frames_model

Asserts the output is an image and has a specific number of frames (number of time steps). The number of frames is plus/minus ``delta`` (e.g., ``<has_image_frames depth="512" delta="2" />``). Alternatively the range of the expected number of frames can be specified by ``min`` and/or ``max``.

field type req description
that = "has_image_frames" const "has_image_frames"
delta integer Maximum allowed difference of the number of frames in the image sequence (number of time steps, default is 0). The observed number of frames has to be in the range ``value +- delta``. default: 0
frames integer | null Expected number of frames in the image sequence (number of time steps).
max integer | null Maximum allowed number of frames in the image sequence (number of time steps).
min integer | null Minimum allowed number of frames in the image sequence (number of time steps).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Image Frames (Nested)

has_image_frames_model_nested

Nested version of this assertion model.

field type req description
has_image_frames → base_has_image_frames_model ✓

Assert Has Image Height

has_image_height_model

Asserts the output is an image and has a specific height (in pixels). The height is plus/minus ``delta`` (e.g., ``<has_image_height height="512" delta="2" />``). Alternatively the range of the expected height can be specified by ``min`` and/or ``max``.

field type req description
that = "has_image_height" const "has_image_height"
delta integer Maximum allowed difference of the image height (in pixels, default is 0). The observed height has to be in the range ``value +- delta``. default: 0
height integer | null Expected height of the image (in pixels).
max integer | null Maximum allowed height of the image (in pixels).
min integer | null Minimum allowed height of the image (in pixels).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Image Height (Nested)

has_image_height_model_nested

Nested version of this assertion model.

field type req description
has_image_height → base_has_image_height_model ✓

Assert Has Image Mean Intensity

has_image_mean_intensity_model

Asserts the output is an image and has a specific mean intensity value. The mean intensity value is plus/minus ``eps`` (e.g., ``<has_image_mean_intensity mean_intensity="0.83" />``). Alternatively the range of the expected mean intensity value can be specified by ``min`` and/or ``max``.

field type req description
that = "has_image_mean_intensity" const "has_image_mean_intensity"
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
eps integer | number The absolute tolerance to be used for ``value`` (defaults to ``0.01``). The observed mean value of the image intensities has to be in the range ``value +- eps``. default: 0.01
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
max integer | number | null An upper bound of the required mean value of the image intensities.
mean_intensity integer | number | null The required mean value of the image intensities.
min integer | number | null A lower bound of the required mean value of the image intensities.
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

Assert Has Image Mean Intensity (Nested)

has_image_mean_intensity_model_nested

Nested version of this assertion model.

field type req description
has_image_mean_intensity → base_has_image_mean_intensity_model ✓

Assert Has Image Mean Object Size

has_image_mean_object_size_model

Asserts the output is an image with labeled objects which have the specified mean size (number of pixels), The mean size is plus/minus ``eps`` (e.g., ``<has_image_mean_object_size mean_object_size="111.87" exclude_labels="0" />``). The labels must be unique.

field type req description
that = "has_image_mean_object_size" const "has_image_mean_object_size"
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
eps integer | number The absolute tolerance to be used for ``value`` (defaults to ``0.01``). The observed mean size of the uniquely labeled objects has to be in the range ``value +- eps``. default: 0.01
exclude_labels integer | number[] | null List of labels to be excluded from consideration, separated by a comma. The primary usage of this attribute is to exclude the background of a label image. Cannot be used in combination with ``labels``.
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
labels integer | number[] | null List of labels, separated by a comma. Labels *not* on this list will be excluded from consideration. Cannot be used in combination with ``exclude_labels``.
max integer | number | null An upper bound of the required mean size of the uniquely labeled objects.
mean_object_size integer | number | null The required mean size of the uniquely labeled objects.
min integer | number | null A lower bound of the required mean size of the uniquely labeled objects.
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

Assert Has Image Mean Object Size (Nested)

has_image_mean_object_size_model_nested

Nested version of this assertion model.

field type req description
has_image_mean_object_size → base_has_image_mean_object_size_model ✓

Assert Has Image N Labels

has_image_n_labels_model

Asserts the output is an image and has the specified labels. Labels can be a number of labels or unique values (e.g., ``<has_image_n_labels n="187" exclude_labels="0" />``). The primary usage of this assertion is to verify the number of objects in images with uniquely labeled objects.

field type req description
that = "has_image_n_labels" const "has_image_n_labels"
channel integer | null Restricts the assertion to a specific channel of the image (where ``0`` corresponds to the first image channel).
delta integer Maximum allowed difference of the number of labels (default is 0). The observed number of labels has to be in the range ``value +- delta``. default: 0
exclude_labels integer | number[] | null List of labels to be excluded from consideration, separated by a comma. The primary usage of this attribute is to exclude the background of a label image. Cannot be used in combination with ``labels``.
frame integer | null Restricts the assertion to a specific frame of the image sequence (where ``0`` corresponds to the first image frame).
labels integer | number[] | null List of labels, separated by a comma. Labels *not* on this list will be excluded from consideration. Cannot be used in combination with ``exclude_labels``.
max integer | null Maximum allowed number of labels.
min integer | null Minimum allowed number of labels.
n integer | null Expected number of labels.
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
slice integer | null Restricts the assertion to a specific slice of the image (where ``0`` corresponds to the first image slice).

Assert Has Image N Labels (Nested)

has_image_n_labels_model_nested

Nested version of this assertion model.

field type req description
has_image_n_labels → base_has_image_n_labels_model ✓

Assert Has Image Width

has_image_width_model

Asserts the output is an image and has a specific width (in pixels). The width is plus/minus ``delta`` (e.g., ``<has_image_width width="512" delta="2" />``). Alternatively the range of the expected width can be specified by ``min`` and/or ``max``.

field type req description
that = "has_image_width" const "has_image_width"
delta integer Maximum allowed difference of the image width (in pixels, default is 0). The observed width has to be in the range ``value +- delta``. default: 0
max integer | null Maximum allowed width of the image (in pixels).
min integer | null Minimum allowed width of the image (in pixels).
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
width integer | null Expected width of the image (in pixels).

Assert Has Image Width (Nested)

has_image_width_model_nested

Nested version of this assertion model.

field type req description
has_image_width → base_has_image_width_model ✓

Assert Has Json Property With Text

has_json_property_with_text_model

Asserts the JSON document contains a property or key with the specified text (i.e. string) value. ```xml <has_json_property_with_text property="color" text="red" /> ```

field type req description
that = "has_json_property_with_text" const "has_json_property_with_text"
property string ✓ The property name to search the JSON document for.
text string ✓ The expected text value of the target JSON attribute.

Assert Has Json Property With Text (Nested)

has_json_property_with_text_model_nested

Nested version of this assertion model.

field type req description
has_json_property_with_text → base_has_json_property_with_text_model ✓

Assert Has Json Property With Value

has_json_property_with_value_model

Asserts the JSON document contains a property or key with the specified JSON value. ```xml <has_json_property_with_value property="skipped_columns" value="[1, 3, 5]" /> ```

field type req description
that = "has_json_property_with_value" const "has_json_property_with_value"
property string ✓ The property name to search the JSON document for.
value string ✓ The expected JSON value of the target JSON attribute (as a JSON encoded string).

Assert Has Json Property With Value (Nested)

has_json_property_with_value_model_nested

Nested version of this assertion model.

field type req description
has_json_property_with_value → base_has_json_property_with_value_model ✓

Assert Has Line Matching

has_line_matching_model

Asserts the specified output contains a line matching the regular expression specified by the argument expression. If n is given the assertion checks for exactly n occurrences.

field type req description
that = "has_line_matching" const "has_line_matching"
expression string ✓ The regular expressions to attempt match in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Line Matching (Nested)

has_line_matching_model_nested

Nested version of this assertion model.

field type req description
has_line_matching → base_has_line_matching_model ✓

Assert Has Line

has_line_model

Asserts the specified output contains the line specified by the argument line. The exact number of occurrences can be optionally specified by the argument n

field type req description
that = "has_line" const "has_line"
line string ✓ The full line of text to search for in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Line (Nested)

has_line_model_nested

Nested version of this assertion model.

field type req description
has_line → base_has_line_model ✓

Assert Has N Columns

has_n_columns_model

Asserts tabular output contains the specified number (``n``) of columns. For instance, ``<has_n_columns n="3"/>``. The assertion tests only the first line. Number of columns can optionally also be specified with ``delta``. Alternatively the range of expected occurrences can be specified by ``min`` and/or ``max``. Optionally a column separator (``sep``, default is `` ``) `and comment character(s) can be specified (``comment``, default is empty string). The first non-comment line is used for determining the number of columns.

field type req description
that = "has_n_columns" const "has_n_columns"
comment string Comment character(s) used to skip comment lines (which should not be used for counting columns) default: ""
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
sep string Separator defining columns, default: tab default: "\t"

Assert Has N Columns (Nested)

has_n_columns_model_nested

Nested version of this assertion model.

field type req description
has_n_columns → base_has_n_columns_model ✓

Assert Has N Elements With Path

has_n_elements_with_path_model

Asserts the XML output contains the specified number (``n``, optionally with ``delta``) of elements (or tags) with the specified XPath-like ``path``. For example: ```xml <has_n_elements_with_path n="9" path="BlastOutput_iterations/Iteration/Iteration_hits/Hit/Hit_num" /> ``` Alternatively to ``n`` and ``delta`` also the ``min`` and ``max`` attributes can be used to specify the range of the expected number of occurrences. With ``negate`` the result of the assertion can be inverted.

field type req description
that = "has_n_elements_with_path" const "has_n_elements_with_path"
path string ✓ The Python xpath-like expression to find the target element.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has N Elements With Path (Nested)

has_n_elements_with_path_model_nested

Nested version of this assertion model.

field type req description
has_n_elements_with_path → base_has_n_elements_with_path_model ✓

Assert Has N Lines

has_n_lines_model

Asserts the specified output contains ``n`` lines allowing for a difference in the number of lines (delta) or relative differebce in the number of lines

field type req description
that = "has_n_lines" const "has_n_lines"
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has N Lines (Nested)

has_n_lines_model_nested

Nested version of this assertion model.

field type req description
has_n_lines → base_has_n_lines_model ✓

Assert Has Size

has_size_model

Asserts the specified output has a size of the specified value Attributes size and value or synonyms though value is considered deprecated. The size optionally allows for absolute (``delta``) difference.

field type req description
that = "has_size" const "has_size"
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false
size string | integer | null Desired size of the output (in bytes), can be suffixed by ``(k|M|G|T|P|E)i?``
value string | integer | null Deprecated alias for `size`

Assert Has Size (Nested)

has_size_model_nested

Nested version of this assertion model.

field type req description
has_size → base_has_size_model ✓

Assert Has Text Matching

has_text_matching_model

Asserts the specified output contains text matching the regular expression specified by the argument expression. If n is given the assertion checks for exactly n (nonoverlapping) occurrences.

field type req description
that = "has_text_matching" const "has_text_matching"
expression string ✓ The regular expressions to attempt match in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Text Matching (Nested)

has_text_matching_model_nested

Nested version of this assertion model.

field type req description
has_text_matching → base_has_text_matching_model ✓

Assert Has Text

has_text_model

Asserts specified output contains the substring specified by the argument text. The exact number of occurrences can be optionally specified by the argument n

field type req description
that = "has_text" const "has_text"
text string ✓ The text to search for in the output.
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Has Text (Nested)

has_text_model_nested

Nested version of this assertion model.

field type req description
has_text → base_has_text_model ✓

HashEntry

HashEntry
field type req description
hash_function string ✓
hash_value string ✓

Assert Is Valid Xml

is_valid_xml_model

Asserts the output is a valid XML file (e.g. ``<is_valid_xml />``).

field type req description
that = "is_valid_xml" const "is_valid_xml"

Assert Is Valid Xml (Nested)

is_valid_xml_model_nested

Nested version of this assertion model.

field type req description
is_valid_xml → base_is_valid_xml_model ✓

Job

Job

LocationFile

LocationFile
field type req description
class = "File" const "File" ✓
location string ✓
composite_data string[] | null
contents string | null
dbkey string | null
decompress boolean | null
deferred boolean | null
filetype string | null
hashes → HashEntry[] | null
identifier string | null
info string | null
name string | null
path string | null
space_to_tab boolean | null
tags string[] | null
to_posix_lines boolean | null

Assert Not Has Text

not_has_text_model

Asserts specified output does not contain the substring specified by the argument text

field type req description
that = "not_has_text" const "not_has_text"
text string ✓ The text to search for in the output.

Assert Not Has Text (Nested)

not_has_text_model_nested

Nested version of this assertion model.

field type req description
not_has_text → base_not_has_text_model ✓

OutputCompareType

OutputCompareType

PathFile

PathFile
field type req description
class = "File" const "File" ✓
path string ✓
composite_data string[] | null
contents string | null
dbkey string | null
decompress boolean | null
deferred boolean | null
filetype string | null
hashes → HashEntry[] | null
identifier string | null
info string | null
location string | null
name string | null
space_to_tab boolean | null
tags string[] | null
to_posix_lines boolean | null

TestCollectionCollectionElementAssertions

TestCollectionCollectionElementAssertions
field type req description
class const "Collection" | null default: "Collection"
element_tests object | null
elements object | null

TestCollectionDatasetElementAssertions

TestCollectionDatasetElementAssertions
field type req description
asserts → assertion_list | → assertion_dict | null Assertions about the content of the output.
checksum string | null The target output's checksum should match the value specified here, in the form `hash_type$hash_value` (e.g. `sha1$8156d7ca0f46ed7abac98f82e36cfaddb2aca041`). Useful for large static files where uploading the whole file is inconvenient.
class const "File" | null default: "File"
compare → OutputCompareType | null Comparison mode used when matching the output against the reference file.
decompress boolean | null If true, decompress files before comparison. Applies to assertions expressed with `assert_contents` or `compare` set to anything but `sim_size`. Useful for testing compressed outputs that are non-deterministic despite having deterministic decompressed contents. By default, only files compressed with bz2, gzip and zip are automatically decompressed.
delta integer | null If `compare` is set to `sim_size`, the maximum allowed absolute size difference (in bytes) between the generated data set and the reference file in `test-data/`. Default is 10000 bytes. Can be combined with `delta_frac`.
delta_frac number | null If `compare` is set to `sim_size`, the maximum allowed relative size difference between the generated data set and the reference file in `test-data/`. 0.1 means the generated file can differ by at most 10%. Default is not to check for relative size difference. Can be combined with `delta`.
file string | null Name of the output file stored in the target `test-data` directory that will be used to compare against the results of executing the tool via the functional test framework.
ftype string | null If specified, this value is checked against the corresponding output's data type. If these do not match, the test will fail.
lines_diff integer | null Applies when `compare` is set to `diff`, `re_match`, or `contains`. For `diff`, the number of lines of difference to allow (a modified line counts as two: one added, one removed).
location string | null URL that points to a remote output file that will be downloaded and used for output comparison. Use only when the file cannot be included in the `test-data` folder. May be combined with `file` (downloads when missing on disk) or used alone (filename inferred from the URL). A `checksum` is also used to verify the download when provided.
metadata object | null Mapping of metadata keys to expected values for this output.
path string | null Filesystem path to a local output file used for comparison.
sort boolean | null Applies only if `compare` is `diff`, `re_match` or `re_match_multiline`. Sorts the lines of the history data set before comparison; for `diff` and `re_match` the local file is also sorted. Useful for non-deterministic output.

TestCollectionOutputAssertions

TestCollectionOutputAssertions
field type req description
attributes → CollectionAttributes | null
class const "Collection" | null default: "Collection"
collection_type string | null
element_count integer | null
element_tests object | null
elements object | null

TestDataOutputAssertions

TestDataOutputAssertions
field type req description
asserts → assertion_list | → assertion_dict | null Assertions about the content of the output.
checksum string | null The target output's checksum should match the value specified here, in the form `hash_type$hash_value` (e.g. `sha1$8156d7ca0f46ed7abac98f82e36cfaddb2aca041`). Useful for large static files where uploading the whole file is inconvenient.
class const "File" | null default: "File"
compare → OutputCompareType | null Comparison mode used when matching the output against the reference file.
decompress boolean | null If true, decompress files before comparison. Applies to assertions expressed with `assert_contents` or `compare` set to anything but `sim_size`. Useful for testing compressed outputs that are non-deterministic despite having deterministic decompressed contents. By default, only files compressed with bz2, gzip and zip are automatically decompressed.
delta integer | null If `compare` is set to `sim_size`, the maximum allowed absolute size difference (in bytes) between the generated data set and the reference file in `test-data/`. Default is 10000 bytes. Can be combined with `delta_frac`.
delta_frac number | null If `compare` is set to `sim_size`, the maximum allowed relative size difference between the generated data set and the reference file in `test-data/`. 0.1 means the generated file can differ by at most 10%. Default is not to check for relative size difference. Can be combined with `delta`.
file string | null Name of the output file stored in the target `test-data` directory that will be used to compare against the results of executing the tool via the functional test framework.
ftype string | null If specified, this value is checked against the corresponding output's data type. If these do not match, the test will fail.
lines_diff integer | null Applies when `compare` is set to `diff`, `re_match`, or `contains`. For `diff`, the number of lines of difference to allow (a modified line counts as two: one added, one removed).
location string | null URL that points to a remote output file that will be downloaded and used for output comparison. Use only when the file cannot be included in the `test-data` folder. May be combined with `file` (downloads when missing on disk) or used alone (filename inferred from the URL). A `checksum` is also used to verify the download when provided.
metadata object | null Mapping of metadata keys to expected values for this output.
path string | null Filesystem path to a local output file used for comparison.
sort boolean | null Applies only if `compare` is `diff`, `re_match` or `re_match_multiline`. Sorts the lines of the history data set before comparison; for `diff` and `re_match` the local file is also sorted. Useful for non-deterministic output.

TestJob

TestJob
field type req description
job → Job ✓ Defines the job to execute. Can be a path to a file or an inline dictionary describing the job inputs.
outputs object ✓ Defines assertions about outputs (datasets, collections or parameters). Each key corresponds to a labeled output; values are dictionaries describing the expected output.
doc string | null Describes the purpose of the test.
expect_failure boolean | null If true, the workflow is expected to produce an error. default: false

Assert Xml Element

xml_element_model

Assert if the XML file contains element(s) or tag(s) with the specified [XPath-like ``path``](https://lxml.de/xpathxslt.html). If ``n`` and ``delta`` or ``min`` and ``max`` are given also the number of occurrences is checked. ```xml <assert_contents> <xml_element path="./elem"/> <xml_element path="./elem/more[2]"/> <xml_element path=".//more" n="3" delta="1"/> </assert_contents> ``` With ``negate="true"`` the outcome of the assertions wrt the presence and number of ``path`` can be negated. If there are any sub assertions then check them against - the content of the attribute ``attribute`` - the element's text if no attribute is given ```xml <assert_contents> <xml_element path="./elem/more[2]" attribute="name"> <has_text_matching expression="foo$"/> </xml_element> </assert_contents> ``` Sub-assertions are not subject to the ``negate`` attribute of ``xml_element``. If ``all`` is ``true`` then the sub assertions are checked for all occurrences. Note that all other XML assertions can be expressed by this assertion (Galaxy also implements the other assertions by calling this one).

field type req description
that = "xml_element" const "xml_element"
path string ✓ The Python xpath-like expression to find the target element.
all boolean | string Check the sub-assertions for all paths matching the path. Default: false, i.e. only the first default: false
asserts → assertion_list | null
attribute string | null The XML attribute name to test against from the target XML element.
children → assertion_list | null
delta integer | string Allowed difference with respect to n (default: 0), can be suffixed by ``(k|M|G|T|P|E)i?`` default: 0
max string | integer | null Maximum number (default: infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
min string | integer | null Minimum number (default: -infinity), can be suffixed by ``(k|M|G|T|P|E)i?``
n string | integer | null Desired number, can be suffixed by ``(k|M|G|T|P|E)i?``
negate boolean | string A boolean that can be set to true to negate the outcome of the assertion. default: false

Assert Xml Element (Nested)

xml_element_model_nested

Nested version of this assertion model.

field type req description
xml_element → base_xml_element_model ✓

This page is auto-rendered from the JSON Schema vendored in @galaxy-tool-util/schema. The concrete Foundry artifact is packages/foundry/src/schemas/tests-format/tests.schema.json, synced from the upstream package by packages/foundry/scripts/sync-schema.mjs. Each $def becomes a section below with a stable anchor ID — research notes and Mold bodies can deep-link individual assertions via tests-format#has_text_model.

Source-of-truth chain:

  1. lib/galaxy/tool_util_models/__init__.py (Tests Pydantic model) in galaxyproject/galaxy — see PR #22566.
  2. scripts/dump-test-format-schema.py + make sync-test-format-schema in jmchilton/galaxy-tool-util-ts write tests.schema.json with a .sha256 integrity file — see PR #75.
  3. Published as @galaxy-tool-util/schema on npm; the Foundry pins a version in package.json, packages/foundry/scripts/sync-schema.mjs mirrors the JSON into packages/foundry/src/schemas/tests-format/tests.schema.json at prebuild, and the site re-renders. Mold frontmatter cites it via tests-format wiki-links; cast imports the testsFormatSchema runtime export from @galaxy-foundry/foundry (declared in this note’s package / package_export frontmatter) and serializes it into cast bundles.

At runtime in cast skills: the same vendored schema is copied verbatim into references/schemas/tests.schema.json per the casting policy in docs/COMPILATION_PIPELINE.md. @galaxy-tool-util/schema continues to provide validateTestsFile (AJV gate) and checkTestsAgainstWorkflow (label/type cross-check against a .ga or format2 workflow); foundry validate-tests-format wraps both as a CLI subcommand for cast skills and harnesses.

Incoming References (10)

  • Validate Tests Formatrelated note— AJV gate for Galaxy workflow tests YAML, with optional workflow cross-check.
  • Validate Testsrelated note— Validate Galaxy workflow test files and optionally cross-check labels against their workflow.
  • implement-galaxy-workflow-testrelated note— Assemble Galaxy workflow test fixtures and assertions.
  • nextflow-test-to-galaxy-test-planrelated note— Translate Nextflow test evidence into a Galaxy workflow test plan.
  • run-workflow-testrelated note— Execute a workflow's tests via Planemo; emit structured pass/fail and outputs.
  • Component Nextflow Testingrelated note— nf-test patterns mapped to Galaxy planemo asserts and CWL test equivalents — backs nextflow-test-to-target-tests Mold and summarize-nextflow §7.
  • Iwc Shortcuts Anti Patternsrelated note— What IWC test suites cut corners on (accepted) vs what's a code smell — existence-only probes, sim_size deltas, image dim checks, label coupling.
  • Iwc Test Data Conventionsrelated note— How IWC workflows organize and reference test data — Zenodo-first, SHA-1 integrity, collection shapes, CVMFS gotchas.
  • Nextflow nf-test snapshots to Galaxy/Planemo assertionsrelated note— Translates nf-test snapshot assertions into Galaxy workflow test-format assertions, broken out by module-level vs pipeline-level test shape.
  • Planemo Asserts Idiomsrelated note— Decision and idiom guide for picking planemo workflow-test assertions: which family per output type, how to size tolerances, when to validate.