User-Defined Tools (Beta)

PR: #19434 Author: Marius van den Beek (@mvdbeek) Target Release: Galaxy 25.0

Overview

User-Defined Tools allows regular Galaxy users to create their own tools without requiring administrator privileges. These tools are:

• Written in YAML format
• Defined through the Galaxy user interface
• Stored in the database
• Executed inside containers

Key Differences from Standard Galaxy Tools

Security Model

Standard XML tools have broad access to Galaxy’s internals via Cheetah:

<command><![CDATA[
    #from pathlib import Path
    #user_id = $__app__.model.session().query($__app__.model.User.id).one()
    #open(f"{Path.home()}/a_file", "w").write("Hello!")
]]></command>

User-defined tools use sandboxed JavaScript expressions that cannot access the database or filesystem:

class: GalaxyUserTool
id: cat_user_defined
version: "0.1"
name: Concatenate Files
description: tail-to-head
container: busybox
shell_command: |
  cat $(inputs.datasets.map((input) => input.path).join(' ')) > output.txt
inputs:
  - name: datasets
    multiple: true
    type: data
outputs:
  - name: output1
    type: data
    format_source: datasets
    from_work_dir: output.txt

Tool Source Models

The PR introduces two tool source classes:

UserToolSource (class: GalaxyUserTool)

• For regular users with the Custom Tool Execution role
• Strict sandboxed execution
• Container required

AdminToolSource (class: GalaxyTool)

• For admin-created dynamic tools
• More flexible command templating
• Uses command key instead of shell_command

Command Styles

shell_command Style

Best for shell scripts with embedded JavaScript:

shell_command: cat '$(inputs.input.path)' > output.fastq

base_command + arguments Style

Builds an escaped argv list:

base_command: cat
arguments:
  - $(inputs.input.path)
  - '>'
  - output.fastq

Enabling User-Defined Tools

1. Set in galaxy.yml:

   enable_beta_tool_formats: true

2. Create a role of type “Custom Tool Execution” in admin UI

3. Assign users/groups to this role

API Endpoints

New endpoints added:

Client-Side Features

Monaco Editor Integration

• Full YAML schema validation
• JavaScript intellisense for embedded $() expressions
• Syntax highlighting for mixed YAML/JS content
• Real-time diagnostics

User Tool Panel

• New panel in sidebar for user’s custom tools
• Run tools directly or embed in workflows

Key Components Added

• UserToolPanel.vue - List/manage user tools
• UserToolEditor.vue - Monaco-based tool editor
• yaml-with-js.ts - Language config for YAML+JS syntax
• ScrollList.vue - Reusable scroll list component

Database Changes

New migration adds user_dynamic_tool_association table linking:

• user_id - Tool creator
• dynamic_tool_id - The dynamic tool record

Sharing Tools

• Tools are private to creators by default
• When a tool is embedded in a workflow, importing the workflow creates a copy for the new user
• Tools can be exported to disk and loaded as regular tools for instance-wide availability

Current Limitations

1. No configfiles support
2. No reference data access
3. No metadata/metadata files access (e.g., BAM indexes)
4. No extra_files directory access
5. JavaScript environment locked to ES2017

Security Considerations

User-defined tools share similar security risks as interactive tools. While in beta:

• Only grant Custom Tool Execution role to trusted users
• Tools must run in containers for isolation
• See Galaxy IT security docs

Technical Implementation Details

JavaScript Expression Evaluation

• Expressions in $() are evaluated at job creation time
• The inputs object contains file metadata (path, name, extension, etc.)
• Runtime model is generated from tool inputs for intellisense

Tool UUID Flow

• Each user tool gets a UUID
• UUID passed through tool form and workflow editor
• Allows tool lookup without global toolbox registration

Files Changed (Major)

Backend:

• lib/galaxy/tool_util/models.py - UserToolSource/AdminToolSource models
• lib/galaxy/webapps/galaxy/api/unprivileged_tools.py - New API
• lib/galaxy/webapps/galaxy/api/dynamic_tools.py - Updated admin API
• lib/galaxy/managers/unprivileged_tools.py - Tool management
• lib/galaxy/tools/evaluation.py - Expression evaluation

Frontend:

• client/src/components/UserToolPanel/* - Tool panel UI
• client/src/components/Tool/Editor/* - Monaco editor
• client/src/composables/useMonaco.ts - Monaco setup
• client/src/utils/yaml-with-js.ts - Mixed language support

TODO (from PR)

• More tests, especially Selenium
• Parse into separate tool_type for easier job_conf.yml/TPV addressing
• Usage documentation
• Publish UserToolSource schema

Example Tools

Simple File Concatenation

class: GalaxyUserTool
id: cat_user_defined
version: "0.1"
name: Concatenate Files
description: tail-to-head
container: busybox
shell_command: |
  cat $(inputs.datasets.map((input) => input.path).join(' ')) > output.txt
inputs:
  - name: datasets
    multiple: true
    type: data
outputs:
  - name: output1
    type: data
    format_source: datasets
    from_work_dir: output.txt

Tool with Select Parameter

class: GalaxyUserTool
id: grep_tool
version: "1.0"
name: Grep Lines
container: busybox
shell_command: |
  grep $(inputs.invert ? '-v' : '') '$(inputs.pattern)' '$(inputs.input.path)' > output.txt
inputs:
  - name: input
    type: data
  - name: pattern
    type: text
  - name: invert
    type: boolean
    label: Invert match
outputs:
  - name: output
    type: data
    from_work_dir: output.txt