Galaxy Worktree Bootstrapping Reference

Research document describing Galaxy’s backend and frontend dependency management, configuration, and startup scripts.

Table of Contents

1. Overview
2. Backend Dependencies
3. Frontend Dependencies
4. Configuration System
5. Startup Scripts
6. Testing Infrastructure
7. Development Workflows

Overview

Galaxy uses a multi-layered approach to dependency management and configuration:

• Backend: Python 3.10+ with uv or pip for package management
• Frontend: Node.js + pnpm for client build
• Build Tool: Vite
• Process Management: Gravity (wrapper around supervisor/systemd)
• Web Server: Gunicorn with Uvicorn workers (ASGI)

A fresh worktree requires:

1. Virtual environment creation
2. Python dependency installation
3. Client build (optional for dev)
4. Configuration file setup (optional)

Backend Dependencies

pyproject.toml

Root pyproject.toml defines:

• Core dependencies in [project.dependencies]
• Dependency groups: test, dev, typecheck
• uv configuration under [tool.uv]

Key settings:

requires-python = ">=3.10"

[tool.uv]
extra-index-url = ["https://wheels.galaxyproject.org/simple"]
index-strategy = "unsafe-best-match"
package = false
default-groups = []  # No groups installed by default

Requirements Files

All live under lib/galaxy/dependencies/:

The root requirements.txt is a symlink to pinned-requirements.txt.

Conditional Dependencies

lib/galaxy/dependencies/__init__.py implements ConditionalDependencies class that:

1. Parses Galaxy’s config files (galaxy.yml, job_conf.yml, file_source_templates, etc.)
2. Determines which optional packages are needed
3. Returns requirements from conditional-requirements.txt

Examples:

• psycopg2-binary if database uses PostgreSQL
• drmaa if job_conf uses DRMAA runner
• python-ldap if auth uses LDAP
• gcsfs if file_sources includes googlecloudstorage
• pydantic-ai if ai_api_key or inference_services configured

Updating Dependencies

Run make update-dependencies or directly:

./lib/galaxy/dependencies/update.sh

This script:

1. Installs uv if not available
2. Runs uv lock --upgrade (updates uv.lock)
3. Exports to pinned requirements files

Python Version Requirements

Minimum: Python 3.10 (checked by scripts/check_python.py)

Frontend Dependencies

Client Structure

client/
├── package.json       # Client dependencies and scripts
├── pnpm-lock.yaml     # Lockfile
├── vite.config.js     # Vite build config
├── src/               # Source code
└── dist/              # Build output (copied to static/dist/)

package.json (root)    # Prebuilt client staging

Package Manager: pnpm

Galaxy uses pnpm (not yarn). Key differences:

• Install: pnpm install --frozen-lockfile
• Run scripts: pnpm run <script> or pnpm <script>
• Config overrides in package.json under pnpm.overrides

Two Build Modes

1. Build from Source (Development)

cd client && pnpm install && pnpm build
# or
make client

2. Install Prebuilt Client

pnpm install && pnpm run stage
# or
make install-client

Root package.json references @galaxyproject/galaxy-client (npm package) for prebuilt installation.

Client Scripts

From client/package.json:

Build Hash Tracking

After client build, static/client_build_hash.txt stores the git commit hash. common_startup.sh compares this against current HEAD to determine if rebuild is needed.

Configuration System

Configuration Files

galaxy.yml Structure

Two main sections:

1. gravity: - Process Manager Config

gravity:
  gunicorn:
    bind: localhost:8080
    workers: 1
    timeout: 300
  celery:
    enable: true
    concurrency: 2

2. galaxy: - Application Config

galaxy:
  database_connection: sqlite:///./database/universe.sqlite
  file_path: database/objects
  tool_config_file: config/tool_conf.xml

Configuration Resolution

1. Environment variables prefixed GALAXY_CONFIG_ or GALAXY_CONFIG_OVERRIDE_
2. config/galaxy.yml if exists
3. config/galaxy.yml.sample defaults

The GALAXY_CONFIG_OVERRIDE_* variants always take precedence over config files.

Key Config Paths

Startup Scripts

run.sh

Main entry point. Flow:

run.sh
├── source common_startup_functions.sh
├── source config/local_env.sh (if exists)
├── parse_common_args()
├── run_common_start_up()  → calls common_startup.sh
├── setup_python()
├── set_galaxy_config_file_var()
├── find_server()
└── exec galaxy/gunicorn/galaxyctl

Command variations:

• ./run.sh - Start in foreground
• ./run.sh start or --daemon - Background daemon
• ./run.sh stop - Stop daemon
• ./run.sh restart - Restart

common_startup.sh

Heavy lifting for environment setup:

1. Virtual Environment

   • Creates .venv/ if missing
   • Uses Conda’s _galaxy_ env if available
   • Prefers uv venv over python -m venv

2. Dependency Installation

   pip install -r requirements.txt
   # If --dev-wheels:
   pip install -r lib/galaxy/dependencies/dev-requirements.txt
   # Then conditional deps based on config

3. Client Build Check

   • Compares static/client_build_hash.txt vs current git HEAD
   • Rebuilds if client/ or config/plugins/visualizations/ changed
   • Uses pnpm for installation and build

4. Sample File Copying

   • Copies *.sample files if destination doesn’t exist

common_startup_functions.sh

Utility functions:

Environment Variables

Testing Infrastructure

run_tests.sh

Wrapper script with test type selection:

Test Execution Flow

run_tests.sh
├── Parse arguments (test type, options)
├── ./scripts/common_startup.sh --dev-wheels
├── setup_python()
├── Configure test-specific env vars
└── pytest <args>

Key Test Environment Variables

End-to-End Test Configuration

Tests can use a YAML config file for credentials:

cp lib/galaxy_test/selenium/jupyter/galaxy_selenium_context.yml.sample ./galaxy_selenium_context.yml

Config keys map to environment variables:

Use with:

GALAXY_TEST_END_TO_END_CONFIG=./galaxy_selenium_context.yml pytest ...

Playwright Setup

After venv is active:

playwright install --with-deps

Then run:

./run_tests.sh -playwright
# or
GALAXY_TEST_DRIVER_BACKEND=playwright pytest lib/galaxy_test/selenium/

pytest Configuration

pytest.ini:

pythonpath = lib
asyncio_mode = auto
log_level = DEBUG
markers =
  tool: marks test as a tool test
  cwl_conformance: all CWL conformance tests
  # ... many more

Development Workflows

Quick Start (Backend Only)

# 1. Bootstrap (creates venv, installs deps, skips client)
GALAXY_SKIP_CLIENT_BUILD=1 ./run.sh

# 2. Activate venv for CLI tools
. .venv/bin/activate

# 3. Run server (subsequent starts)
./run.sh

Quick Start (With Frontend Dev)

# Terminal 1: Start Galaxy without client build
GALAXY_SKIP_CLIENT_BUILD=1 GALAXY_RUN_WITH_TEST_TOOLS=1 ./run.sh

# Terminal 2: Start Vite dev server (HMR)
make client-dev-server

# Browser: http://localhost:5173

Running Tests Against Dev Server

# Start server with test tools
GALAXY_SKIP_CLIENT_BUILD=1 GALAXY_RUN_WITH_TEST_TOOLS=1 ./run.sh &

# Activate venv
. .venv/bin/activate

# Run specific test against running server
export GALAXY_TEST_EXTERNAL=http://localhost:8080/
pytest lib/galaxy_test/selenium/test_workflow_editor.py -k test_data_input

Useful Make Targets

Minimal Bootstrap Commands

For a completely fresh worktree:

# Option A: Full auto (builds client)
./run.sh

# Option B: Dev mode (skip client, use prebuilt)
GALAXY_INSTALL_PREBUILT_CLIENT=1 GALAXY_SKIP_CLIENT_BUILD=0 ./run.sh

# Option C: Backend only (no client at all)
GALAXY_SKIP_CLIENT_BUILD=1 ./run.sh

Key Files Reference

Notes

• Galaxy uses Gravity (process manager) which wraps supervisor/systemd
• Default server: Gunicorn with Uvicorn workers (ASGI) on localhost:8080
• Celery is enabled by default for async tasks
• The _galaxy_ Conda environment is auto-created if Conda is available
• Client is built from client/ using Vite and output to static/dist/
• Tests create temporary Galaxy instances with their own databases
• Frontend uses pnpm (not yarn) - see migration cleanup issue for stale references