IBM Spectrum Symphony Worker Manager¶

The Symphony worker manager integrates Scaler with IBM Spectrum Symphony, allowing Scaler to offload task execution to a Symphony cluster via the SOAM (Service-Oriented Architecture Middleware) API.

Quick Start¶

Prerequisites¶

An IBM Spectrum Symphony cluster with a configured service
The soamapi Python package installed (pip install soamapi)
Python packages: pip install opengris-scaler
Network connectivity between the machine running the worker manager and both the Scaler scheduler and the Symphony cluster

Step 1: Install Dependencies¶

pip install opengris-scaler soamapi

Step 2: Start the Scheduler¶

scaler_scheduler tcp://0.0.0.0:8516 \
    --policy-content "allocate=even_load; scaling=vanilla"

Step 3: Start the Symphony Worker Manager¶

scaler_worker_manager symphony tcp://<SCHEDULER_IP>:8516 \
    --service-name MyScalerService \
    --max-task-concurrency 8

Or use a TOML configuration file:

$ scaler config.toml

config.toml¶

[[worker_manager]]
type = "symphony"
scheduler_address = "tcp://<SCHEDULER_IP>:8516"
worker_manager_id = "wm-symphony"
service_name = "MyScalerService"
max_task_concurrency = 8
logging_level = "INFO"

Step 4: Submit Tasks¶

from scaler import Client

def compute(x):
    return x ** 2

with Client(address="tcp://<SCHEDULER_IP>:8516") as client:
    futures = client.map(compute, range(50))
    results = [f.result() for f in futures]
    print(results)

How It Works¶

The Symphony worker manager connects to the Scaler scheduler as a worker.
It establishes a SOAM connection and session to the configured Symphony service.
When the worker manager receives a task from the scheduler, it serializes the function and arguments with cloudpickle and submits them as a Symphony task via the SOAM API.
Symphony schedules the task on its compute hosts. On completion, the SOAM callback delivers the result back to the worker manager.
The worker manager deserializes the result and returns it to the Scaler scheduler.

The worker manager uses a concurrency semaphore to limit the number of tasks in flight.

Configuration Reference¶

Symphony-Specific Parameters¶

scheduler_address (positional, required): Address of the Scaler scheduler.
--service-name (-sn, required): The name of the Symphony service to connect to.
--max-task-concurrency (-mtc): Maximum number of concurrent Symphony workers (default: number of CPUs − 1).

Common Parameters¶

For networking, worker behavior, logging, and event loop options, see Common Worker Manager Parameters.