Welcome to Scaler’s documentation!

Scaler is a lightweight distributed computing Python framework that lets you easily distribute tasks across multiple machines or parallelize on a single machine.

Scaler is designed to be a drop-in replacement for Dask requiring minimal code changes. Scaler’s design focuses on performance, simplicity, reduced overhead, debuggable errors.

Key features include:

• Python’s multiprocessing module like API - e.g. client.map() and client.submit()

• Graph Tasks - submit DAG tasks with complex interdependence

• Monitoring Dashboard - monitor utilization of workers and task completion in real time

• Task Profiling - profile and trace execution of tasks

Content

• Quickstart
  • When to use it
  • Architecture
  • Installation
  • First Look (Code API)
  • Client.map
  • Client.submit
  • Things to Avoid
  • Spinning up Scheduler and Cluster Separately
• Additional Features
  • Scaler Top (Monitoring)
  • Task Profiling
  • Send Object
  • Graph Submission
  • Nested Tasks
  • Dynamically Building Graph Within a Task
  • Capability Allocation
  • Client Disconnect and Shutdown
  • Custom Serialization
• Ray Compatibility Layer
  • Quickstart
  • Remote Function Limitations
  • Shutting Down
  • A Note about the Actor Model
  • Full Examples
  • Supported APIs
• Configuration
  • Object storage server settings
  • Scheduler Settings
  • Worker Settings
  • Configuring with TOML Files
• Examples
  • Basic Usage
  • Applications
• Dev Container Setup
  • Overview
  • Prerequisites
  • Getting Started
• Development Guidelines
  • Common Guidelines
  • Python Guidelines
  • C++ Guidelines