ExponentialSchedule

class empulse.optimizers.ExponentialSchedule(start_value, gamma, min_value=0.0, max_value=None)

Exponential schedule: value = start_value * gamma ** epoch.

Optionally clipped from below at min_value and/or from above at max_value.

\[v_t = \min\Bigl(v_{\max},\; \max\bigl(v_{\min},\; v_0 \cdot \gamma^t\bigr)\Bigr)\]

Parameters:

start_valuefloat

Value at epoch 0.

gammafloat

Multiplicative decay factor per epoch. Typically 0 < gamma < 1 for decay; gamma > 1 can be used for growth (e.g. alpha annealing).

min_valuefloat, default=0.0

Lower bound on the returned value.

max_valuefloat, optional

Upper bound on the returned value. Useful for capping a growth schedule (gamma > 1), e.g. an annealed smoothing parameter that should not exceed a fixed ceiling. If None (default), the value is unbounded above; on overflow of gamma ** epoch it then falls back to start_value. If given, it must be >= min_value, and an overflow falls back to max_value instead.

Examples

from empulse.optimizers import ExponentialSchedule

# Decaying learning rate
lr_schedule = ExponentialSchedule(start_value=1e-2, gamma=0.99, min_value=1e-5)

# Growing alpha schedule (smoothing parameter warm-up), capped at 100.0
alpha_schedule = ExponentialSchedule(start_value=1.0, gamma=1.1, max_value=100.0)

__call__(epoch)

Return the scheduled value at epoch (0-based).

Parameters:

epochint

Current epoch index, starting from 0.

Returns:

float

Scheduled parameter value.