StepSchedule#

class empulse.optimizers.StepSchedule(start_value, step_size, gamma=0.1, min_value=0.0, max_value=None)[source]#

Step decay: multiply by gamma every step_size epochs.

\[v_t = \min\Bigl(v_{\max},\; \max\bigl(v_{\min},\; v_0 \cdot \gamma^{\lfloor t / s \rfloor}\bigr)\Bigr)\]

Parameters:

start_valuefloat: Value at epoch 0.
step_sizeint: Number of epochs between each reduction. Must be at least 1.
gammafloat, default=0.1: Multiplicative factor applied at each drop. Use gamma < 1 for decay (LR reduction) or gamma > 1 for growth (alpha warm-up).
min_valuefloat, default=0.0: Lower bound on the returned value.
max_valuefloat, optional: Upper bound on the returned value. Useful for capping a growth schedule (gamma > 1), e.g. an annealed smoothing parameter that should not exceed a fixed ceiling. If None (default), the value is unbounded above. If given, it must be >= min_value.

Examples

from empulse.optimizers import StepSchedule, SGD

# Halve the learning rate every 100 epochs
lr_schedule = StepSchedule(start_value=1e-2, step_size=100, gamma=0.5)
optimizer = SGD(lr=1e-2, lr_schedule=lr_schedule)

# Double alpha every 50 epochs, capped at 100.0
alpha_schedule = StepSchedule(start_value=1.0, step_size=50, gamma=2.0, max_value=100.0)

__call__(epoch)[source]#

Return the scheduled value at epoch (0-based).

Parameters:

epochint: Current epoch index, starting from 0.

Returns:

float: Scheduled parameter value.