WarmupSchedule#

class empulse.optimizers.WarmupSchedule(warmup_steps, after_schedule)[source]#

Linear warm-up for warmup_steps epochs, then delegates to after_schedule.

Let target = after_schedule(0). During warm-up (epochs 0 to warmup_steps - 1), the value increases linearly from target / warmup_steps at epoch 0 to the full target at epoch warmup_steps - 1. From epoch warmup_steps onward, after_schedule is called with the shifted epoch (epoch - warmup_steps), so epoch warmup_steps itself also maps to after_schedule(0) - the same value the warm-up just reached, so the schedule holds steady across the boundary rather than jumping.

Parameters:

warmup_stepsint: Number of epochs for the linear warm-up phase. Must be at least 1.
after_scheduleSchedule: Schedule to use after the warm-up. Its 0-based epoch counter restarts at the end of the warm-up.

Examples

from empulse.optimizers import WarmupSchedule, CosineAnnealingSchedule

cosine = CosineAnnealingSchedule(max_value=1e-2, min_value=1e-5, t_max=400)
schedule = WarmupSchedule(warmup_steps=50, after_schedule=cosine)
# For epochs 0..49: linearly warm up to 1e-2; then cosine-anneal.

__call__(epoch)[source]#

Return the scheduled value at epoch (0-based).

Parameters:

epochint: Current epoch index, starting from 0.

Returns:

float: Scheduled parameter value.