CosineAnnealingSchedule#

class empulse.optimizers.CosineAnnealingSchedule(max_value, min_value, t_max, cycle=False)[source]#

Cosine annealing between max_value and min_value over t_max epochs.

\[v_t = v_{\min} + \tfrac{1}{2}(v_{\max} - v_{\min}) \bigl(1 + \cos(\pi\, t / T_{\max})\bigr)\]

When cycle=True the schedule restarts after t_max epochs.

Parameters:

max_valuefloat: Value at epoch 0 (and after restarts when cycle=True).
min_valuefloat: Value reached at epoch t_max.
t_maxint: Half-period of the cosine curve (epochs from max to min). Must be at least 1.
cyclebool, default=False: If True, restart the annealing cycle after t_max epochs.

Examples

from empulse.optimizers import CosineAnnealingSchedule, Adam

# Cosine-anneal LR from 1e-2 to 1e-6 over 500 epochs, then restart
lr_schedule = CosineAnnealingSchedule(
    max_value=1e-2, min_value=1e-6, t_max=500, cycle=True
)
optimizer = Adam(lr=1e-2, lr_schedule=lr_schedule)

__call__(epoch)[source]#

Return the scheduled value at epoch (0-based).

Parameters:

epochint: Current epoch index, starting from 0.

Returns:

float: Scheduled parameter value.