Lines Matching full:optimizer
6 How to use an optimizer
9 To use :mod:`torch.optim` you have to construct an optimizer object that will hold
15 To construct an :class:`Optimizer` you have to give it an iterable containing the
17 you can specify optimizer-specific options such as the learning rate, weight decay, etc.
21 optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
22 optimizer = optim.Adam([var1, var2], lr=0.0001)
27 :class:`Optimizer` s also support specifying per-parameter options. To do this, instead
74 All optimizers implement a :func:`~Optimizer.step` method, that updates the
77 ``optimizer.step()``
87 optimizer.zero_grad()
91 optimizer.step()
93 ``optimizer.step(closure)``
105 optimizer.zero_grad()
110 optimizer.step(closure)
117 .. autoclass:: Optimizer
123 Optimizer.add_param_group
124 Optimizer.load_state_dict
125 Optimizer.register_load_state_dict_pre_hook
126 Optimizer.register_load_state_dict_post_hook
127 Optimizer.state_dict
128 Optimizer.register_state_dict_pre_hook
129 Optimizer.register_state_dict_post_hook
130 Optimizer.step
131 Optimizer.register_step_pre_hook
132 Optimizer.register_step_post_hook
133 Optimizer.zero_grad
230 Learning rate scheduling should be applied after optimizer's update; e.g., you
235 optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
236 scheduler = ExponentialLR(optimizer, gamma=0.9)
240 optimizer.zero_grad()
244 optimizer.step()
253 optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
254 scheduler1 = ExponentialLR(optimizer, gamma=0.9)
255 scheduler2 = MultiStepLR(optimizer, milestones=[30,80], gamma=0.1)
259 optimizer.zero_grad()
263 optimizer.step()
278 the optimizer's update; 1.1.0 changed this behavior in a BC-breaking way. If you use
279 the learning rate scheduler (calling ``scheduler.step()``) before the optimizer's update
280 (calling ``optimizer.step()``), this will skip the first value of the learning rate schedule.
346 averages, you should use the :func:`update_parameters` function after the `optimizer.step()`:
350 For SWA and EMA, this call is usually done right after the optimizer ``step()``. In the case of SWA…
382 >>> swa_scheduler = torch.optim.swa_utils.SWALR(optimizer, \
416 >>> loader, optimizer, model, loss_fn = ...
418 >>> scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=300)
420 >>> swa_scheduler = SWALR(optimizer, swa_lr=0.05)
424 >>> optimizer.zero_grad()
426 >>> optimizer.step()
445 >>> loader, optimizer, model, loss_fn = ...
451 >>> optimizer.zero_grad()
453 >>> optimizer.step()
484 .. py:module:: torch.optim.optimizer