Pytorch

torch.optim.lr_scheduler.ReduceLROnPlateau学習率調整pytorch



Torch Optim Lr_scheduler



公式ウェブサイトの説明： https://pytorch.org/docs/stable/optim.html?highlight=reducelronplateau#torch.optim.lr_scheduler.ReduceLROnPlateau

CLASS torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', factor=0.1, patience=10, verbose=False, threshold=0.0001, threshold_mode='rel', cooldown=0, min_lr=0, eps=1e-08)[SOURCE] Reduce learning rate when a metric has stopped improving. Models often benefit from reducing the learning rate by a factor of 2-10 once learning stagnates. This scheduler reads a metrics quantity and if no improvement is seen for a ‘patience’ number of epochs, the learning rate is reduced. Parameters: optimizer (Optimizer) – Wrapped optimizer. mode (str) – One of min, max. In min mode, lr will be reduced when the quantity monitored has stopped decreasing in max mode it will be reduced when the quantity monitored has stopped increasing. Default: ‘min’. factor (float) – Factor by which the learning rate will be reduced. new_lr = lr * factor. Default: 0.1. patience (int) – Number of epochs with no improvement after which learning rate will be reduced. For example, if patience = 2, then we will ignore the first 2 epochs with no improvement, and will only decrease the LR after the 3rd epoch if the loss still hasn’t improved then. Default: 10. verbose (bool) – If True, prints a message to stdout for each update. Default: False. threshold (float) – Threshold for measuring the new optimum, to only focus on significant changes. Default: 1e-4. threshold_mode (str) – One of rel, abs. In rel mode, dynamic_threshold = best * ( 1 + threshold ) in ‘max’ mode or best * ( 1 - threshold ) in min mode. In abs mode, dynamic_threshold = best + threshold in max mode or best - threshold in min mode. Default: ‘rel’. cooldown (int) – Number of epochs to wait before resuming normal operation after lr has been reduced. Default: 0. min_lr (float or list) – A scalar or a list of scalars. A lower bound on the learning rate of all param groups or each group respectively. Default: 0. eps (float) – Minimal decay applied to lr. If the difference between new and old lr is smaller than eps, the update is ignored. Default: 1e-8.

詳細なパラメータ



  • モデル：最小および最大モデル。例として最小を取り上げます。最適化された指標が下がっていない場合は、学習率を変更します。通常、最小モードを使用します。使用する場合は、最初にクラスを宣言し、次にscheduler.step（test_acc）を宣言します。括弧は、通常、検証セットの損失を使用するインジケーターです。
  • Factor：new_lr = lr * factor、デフォルトは0.1
  • 忍耐：いくつかのエポックが変更されていない場合、学習率が変更されます。デフォルトは10です
  • 詳細：情報を印刷するかどうか

他の基本的なデフォルトは問題ありません。