GitHub/ReAgent
1
0
Fork 0
mirror of https://github.com/facebookresearch/ReAgent.git synced 2026-06-16 12:44:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
gh-pages
ReAgent/api/reagent.optimizer.html
T
Zhengxing Chen f2d2a25aff Update docs 11/20/2021
2021-11-20 20:47:01 -08:00

1249 lines
216 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>reagent.optimizer package &mdash; ReAgent 1.0 documentation</title>
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<script src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="reagent.models package" href="reagent.models.html" />
<link rel="prev" title="reagent.net_builder.value package" href="reagent.net_builder.value.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html" class="icon icon-home"> ReAgent
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../usage.html">Usage</a></li>
<li class="toctree-l1"><a class="reference internal" href="../rasp_tutorial.html">RASP (Not Actively Maintained)</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Advanced Topics</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../continuous_integration.html">Continuous Integration</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Package Reference</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="reagent.core.html">Core</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.data.html">Data</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.gym.html">Gym</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.evaluation.html">Evaluation</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.lite.html">Lite</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.mab.html">MAB</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.model_managers.html">Model Managers</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.model_utils.html">Model Utils</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.net_builder.html">Net Builders</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Optimizers</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#submodules">Submodules</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.optimizer">reagent.optimizer.optimizer module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.scheduler">reagent.optimizer.scheduler module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.scheduler_union">reagent.optimizer.scheduler_union module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.soft_update">reagent.optimizer.soft_update module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.uninferrable_optimizers">reagent.optimizer.uninferrable_optimizers module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.uninferrable_schedulers">reagent.optimizer.uninferrable_schedulers module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.union">reagent.optimizer.union module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer.utils">reagent.optimizer.utils module</a></li>
<li class="toctree-l2"><a class="reference internal" href="#module-reagent.optimizer">Module contents</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="reagent.models.html">Models</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.prediction.html">Prediction</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.preprocessing.html">Preprocessing</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.training.html">Training</a></li>
<li class="toctree-l1"><a class="reference internal" href="reagent.workflow.html">Workflow</a></li>
<li class="toctree-l1"><a class="reference internal" href="modules.html">All Modules</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Others</span></p>
<ul>
<li class="toctree-l1"><a class="reference external" href="https://github.com/facebookresearch/ReAgent">Github</a></li>
<li class="toctree-l1"><a class="reference internal" href="../license.html">License</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">ReAgent</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html" class="icon icon-home"></a> &raquo;</li>
<li>reagent.optimizer package</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/api/reagent.optimizer.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="reagent-optimizer-package">
<h1>reagent.optimizer package<a class="headerlink" href="#reagent-optimizer-package" title="Permalink to this headline"></a></h1>
<section id="submodules">
<h2>Submodules<a class="headerlink" href="#submodules" title="Permalink to this headline"></a></h2>
</section>
<section id="module-reagent.optimizer.optimizer">
<span id="reagent-optimizer-optimizer-module"></span><h2>reagent.optimizer.optimizer module<a class="headerlink" href="#module-reagent.optimizer.optimizer" title="Permalink to this headline"></a></h2>
<p>For each Torch optimizer, we create a wrapper pydantic dataclass around it.
We also add this class to our Optimizer registry.</p>
<p>Usage:</p>
<p>Whenever you want to use this Optimizer__Union, specify it as the type.
E.g.
class Parameters:</p>
<blockquote>
<div><p>rl: RLParameters = field(default_factory=RLParameters)
minibatch_size: int = 64
optimizer: Optimizer__Union = field(default_factory=Optimizer__Union.default)</p>
</div></blockquote>
<p>To instantiate it, specify desired optimzier in YAML file.
E.g.
rl:</p>
<blockquote>
<div><p>…</p>
</div></blockquote>
<p>minibatch: 64
optimizer:</p>
<blockquote>
<div><dl>
<dt>Adam:</dt><dd><p>lr: 0.001
eps: 1e-08
lr_schedulers:</p>
<blockquote>
<div><ul class="simple">
<li><dl class="simple">
<dt>OneCycleLR:</dt><dd><p>…</p>
</dd>
</dl>
</li>
</ul>
</div></blockquote>
</dd>
</dl>
</div></blockquote>
<p>Since we don’t know which network parameters we want to optimize,
Optimizer__Union will be a factory for the optimizer it contains.</p>
<p>Following the above example, we create an optimizer as follows:</p>
<dl class="simple">
<dt>class Trainer:</dt><dd><dl class="simple">
<dt>def __init__(self, network, params):</dt><dd><p>self.optimizer = params.optimizer.make_optimizer_scheduler(network.parameters())[“optimizer”]</p>
</dd>
<dt>def train(self, data):</dt><dd><p>…
loss.backward()
# steps both optimizer and chained lr_schedulers
self.optimizer.step()</p>
</dd>
</dl>
</dd>
</dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.optimizer.OptimizerConfig">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.optimizer.</span></span><span class="sig-name descname"><span class="pre">OptimizerConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.optimizer.OptimizerConfig" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.optimizer.OptimizerConfig.REGISTRY">
<span class="sig-name descname"><span class="pre">REGISTRY</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">{'ASGD':</span> <span class="pre">&lt;class</span> <span class="pre">'abc.ASGD'&gt;,</span> <span class="pre">'Adadelta':</span> <span class="pre">&lt;class</span> <span class="pre">'abc.Adadelta'&gt;,</span> <span class="pre">'Adagrad':</span> <span class="pre">&lt;class</span> <span class="pre">'abc.Adagrad'&gt;,</span> <span class="pre">'Adam':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.Adam'&gt;,</span> <span class="pre">'AdamW':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.AdamW'&gt;,</span> <span class="pre">'Adamax':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.Adamax'&gt;,</span> <span class="pre">'LBFGS':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.LBFGS'&gt;,</span> <span class="pre">'NAdam':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.NAdam'&gt;,</span> <span class="pre">'RAdam':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.RAdam'&gt;,</span> <span class="pre">'RMSprop':</span> <span class="pre">&lt;class</span> <span class="pre">'abc.RMSprop'&gt;,</span> <span class="pre">'Rprop':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.Rprop'&gt;,</span> <span class="pre">'SGD':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.SGD'&gt;,</span> <span class="pre">'SparseAdam':</span> <span class="pre">&lt;class</span> <span class="pre">'reagent.optimizer.uninferrable_optimizers.SparseAdam'&gt;}</span></em><a class="headerlink" href="#reagent.optimizer.optimizer.OptimizerConfig.REGISTRY" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.optimizer.OptimizerConfig.REGISTRY_FROZEN">
<span class="sig-name descname"><span class="pre">REGISTRY_FROZEN</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#reagent.optimizer.optimizer.OptimizerConfig.REGISTRY_FROZEN" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.optimizer.OptimizerConfig.REGISTRY_NAME">
<span class="sig-name descname"><span class="pre">REGISTRY_NAME</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'OptimizerConfig'</span></em><a class="headerlink" href="#reagent.optimizer.optimizer.OptimizerConfig.REGISTRY_NAME" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.optimizer.OptimizerConfig.lr_schedulers">
<span class="sig-name descname"><span class="pre">lr_schedulers</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></a><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#reagent.optimizer.optimizer.OptimizerConfig.lr_schedulers" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.optimizer.OptimizerConfig.make_optimizer_scheduler">
<span class="sig-name descname"><span class="pre">make_optimizer_scheduler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">params</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">torch.optim.optimizer.Optimizer</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">torch.optim.lr_scheduler._LRScheduler</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#reagent.optimizer.optimizer.OptimizerConfig.make_optimizer_scheduler" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
</section>
<section id="module-reagent.optimizer.scheduler">
<span id="reagent-optimizer-scheduler-module"></span><h2>reagent.optimizer.scheduler module<a class="headerlink" href="#module-reagent.optimizer.scheduler" title="Permalink to this headline"></a></h2>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler.LearningRateSchedulerConfig">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.scheduler.</span></span><span class="sig-name descname"><span class="pre">LearningRateSchedulerConfig</span></span><a class="headerlink" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler.LearningRateSchedulerConfig.REGISTRY">
<span class="sig-name descname"><span class="pre">REGISTRY</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">{}</span></em><a class="headerlink" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig.REGISTRY" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler.LearningRateSchedulerConfig.REGISTRY_FROZEN">
<span class="sig-name descname"><span class="pre">REGISTRY_FROZEN</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig.REGISTRY_FROZEN" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler.LearningRateSchedulerConfig.REGISTRY_NAME">
<span class="sig-name descname"><span class="pre">REGISTRY_NAME</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'LearningRateSchedulerConfig'</span></em><a class="headerlink" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig.REGISTRY_NAME" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler.LearningRateSchedulerConfig.decode_lambdas">
<span class="sig-name descname"><span class="pre">decode_lambdas</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">args</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">None</span></span></span><a class="headerlink" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig.decode_lambdas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler.LearningRateSchedulerConfig.make_from_optimizer">
<span class="sig-name descname"><span class="pre">make_from_optimizer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">optimizer</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">torch.optim.optimizer.Optimizer</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">torch.optim.lr_scheduler._LRScheduler</span></span></span><a class="headerlink" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig.make_from_optimizer" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
</section>
<section id="module-reagent.optimizer.scheduler_union">
<span id="reagent-optimizer-scheduler-union-module"></span><h2>reagent.optimizer.scheduler_union module<a class="headerlink" href="#module-reagent.optimizer.scheduler_union" title="Permalink to this headline"></a></h2>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.scheduler_union.</span></span><span class="sig-name descname"><span class="pre">LearningRateScheduler__Union</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">LambdaLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.LambdaLR" title="reagent.optimizer.uninferrable_schedulers.LambdaLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.LambdaLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">MultiplicativeLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.MultiplicativeLR" title="reagent.optimizer.uninferrable_schedulers.MultiplicativeLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.MultiplicativeLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">CyclicLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR" title="reagent.optimizer.uninferrable_schedulers.CyclicLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.CyclicLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">StepLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.StepLR" title="reagent.optimizer.uninferrable_schedulers.StepLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.StepLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">MultiStepLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.MultiStepLR" title="reagent.optimizer.uninferrable_schedulers.MultiStepLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.MultiStepLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ExponentialLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.ExponentialLR" title="reagent.optimizer.uninferrable_schedulers.ExponentialLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.ExponentialLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">CosineAnnealingLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR" title="reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">OneCycleLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR" title="reagent.optimizer.uninferrable_schedulers.OneCycleLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.OneCycleLR</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">CosineAnnealingWarmRestarts</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts" title="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts"><span class="pre">reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ChainedScheduler</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.ChainedScheduler</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ConstantLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.ConstantLR</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">LinearLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.LinearLR</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">SequentialLR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.SequentialLR</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="reagent.core.html#reagent.core.tagged_union.TaggedUnion" title="reagent.core.tagged_union.TaggedUnion"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.core.tagged_union.TaggedUnion</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.ChainedScheduler">
<span class="sig-name descname"><span class="pre">ChainedScheduler</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.ChainedScheduler</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.ChainedScheduler" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.ConstantLR">
<span class="sig-name descname"><span class="pre">ConstantLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.ConstantLR</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.ConstantLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.CosineAnnealingLR">
<span class="sig-name descname"><span class="pre">CosineAnnealingLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR" title="reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.CosineAnnealingLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.CosineAnnealingWarmRestarts">
<span class="sig-name descname"><span class="pre">CosineAnnealingWarmRestarts</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts" title="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts"><span class="pre">reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.CosineAnnealingWarmRestarts" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.CyclicLR">
<span class="sig-name descname"><span class="pre">CyclicLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR" title="reagent.optimizer.uninferrable_schedulers.CyclicLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.CyclicLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.CyclicLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.ExponentialLR">
<span class="sig-name descname"><span class="pre">ExponentialLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.ExponentialLR" title="reagent.optimizer.uninferrable_schedulers.ExponentialLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.ExponentialLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.ExponentialLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.LambdaLR">
<span class="sig-name descname"><span class="pre">LambdaLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.LambdaLR" title="reagent.optimizer.uninferrable_schedulers.LambdaLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.LambdaLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.LambdaLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.LinearLR">
<span class="sig-name descname"><span class="pre">LinearLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.LinearLR</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.LinearLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.MultiStepLR">
<span class="sig-name descname"><span class="pre">MultiStepLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.MultiStepLR" title="reagent.optimizer.uninferrable_schedulers.MultiStepLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.MultiStepLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.MultiStepLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.MultiplicativeLR">
<span class="sig-name descname"><span class="pre">MultiplicativeLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.MultiplicativeLR" title="reagent.optimizer.uninferrable_schedulers.MultiplicativeLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.MultiplicativeLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.MultiplicativeLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.OneCycleLR">
<span class="sig-name descname"><span class="pre">OneCycleLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR" title="reagent.optimizer.uninferrable_schedulers.OneCycleLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.OneCycleLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.OneCycleLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.SequentialLR">
<span class="sig-name descname"><span class="pre">SequentialLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">reagent.optimizer.scheduler_union.SequentialLR</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.SequentialLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.StepLR">
<span class="sig-name descname"><span class="pre">StepLR</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_schedulers.StepLR" title="reagent.optimizer.uninferrable_schedulers.StepLR"><span class="pre">reagent.optimizer.uninferrable_schedulers.StepLR</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.StepLR" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.LearningRateScheduler__Union.make_union_instance">
<span class="sig-name descname"><span class="pre">make_union_instance</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instance_class</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.scheduler_union.LearningRateScheduler__Union.make_union_instance" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py function">
<dt class="sig sig-object py" id="reagent.optimizer.scheduler_union.get_torch_lr_schedulers">
<span class="sig-prename descclassname"><span class="pre">reagent.optimizer.scheduler_union.</span></span><span class="sig-name descname"><span class="pre">get_torch_lr_schedulers</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#reagent.optimizer.scheduler_union.get_torch_lr_schedulers" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</section>
<section id="module-reagent.optimizer.soft_update">
<span id="reagent-optimizer-soft-update-module"></span><h2>reagent.optimizer.soft_update module<a class="headerlink" href="#module-reagent.optimizer.soft_update" title="Permalink to this headline"></a></h2>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.soft_update.SoftUpdate">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.soft_update.</span></span><span class="sig-name descname"><span class="pre">SoftUpdate</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">target_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">source_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tau</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.1</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.soft_update.SoftUpdate" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">torch.optim.optimizer.Optimizer</span></code></p>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.soft_update.SoftUpdate.make_optimizer_scheduler">
<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">make_optimizer_scheduler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">target_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">source_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tau</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.soft_update.SoftUpdate.make_optimizer_scheduler" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.soft_update.SoftUpdate.step">
<span class="sig-name descname"><span class="pre">step</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">closure</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.soft_update.SoftUpdate.step" title="Permalink to this definition"></a></dt>
<dd><p>Performs a single optimization step.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><p><strong>closure</strong> (<em>callable</em><em>, </em><em>optional</em>) – A closure that reevaluates the model
and returns the loss.</p>
</dd>
</dl>
</dd></dl>
</dd></dl>
</section>
<section id="module-reagent.optimizer.uninferrable_optimizers">
<span id="reagent-optimizer-uninferrable-optimizers-module"></span><h2>reagent.optimizer.uninferrable_optimizers module<a class="headerlink" href="#module-reagent.optimizer.uninferrable_optimizers" title="Permalink to this headline"></a></h2>
<p>This file contains configs that could not be inferred from the default values
provided by PyTorch. If PyTorch optimizers and lr_schedulers had type annotations
then we could infer everything.
default values that cannot be inferred:
- tuple
- None
TODO: remove this file once we can infer everything.</p>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adam">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">Adam</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">betas:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(0.9</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">0.999)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eps:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-08</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">amsgrad:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adam" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adam.amsgrad">
<span class="sig-name descname"><span class="pre">amsgrad</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adam.amsgrad" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adam.betas">
<span class="sig-name descname"><span class="pre">betas</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(0.9,</span> <span class="pre">0.999)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adam.betas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adam.eps">
<span class="sig-name descname"><span class="pre">eps</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-08</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adam.eps" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adam.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.001</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adam.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adam.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adam.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adam.weight_decay">
<span class="sig-name descname"><span class="pre">weight_decay</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adam.weight_decay" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.AdamW">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">AdamW</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">betas:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(0.9</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">0.999)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eps:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-08</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.01</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">amsgrad:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.AdamW" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.AdamW.amsgrad">
<span class="sig-name descname"><span class="pre">amsgrad</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.AdamW.amsgrad" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.AdamW.betas">
<span class="sig-name descname"><span class="pre">betas</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(0.9,</span> <span class="pre">0.999)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.AdamW.betas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.AdamW.eps">
<span class="sig-name descname"><span class="pre">eps</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-08</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.AdamW.eps" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.AdamW.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.001</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.AdamW.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.AdamW.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.AdamW.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.AdamW.weight_decay">
<span class="sig-name descname"><span class="pre">weight_decay</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.01</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.AdamW.weight_decay" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adamax">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">Adamax</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">betas:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(0.9</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">0.999)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eps:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-08</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adamax" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adamax.betas">
<span class="sig-name descname"><span class="pre">betas</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(0.9,</span> <span class="pre">0.999)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adamax.betas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adamax.eps">
<span class="sig-name descname"><span class="pre">eps</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-08</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adamax.eps" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adamax.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.001</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adamax.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adamax.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adamax.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Adamax.weight_decay">
<span class="sig-name descname"><span class="pre">weight_decay</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Adamax.weight_decay" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">LBFGS</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_iter:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">20</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_eval:</span> <span class="pre">Optional[int]</span> <span class="pre">=</span> <span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tolerance_grad:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-07</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tolerance_change:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-09</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">history_size:</span> <span class="pre">int</span> <span class="pre">=</span> <span class="pre">100</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">line_search_fn:</span> <span class="pre">Optional[str]</span> <span class="pre">=</span> <span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.history_size">
<span class="sig-name descname"><span class="pre">history_size</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">100</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.history_size" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.line_search_fn">
<span class="sig-name descname"><span class="pre">line_search_fn</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.line_search_fn" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.max_eval">
<span class="sig-name descname"><span class="pre">max_eval</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.max_eval" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.max_iter">
<span class="sig-name descname"><span class="pre">max_iter</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">20</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.max_iter" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.tolerance_change">
<span class="sig-name descname"><span class="pre">tolerance_change</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-09</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.tolerance_change" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.LBFGS.tolerance_grad">
<span class="sig-name descname"><span class="pre">tolerance_grad</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-07</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.LBFGS.tolerance_grad" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.NAdam">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">NAdam</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">betas:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(0.9</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">0.999)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eps:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-08</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">momentum_decay:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.004</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.NAdam" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.NAdam.betas">
<span class="sig-name descname"><span class="pre">betas</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(0.9,</span> <span class="pre">0.999)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.NAdam.betas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.NAdam.eps">
<span class="sig-name descname"><span class="pre">eps</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-08</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.NAdam.eps" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.NAdam.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.001</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.NAdam.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.NAdam.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.NAdam.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.NAdam.momentum_decay">
<span class="sig-name descname"><span class="pre">momentum_decay</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.004</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.NAdam.momentum_decay" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.NAdam.weight_decay">
<span class="sig-name descname"><span class="pre">weight_decay</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.NAdam.weight_decay" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.RAdam">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">RAdam</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">betas:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(0.9</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">0.999)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eps:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-08</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.RAdam" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.RAdam.betas">
<span class="sig-name descname"><span class="pre">betas</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(0.9,</span> <span class="pre">0.999)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.RAdam.betas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.RAdam.eps">
<span class="sig-name descname"><span class="pre">eps</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-08</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.RAdam.eps" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.RAdam.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.001</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.RAdam.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.RAdam.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.RAdam.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.RAdam.weight_decay">
<span class="sig-name descname"><span class="pre">weight_decay</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.RAdam.weight_decay" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Rprop">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">Rprop</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.01</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">etas:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(0.5</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">1.2)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">step_sizes:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(1e-06</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">50)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Rprop" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Rprop.etas">
<span class="sig-name descname"><span class="pre">etas</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(0.5,</span> <span class="pre">1.2)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Rprop.etas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Rprop.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.01</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Rprop.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Rprop.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Rprop.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.Rprop.step_sizes">
<span class="sig-name descname"><span class="pre">step_sizes</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(1e-06,</span> <span class="pre">50)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.Rprop.step_sizes" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SGD">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">SGD</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">momentum:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight_decay:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">dampening:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">nesterov:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SGD" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SGD.dampening">
<span class="sig-name descname"><span class="pre">dampening</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SGD.dampening" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SGD.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.001</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SGD.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SGD.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SGD.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SGD.momentum">
<span class="sig-name descname"><span class="pre">momentum</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SGD.momentum" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SGD.nesterov">
<span class="sig-name descname"><span class="pre">nesterov</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SGD.nesterov" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SGD.weight_decay">
<span class="sig-name descname"><span class="pre">weight_decay</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SGD.weight_decay" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SparseAdam">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_optimizers.</span></span><span class="sig-name descname"><span class="pre">SparseAdam</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_schedulers:</span> <span class="pre">List[reagent.optimizer.scheduler.LearningRateSchedulerConfig]</span> <span class="pre">=</span> <span class="pre">&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lr:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">0.001</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">betas:</span> <span class="pre">Tuple[float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">float]</span> <span class="pre">=</span> <span class="pre">(0.9</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">0.999)</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eps:</span> <span class="pre">float</span> <span class="pre">=</span> <span class="pre">1e-08</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">maximize:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.optimizer.OptimizerConfig" title="reagent.optimizer.optimizer.OptimizerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.optimizer.OptimizerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SparseAdam.betas">
<span class="sig-name descname"><span class="pre">betas</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Tuple</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">(0.9,</span> <span class="pre">0.999)</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam.betas" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SparseAdam.eps">
<span class="sig-name descname"><span class="pre">eps</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1e-08</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam.eps" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SparseAdam.lr">
<span class="sig-name descname"><span class="pre">lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.001</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam.lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_optimizers.SparseAdam.maximize">
<span class="sig-name descname"><span class="pre">maximize</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam.maximize" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
</section>
<section id="module-reagent.optimizer.uninferrable_schedulers">
<span id="reagent-optimizer-uninferrable-schedulers-module"></span><h2>reagent.optimizer.uninferrable_schedulers module<a class="headerlink" href="#module-reagent.optimizer.uninferrable_schedulers" title="Permalink to this headline"></a></h2>
<p>This file contains configs that could not be inferred from the default values
provided by PyTorch. If PyTorch optimizers and lr_schedulers had type annotations
then we could infer everything.
default values that cannot be inferred:
- tuple
- None
- required parameters (no default value)</p>
<p>Sometimes there are no defaults to infer from, so we got to include those here.
TODO: remove this file once we can infer everything.</p>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">CosineAnnealingLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">T_max</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eta_min</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.T_max">
<span class="sig-name descname"><span class="pre">T_max</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.T_max" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.eta_min">
<span class="sig-name descname"><span class="pre">eta_min</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.eta_min" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">CosineAnnealingWarmRestarts</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">T_0</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">T_mult</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">eta_min</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.T_0">
<span class="sig-name descname"><span class="pre">T_0</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.T_0" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.T_mult">
<span class="sig-name descname"><span class="pre">T_mult</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.T_mult" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.eta_min">
<span class="sig-name descname"><span class="pre">eta_min</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.eta_min" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CosineAnnealingWarmRestarts.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">CyclicLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">base_lr</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_lr</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">step_size_up</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">2000</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">step_size_down</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">mode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">'triangular'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">gamma</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">1.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">scale_fn</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">NoneType</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">scale_mode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">'cycle'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">cycle_momentum</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">base_momentum</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0.8</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_momentum</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0.9</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.uninferrable_schedulers._ScaleFnLambdaMixin</span></code>, <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.base_lr">
<span class="sig-name descname"><span class="pre">base_lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.base_lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.base_momentum">
<span class="sig-name descname"><span class="pre">base_momentum</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.8</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.base_momentum" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.cycle_momentum">
<span class="sig-name descname"><span class="pre">cycle_momentum</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.cycle_momentum" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.gamma">
<span class="sig-name descname"><span class="pre">gamma</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1.0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.gamma" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.max_lr">
<span class="sig-name descname"><span class="pre">max_lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.max_lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.max_momentum">
<span class="sig-name descname"><span class="pre">max_momentum</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.9</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.max_momentum" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.mode">
<span class="sig-name descname"><span class="pre">mode</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">str</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'triangular'</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.mode" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.scale_fn">
<span class="sig-name descname"><span class="pre">scale_fn</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.scale_fn" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.scale_mode">
<span class="sig-name descname"><span class="pre">scale_mode</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">str</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'cycle'</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.scale_mode" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.step_size_down">
<span class="sig-name descname"><span class="pre">step_size_down</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.step_size_down" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.step_size_up">
<span class="sig-name descname"><span class="pre">step_size_up</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">2000</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.step_size_up" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.CyclicLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.CyclicLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.ExponentialLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">ExponentialLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">gamma</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.ExponentialLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.ExponentialLR.gamma">
<span class="sig-name descname"><span class="pre">gamma</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.ExponentialLR.gamma" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.ExponentialLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.ExponentialLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.ExponentialLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.ExponentialLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.LambdaLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">LambdaLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_lambda</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.LambdaLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.uninferrable_schedulers._LRLambdaMixin</span></code>, <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.LambdaLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.LambdaLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.LambdaLR.lr_lambda">
<span class="sig-name descname"><span class="pre">lr_lambda</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.LambdaLR.lr_lambda" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.LambdaLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.LambdaLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiStepLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">MultiStepLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">milestones</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">gamma</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0.1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiStepLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiStepLR.gamma">
<span class="sig-name descname"><span class="pre">gamma</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiStepLR.gamma" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiStepLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiStepLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiStepLR.milestones">
<span class="sig-name descname"><span class="pre">milestones</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiStepLR.milestones" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiStepLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiStepLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiplicativeLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">MultiplicativeLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lr_lambda</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiplicativeLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.uninferrable_schedulers._LRLambdaMixin</span></code>, <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiplicativeLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiplicativeLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiplicativeLR.lr_lambda">
<span class="sig-name descname"><span class="pre">lr_lambda</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">Callable</span><span class="p"><span class="pre">[</span></span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiplicativeLR.lr_lambda" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.MultiplicativeLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.MultiplicativeLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">OneCycleLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">max_lr</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">total_steps</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">epochs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">steps_per_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">pct_start</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0.3</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">anneal_strategy</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">'cos'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">cycle_momentum</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">base_momentum</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0.85</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_momentum</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0.95</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">div_factor</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">25.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">final_div_factor</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">10000.0</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">three_phase</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.anneal_strategy">
<span class="sig-name descname"><span class="pre">anneal_strategy</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">str</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'cos'</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.anneal_strategy" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.base_momentum">
<span class="sig-name descname"><span class="pre">base_momentum</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.85</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.base_momentum" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.cycle_momentum">
<span class="sig-name descname"><span class="pre">cycle_momentum</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.cycle_momentum" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.div_factor">
<span class="sig-name descname"><span class="pre">div_factor</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">25.0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.div_factor" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.epochs">
<span class="sig-name descname"><span class="pre">epochs</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.epochs" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.final_div_factor">
<span class="sig-name descname"><span class="pre">final_div_factor</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">10000.0</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.final_div_factor" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.max_lr">
<span class="sig-name descname"><span class="pre">max_lr</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Union</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.max_lr" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.max_momentum">
<span class="sig-name descname"><span class="pre">max_momentum</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.95</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.max_momentum" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.pct_start">
<span class="sig-name descname"><span class="pre">pct_start</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.3</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.pct_start" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.steps_per_epoch">
<span class="sig-name descname"><span class="pre">steps_per_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.steps_per_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.three_phase">
<span class="sig-name descname"><span class="pre">three_phase</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.three_phase" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.total_steps">
<span class="sig-name descname"><span class="pre">total_steps</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">int</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.total_steps" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.OneCycleLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.OneCycleLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.StepLR">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.uninferrable_schedulers.</span></span><span class="sig-name descname"><span class="pre">StepLR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">step_size</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">gamma</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">0.1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">last_epoch</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">-</span> <span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.StepLR" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="#reagent.optimizer.scheduler.LearningRateSchedulerConfig" title="reagent.optimizer.scheduler.LearningRateSchedulerConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.optimizer.scheduler.LearningRateSchedulerConfig</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.StepLR.gamma">
<span class="sig-name descname"><span class="pre">gamma</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">float</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">0.1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.StepLR.gamma" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.StepLR.last_epoch">
<span class="sig-name descname"><span class="pre">last_epoch</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">-1</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.StepLR.last_epoch" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.StepLR.step_size">
<span class="sig-name descname"><span class="pre">step_size</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">int</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.StepLR.step_size" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.uninferrable_schedulers.StepLR.verbose">
<span class="sig-name descname"><span class="pre">verbose</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">bool</span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#reagent.optimizer.uninferrable_schedulers.StepLR.verbose" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
</section>
<section id="module-reagent.optimizer.union">
<span id="reagent-optimizer-union-module"></span><h2>reagent.optimizer.union module<a class="headerlink" href="#module-reagent.optimizer.union" title="Permalink to this headline"></a></h2>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.union.</span></span><span class="sig-name descname"><span class="pre">Optimizer__Union</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">Adam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adam" title="reagent.optimizer.uninferrable_optimizers.Adam"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">NAdam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.NAdam" title="reagent.optimizer.uninferrable_optimizers.NAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.NAdam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">RAdam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.RAdam" title="reagent.optimizer.uninferrable_optimizers.RAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.RAdam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">SGD</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SGD" title="reagent.optimizer.uninferrable_optimizers.SGD"><span class="pre">reagent.optimizer.uninferrable_optimizers.SGD</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">AdamW</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.AdamW" title="reagent.optimizer.uninferrable_optimizers.AdamW"><span class="pre">reagent.optimizer.uninferrable_optimizers.AdamW</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">SparseAdam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam" title="reagent.optimizer.uninferrable_optimizers.SparseAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.SparseAdam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Adamax</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adamax" title="reagent.optimizer.uninferrable_optimizers.Adamax"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adamax</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">LBFGS</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.LBFGS" title="reagent.optimizer.uninferrable_optimizers.LBFGS"><span class="pre">reagent.optimizer.uninferrable_optimizers.LBFGS</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Rprop</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Rprop" title="reagent.optimizer.uninferrable_optimizers.Rprop"><span class="pre">reagent.optimizer.uninferrable_optimizers.Rprop</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ASGD</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.ASGD</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Adadelta</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adadelta</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Adagrad</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adagrad</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">RMSprop</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.RMSprop</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="reagent.core.html#reagent.core.tagged_union.TaggedUnion" title="reagent.core.tagged_union.TaggedUnion"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.core.tagged_union.TaggedUnion</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.ASGD">
<span class="sig-name descname"><span class="pre">ASGD</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.ASGD</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.ASGD" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.Adadelta">
<span class="sig-name descname"><span class="pre">Adadelta</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adadelta</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.Adadelta" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.Adagrad">
<span class="sig-name descname"><span class="pre">Adagrad</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adagrad</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.Adagrad" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.Adam">
<span class="sig-name descname"><span class="pre">Adam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adam" title="reagent.optimizer.uninferrable_optimizers.Adam"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.Adam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.AdamW">
<span class="sig-name descname"><span class="pre">AdamW</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.AdamW" title="reagent.optimizer.uninferrable_optimizers.AdamW"><span class="pre">reagent.optimizer.uninferrable_optimizers.AdamW</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.AdamW" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.Adamax">
<span class="sig-name descname"><span class="pre">Adamax</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adamax" title="reagent.optimizer.uninferrable_optimizers.Adamax"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adamax</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.Adamax" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.LBFGS">
<span class="sig-name descname"><span class="pre">LBFGS</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.LBFGS" title="reagent.optimizer.uninferrable_optimizers.LBFGS"><span class="pre">reagent.optimizer.uninferrable_optimizers.LBFGS</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.LBFGS" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.NAdam">
<span class="sig-name descname"><span class="pre">NAdam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.NAdam" title="reagent.optimizer.uninferrable_optimizers.NAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.NAdam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.NAdam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.RAdam">
<span class="sig-name descname"><span class="pre">RAdam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.RAdam" title="reagent.optimizer.uninferrable_optimizers.RAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.RAdam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.RAdam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.RMSprop">
<span class="sig-name descname"><span class="pre">RMSprop</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.RMSprop</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.RMSprop" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.Rprop">
<span class="sig-name descname"><span class="pre">Rprop</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Rprop" title="reagent.optimizer.uninferrable_optimizers.Rprop"><span class="pre">reagent.optimizer.uninferrable_optimizers.Rprop</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.Rprop" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.SGD">
<span class="sig-name descname"><span class="pre">SGD</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SGD" title="reagent.optimizer.uninferrable_optimizers.SGD"><span class="pre">reagent.optimizer.uninferrable_optimizers.SGD</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.SGD" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.SparseAdam">
<span class="sig-name descname"><span class="pre">SparseAdam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam" title="reagent.optimizer.uninferrable_optimizers.SparseAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.SparseAdam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.SparseAdam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.default">
<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">default</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.default" title="Permalink to this definition"></a></dt>
<dd><p>Return default factory for Optimizer (defaulting to Adam).</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.make_optimizer_scheduler">
<span class="sig-name descname"><span class="pre">make_optimizer_scheduler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">params</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.make_optimizer_scheduler" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.union.Optimizer__Union.make_union_instance">
<span class="sig-name descname"><span class="pre">make_union_instance</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instance_class</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.union.Optimizer__Union.make_union_instance" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py function">
<dt class="sig sig-object py" id="reagent.optimizer.union.get_torch_optimizers">
<span class="sig-prename descclassname"><span class="pre">reagent.optimizer.union.</span></span><span class="sig-name descname"><span class="pre">get_torch_optimizers</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#reagent.optimizer.union.get_torch_optimizers" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</section>
<section id="module-reagent.optimizer.utils">
<span id="reagent-optimizer-utils-module"></span><h2>reagent.optimizer.utils module<a class="headerlink" href="#module-reagent.optimizer.utils" title="Permalink to this headline"></a></h2>
<dl class="py function">
<dt class="sig sig-object py" id="reagent.optimizer.utils.is_strict_subclass">
<span class="sig-prename descclassname"><span class="pre">reagent.optimizer.utils.</span></span><span class="sig-name descname"><span class="pre">is_strict_subclass</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">a</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">b</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.utils.is_strict_subclass" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py function">
<dt class="sig sig-object py" id="reagent.optimizer.utils.is_torch_lr_scheduler">
<span class="sig-prename descclassname"><span class="pre">reagent.optimizer.utils.</span></span><span class="sig-name descname"><span class="pre">is_torch_lr_scheduler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">cls</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.utils.is_torch_lr_scheduler" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py function">
<dt class="sig sig-object py" id="reagent.optimizer.utils.is_torch_optimizer">
<span class="sig-prename descclassname"><span class="pre">reagent.optimizer.utils.</span></span><span class="sig-name descname"><span class="pre">is_torch_optimizer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">cls</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.utils.is_torch_optimizer" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</section>
<section id="module-reagent.optimizer">
<span id="module-contents"></span><h2>Module contents<a class="headerlink" href="#module-reagent.optimizer" title="Permalink to this headline"></a></h2>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.</span></span><span class="sig-name descname"><span class="pre">Optimizer__Union</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">Adam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adam" title="reagent.optimizer.uninferrable_optimizers.Adam"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">NAdam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.NAdam" title="reagent.optimizer.uninferrable_optimizers.NAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.NAdam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">RAdam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.RAdam" title="reagent.optimizer.uninferrable_optimizers.RAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.RAdam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">SGD</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SGD" title="reagent.optimizer.uninferrable_optimizers.SGD"><span class="pre">reagent.optimizer.uninferrable_optimizers.SGD</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">AdamW</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.AdamW" title="reagent.optimizer.uninferrable_optimizers.AdamW"><span class="pre">reagent.optimizer.uninferrable_optimizers.AdamW</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">SparseAdam</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam" title="reagent.optimizer.uninferrable_optimizers.SparseAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.SparseAdam</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Adamax</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adamax" title="reagent.optimizer.uninferrable_optimizers.Adamax"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adamax</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">LBFGS</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.LBFGS" title="reagent.optimizer.uninferrable_optimizers.LBFGS"><span class="pre">reagent.optimizer.uninferrable_optimizers.LBFGS</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Rprop</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Rprop" title="reagent.optimizer.uninferrable_optimizers.Rprop"><span class="pre">reagent.optimizer.uninferrable_optimizers.Rprop</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ASGD</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.ASGD</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Adadelta</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adadelta</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Adagrad</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adagrad</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">RMSprop</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.RMSprop</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.Optimizer__Union" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <a class="reference internal" href="reagent.core.html#reagent.core.tagged_union.TaggedUnion" title="reagent.core.tagged_union.TaggedUnion"><code class="xref py py-class docutils literal notranslate"><span class="pre">reagent.core.tagged_union.TaggedUnion</span></code></a></p>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.ASGD">
<span class="sig-name descname"><span class="pre">ASGD</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.ASGD</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.ASGD" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.Adadelta">
<span class="sig-name descname"><span class="pre">Adadelta</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adadelta</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.Adadelta" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.Adagrad">
<span class="sig-name descname"><span class="pre">Adagrad</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.Adagrad</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.Adagrad" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.Adam">
<span class="sig-name descname"><span class="pre">Adam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adam" title="reagent.optimizer.uninferrable_optimizers.Adam"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.Adam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.AdamW">
<span class="sig-name descname"><span class="pre">AdamW</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.AdamW" title="reagent.optimizer.uninferrable_optimizers.AdamW"><span class="pre">reagent.optimizer.uninferrable_optimizers.AdamW</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.AdamW" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.Adamax">
<span class="sig-name descname"><span class="pre">Adamax</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Adamax" title="reagent.optimizer.uninferrable_optimizers.Adamax"><span class="pre">reagent.optimizer.uninferrable_optimizers.Adamax</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.Adamax" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.LBFGS">
<span class="sig-name descname"><span class="pre">LBFGS</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.LBFGS" title="reagent.optimizer.uninferrable_optimizers.LBFGS"><span class="pre">reagent.optimizer.uninferrable_optimizers.LBFGS</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.LBFGS" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.NAdam">
<span class="sig-name descname"><span class="pre">NAdam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.NAdam" title="reagent.optimizer.uninferrable_optimizers.NAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.NAdam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.NAdam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.RAdam">
<span class="sig-name descname"><span class="pre">RAdam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.RAdam" title="reagent.optimizer.uninferrable_optimizers.RAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.RAdam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.RAdam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.RMSprop">
<span class="sig-name descname"><span class="pre">RMSprop</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><span class="pre">abc.RMSprop</span><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.RMSprop" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.Rprop">
<span class="sig-name descname"><span class="pre">Rprop</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.Rprop" title="reagent.optimizer.uninferrable_optimizers.Rprop"><span class="pre">reagent.optimizer.uninferrable_optimizers.Rprop</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.Rprop" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.SGD">
<span class="sig-name descname"><span class="pre">SGD</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SGD" title="reagent.optimizer.uninferrable_optimizers.SGD"><span class="pre">reagent.optimizer.uninferrable_optimizers.SGD</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.SGD" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.SparseAdam">
<span class="sig-name descname"><span class="pre">SparseAdam</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Optional</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#reagent.optimizer.uninferrable_optimizers.SparseAdam" title="reagent.optimizer.uninferrable_optimizers.SparseAdam"><span class="pre">reagent.optimizer.uninferrable_optimizers.SparseAdam</span></a><span class="p"><span class="pre">]</span></span></em><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.SparseAdam" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.default">
<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">default</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.default" title="Permalink to this definition"></a></dt>
<dd><p>Return default factory for Optimizer (defaulting to Adam).</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.make_optimizer_scheduler">
<span class="sig-name descname"><span class="pre">make_optimizer_scheduler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">params</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.make_optimizer_scheduler" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.Optimizer__Union.make_union_instance">
<span class="sig-name descname"><span class="pre">make_union_instance</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">instance_class</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.Optimizer__Union.make_union_instance" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="reagent.optimizer.SoftUpdate">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">reagent.optimizer.</span></span><span class="sig-name descname"><span class="pre">SoftUpdate</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">target_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">source_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tau</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">0.1</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.SoftUpdate" title="Permalink to this definition"></a></dt>
<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">torch.optim.optimizer.Optimizer</span></code></p>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.SoftUpdate.make_optimizer_scheduler">
<em class="property"><span class="pre">classmethod</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">make_optimizer_scheduler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">target_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">source_params</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tau</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.SoftUpdate.make_optimizer_scheduler" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="reagent.optimizer.SoftUpdate.step">
<span class="sig-name descname"><span class="pre">step</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">closure</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#reagent.optimizer.SoftUpdate.step" title="Permalink to this definition"></a></dt>
<dd><p>Performs a single optimization step.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters</dt>
<dd class="field-odd"><p><strong>closure</strong> (<em>callable</em><em>, </em><em>optional</em>) – A closure that reevaluates the model
and returns the loss.</p>
</dd>
</dl>
</dd></dl>
</dd></dl>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="reagent.net_builder.value.html" class="btn btn-neutral float-left" title="reagent.net_builder.value package" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="reagent.models.html" class="btn btn-neutral float-right" title="reagent.models package" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>&#169; Copyright 2022, Meta Platforms, Inc.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink