JezK
Edit File: cookbook.html
<!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" /> <title>Cookbook — Alembic 1.8.1 documentation</title> <link rel="stylesheet" type="text/css" href="_static/pygments.css" /> <link rel="stylesheet" type="text/css" href="_static/nature_override.css" /> <link rel="stylesheet" type="text/css" href="_static/copybutton.css" /> <link rel="stylesheet" type="text/css" href="_static/changelog.css" /> <link rel="stylesheet" type="text/css" href="_static/sphinx_paramlinks.css" /> <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script> <script src="_static/jquery.js"></script> <script src="_static/underscore.js"></script> <script src="_static/doctools.js"></script> <script src="_static/clipboard.min.js"></script> <script src="_static/copybutton.js"></script> <link rel="index" title="Index" href="genindex.html" /> <link rel="search" title="Search" href="search.html" /> <link rel="next" title="API Details" href="api/index.html" /> <link rel="prev" title="Operation Reference" href="ops.html" /> </head><body> <div class="related" role="navigation" aria-label="related navigation"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="genindex.html" title="General Index" accesskey="I">index</a></li> <li class="right" > <a href="py-modindex.html" title="Python Module Index" >modules</a> |</li> <li class="right" > <a href="api/index.html" title="API Details" accesskey="N">next</a> |</li> <li class="right" > <a href="ops.html" title="Operation Reference" accesskey="P">previous</a> |</li> <li class="nav-item nav-item-0"><a href="index.html">Alembic 1.8.1 documentation</a> »</li> <li class="nav-item nav-item-this"><a href="">Cookbook</a></li> </ul> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body" role="main"> <section id="cookbook"> <h1>Cookbook<a class="headerlink" href="#cookbook" title="Permalink to this headline">¶</a></h1> <p>A collection of “How-Tos” highlighting popular ways to extend Alembic.</p> <div class="admonition note"> <p class="admonition-title">Note</p> <p>This is a new section where we catalogue various “how-tos” based on user requests. It is often the case that users will request a feature only to learn it can be provided with a simple customization.</p> </div> <section id="building-an-up-to-date-database-from-scratch"> <span id="building-uptodate"></span><h2>Building an Up to Date Database from Scratch<a class="headerlink" href="#building-an-up-to-date-database-from-scratch" title="Permalink to this headline">¶</a></h2> <p>There’s a theory of database migrations that says that the revisions in existence for a database should be able to go from an entirely blank schema to the finished product, and back again. Alembic can roll this way. Though we think it’s kind of overkill, considering that SQLAlchemy itself can emit the full CREATE statements for any given model using <code class="xref py py-meth docutils literal notranslate"><span class="pre">create_all()</span></code>. If you check out a copy of an application, running this will give you the entire database in one shot, without the need to run through all those migration files, which are instead tailored towards applying incremental changes to an existing database.</p> <p>Alembic can integrate with a <code class="xref py py-meth docutils literal notranslate"><span class="pre">create_all()</span></code> script quite easily. After running the create operation, tell Alembic to create a new version table, and to stamp it with the most recent revision (i.e. <code class="docutils literal notranslate"><span class="pre">head</span></code>):</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># inside of a "create the database" script, first create</span> <span class="c1"># tables:</span> <span class="n">my_metadata</span><span class="o">.</span><span class="n">create_all</span><span class="p">(</span><span class="n">engine</span><span class="p">)</span> <span class="c1"># then, load the Alembic configuration and generate the</span> <span class="c1"># version table, "stamping" it with the most recent rev:</span> <span class="kn">from</span> <span class="nn">alembic.config</span> <span class="kn">import</span> <span class="n">Config</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">command</span> <span class="n">alembic_cfg</span> <span class="o">=</span> <span class="n">Config</span><span class="p">(</span><span class="s2">"/path/to/yourapp/alembic.ini"</span><span class="p">)</span> <span class="n">command</span><span class="o">.</span><span class="n">stamp</span><span class="p">(</span><span class="n">alembic_cfg</span><span class="p">,</span> <span class="s2">"head"</span><span class="p">)</span> </pre></div> </div> <p>When this approach is used, the application can generate the database using normal SQLAlchemy techniques instead of iterating through hundreds of migration scripts. Now, the purpose of the migration scripts is relegated just to movement between versions on out-of-date databases, not <em>new</em> databases. You can now remove old migration files that are no longer represented on any existing environments.</p> <p>To prune old migration files, simply delete the files. Then, in the earliest, still-remaining migration file, set <code class="docutils literal notranslate"><span class="pre">down_revision</span></code> to <code class="docutils literal notranslate"><span class="pre">None</span></code>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># replace this:</span> <span class="c1">#down_revision = '290696571ad2'</span> <span class="c1"># with this:</span> <span class="n">down_revision</span> <span class="o">=</span> <span class="kc">None</span> </pre></div> </div> <p>That file now becomes the “base” of the migration series.</p> </section> <section id="conditional-migration-elements"> <h2>Conditional Migration Elements<a class="headerlink" href="#conditional-migration-elements" title="Permalink to this headline">¶</a></h2> <p>This example features the basic idea of a common need, that of affecting how a migration runs based on command line switches.</p> <p>The technique to use here is simple; within a migration script, inspect the <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.get_x_argument" title="alembic.runtime.environment.EnvironmentContext.get_x_argument"><code class="xref py py-meth docutils literal notranslate"><span class="pre">EnvironmentContext.get_x_argument()</span></code></a> collection for any additional, user-defined parameters. Then take action based on the presence of those arguments.</p> <p>To make it such that the logic to inspect these flags is easy to use and modify, we modify our <code class="docutils literal notranslate"><span class="pre">script.py.mako</span></code> template to make this feature available in all new revision files:</p> <div class="highlight-mako notranslate"><div class="highlight"><pre><span></span><span class="x">"""</span><span class="cp">${</span><span class="n">message</span><span class="cp">}</span><span class="x"></span> <span class="x">Revision ID: </span><span class="cp">${</span><span class="n">up_revision</span><span class="cp">}</span><span class="x"></span> <span class="x">Revises: </span><span class="cp">${</span><span class="n">down_revision</span><span class="cp">}</span><span class="x"></span> <span class="x">Create Date: </span><span class="cp">${</span><span class="n">create_date</span><span class="cp">}</span><span class="x"></span> <span class="x">"""</span> <span class="x"># revision identifiers, used by Alembic.</span> <span class="x">revision = </span><span class="cp">${</span><span class="nb">repr</span><span class="p">(</span><span class="n">up_revision</span><span class="p">)</span><span class="cp">}</span><span class="x"></span> <span class="x">down_revision = </span><span class="cp">${</span><span class="nb">repr</span><span class="p">(</span><span class="n">down_revision</span><span class="p">)</span><span class="cp">}</span><span class="x"></span> <span class="x">from alembic import op</span> <span class="x">import sqlalchemy as sa</span> <span class="cp">${</span><span class="n">imports</span> <span class="k">if</span> <span class="n">imports</span> <span class="k">else</span> <span class="s2">""</span><span class="cp">}</span><span class="x"></span> <span class="x">from alembic import context</span> <span class="x">def upgrade():</span> <span class="x"> schema_upgrades()</span> <span class="x"> if context.get_x_argument(as_dictionary=True).get('data', None):</span> <span class="x"> data_upgrades()</span> <span class="x">def downgrade():</span> <span class="x"> if context.get_x_argument(as_dictionary=True).get('data', None):</span> <span class="x"> data_downgrades()</span> <span class="x"> schema_downgrades()</span> <span class="x">def schema_upgrades():</span> <span class="x"> """schema upgrade migrations go here."""</span> <span class="x"> </span><span class="cp">${</span><span class="n">upgrades</span> <span class="k">if</span> <span class="n">upgrades</span> <span class="k">else</span> <span class="s2">"pass"</span><span class="cp">}</span><span class="x"></span> <span class="x">def schema_downgrades():</span> <span class="x"> """schema downgrade migrations go here."""</span> <span class="x"> </span><span class="cp">${</span><span class="n">downgrades</span> <span class="k">if</span> <span class="n">downgrades</span> <span class="k">else</span> <span class="s2">"pass"</span><span class="cp">}</span><span class="x"></span> <span class="x">def data_upgrades():</span> <span class="x"> """Add any optional data upgrade migrations here!"""</span> <span class="x"> pass</span> <span class="x">def data_downgrades():</span> <span class="x"> """Add any optional data downgrade migrations here!"""</span> <span class="x"> pass</span> </pre></div> </div> <p>Now, when we create a new migration file, the <code class="docutils literal notranslate"><span class="pre">data_upgrades()</span></code> and <code class="docutils literal notranslate"><span class="pre">data_downgrades()</span></code> placeholders will be available, where we can add optional data migrations:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="sd">"""rev one</span> <span class="sd">Revision ID: 3ba2b522d10d</span> <span class="sd">Revises: None</span> <span class="sd">Create Date: 2014-03-04 18:05:36.992867</span> <span class="sd">"""</span> <span class="c1"># revision identifiers, used by Alembic.</span> <span class="n">revision</span> <span class="o">=</span> <span class="s1">'3ba2b522d10d'</span> <span class="n">down_revision</span> <span class="o">=</span> <span class="kc">None</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">op</span> <span class="kn">import</span> <span class="nn">sqlalchemy</span> <span class="k">as</span> <span class="nn">sa</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">String</span><span class="p">,</span> <span class="n">Column</span> <span class="kn">from</span> <span class="nn">sqlalchemy.sql</span> <span class="kn">import</span> <span class="n">table</span><span class="p">,</span> <span class="n">column</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">context</span> <span class="k">def</span> <span class="nf">upgrade</span><span class="p">():</span> <span class="n">schema_upgrades</span><span class="p">()</span> <span class="k">if</span> <span class="n">context</span><span class="o">.</span><span class="n">get_x_argument</span><span class="p">(</span><span class="n">as_dictionary</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="kc">None</span><span class="p">):</span> <span class="n">data_upgrades</span><span class="p">()</span> <span class="k">def</span> <span class="nf">downgrade</span><span class="p">():</span> <span class="k">if</span> <span class="n">context</span><span class="o">.</span><span class="n">get_x_argument</span><span class="p">(</span><span class="n">as_dictionary</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="kc">None</span><span class="p">):</span> <span class="n">data_downgrades</span><span class="p">()</span> <span class="n">schema_downgrades</span><span class="p">()</span> <span class="k">def</span> <span class="nf">schema_upgrades</span><span class="p">():</span> <span class="sd">"""schema upgrade migrations go here."""</span> <span class="n">op</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="s2">"my_table"</span><span class="p">,</span> <span class="n">Column</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="n">String</span><span class="p">))</span> <span class="k">def</span> <span class="nf">schema_downgrades</span><span class="p">():</span> <span class="sd">"""schema downgrade migrations go here."""</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_table</span><span class="p">(</span><span class="s2">"my_table"</span><span class="p">)</span> <span class="k">def</span> <span class="nf">data_upgrades</span><span class="p">():</span> <span class="sd">"""Add any optional data upgrade migrations here!"""</span> <span class="n">my_table</span> <span class="o">=</span> <span class="n">table</span><span class="p">(</span><span class="s1">'my_table'</span><span class="p">,</span> <span class="n">column</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="n">String</span><span class="p">),</span> <span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">bulk_insert</span><span class="p">(</span><span class="n">my_table</span><span class="p">,</span> <span class="p">[</span> <span class="p">{</span><span class="s1">'data'</span><span class="p">:</span> <span class="s1">'data 1'</span><span class="p">},</span> <span class="p">{</span><span class="s1">'data'</span><span class="p">:</span> <span class="s1">'data 2'</span><span class="p">},</span> <span class="p">{</span><span class="s1">'data'</span><span class="p">:</span> <span class="s1">'data 3'</span><span class="p">},</span> <span class="p">]</span> <span class="p">)</span> <span class="k">def</span> <span class="nf">data_downgrades</span><span class="p">():</span> <span class="sd">"""Add any optional data downgrade migrations here!"""</span> <span class="n">op</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"delete from my_table"</span><span class="p">)</span> </pre></div> </div> <p>To invoke our migrations with data included, we use the <code class="docutils literal notranslate"><span class="pre">-x</span></code> flag:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">alembic</span> <span class="o">-</span><span class="n">x</span> <span class="n">data</span><span class="o">=</span><span class="n">true</span> <span class="n">upgrade</span> <span class="n">head</span> </pre></div> </div> <p>The <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.get_x_argument" title="alembic.runtime.environment.EnvironmentContext.get_x_argument"><code class="xref py py-meth docutils literal notranslate"><span class="pre">EnvironmentContext.get_x_argument()</span></code></a> is an easy way to support new commandline options within environment and migration scripts.</p> </section> <section id="sharing-a-connection-across-one-or-more-programmatic-migration-commands"> <span id="connection-sharing"></span><h2>Sharing a Connection across one or more programmatic migration commands<a class="headerlink" href="#sharing-a-connection-across-one-or-more-programmatic-migration-commands" title="Permalink to this headline">¶</a></h2> <p>It is often the case that an application will need to call upon a series of commands within <a class="reference internal" href="api/commands.html#alembic-command-toplevel"><span class="std std-ref">Commands</span></a>, where it would be advantageous for all operations to proceed along a single transaction. The connectivity for a migration is typically solely determined within the <code class="docutils literal notranslate"><span class="pre">env.py</span></code> script of a migration environment, which is called within the scope of a command.</p> <p>The steps to take here are:</p> <ol class="arabic simple"> <li><p>Produce the <code class="xref py py-class docutils literal notranslate"><span class="pre">Connection</span></code> object to use.</p></li> <li><p>Place it somewhere that <code class="docutils literal notranslate"><span class="pre">env.py</span></code> will be able to access it. This can be either a. a module-level global somewhere, or b. an attribute which we place into the <a class="reference internal" href="api/config.html#alembic.config.Config.attributes" title="alembic.config.Config.attributes"><code class="xref py py-attr docutils literal notranslate"><span class="pre">Config.attributes</span></code></a> dictionary (if we are on an older Alembic version, we may also attach an attribute directly to the <a class="reference internal" href="api/config.html#alembic.config.Config" title="alembic.config.Config"><code class="xref py py-class docutils literal notranslate"><span class="pre">Config</span></code></a> object).</p></li> <li><p>The <code class="docutils literal notranslate"><span class="pre">env.py</span></code> script is modified such that it looks for this <code class="xref py py-class docutils literal notranslate"><span class="pre">Connection</span></code> and makes use of it, in lieu of building up its own <code class="xref py py-class docutils literal notranslate"><span class="pre">Engine</span></code> instance.</p></li> </ol> <p>We illustrate using <a class="reference internal" href="api/config.html#alembic.config.Config.attributes" title="alembic.config.Config.attributes"><code class="xref py py-attr docutils literal notranslate"><span class="pre">Config.attributes</span></code></a> a script that will run the <a class="reference internal" href="api/commands.html#alembic.command.upgrade" title="alembic.command.upgrade"><code class="xref py py-func docutils literal notranslate"><span class="pre">command.upgrade()</span></code></a> command programmatically within a transaction declared in a Python file:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">command</span><span class="p">,</span> <span class="n">config</span> <span class="n">cfg</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="n">Config</span><span class="p">(</span><span class="s2">"/path/to/yourapp/alembic.ini"</span><span class="p">)</span> <span class="k">with</span> <span class="n">engine</span><span class="o">.</span><span class="n">begin</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="n">cfg</span><span class="o">.</span><span class="n">attributes</span><span class="p">[</span><span class="s1">'connection'</span><span class="p">]</span> <span class="o">=</span> <span class="n">connection</span> <span class="n">command</span><span class="o">.</span><span class="n">upgrade</span><span class="p">(</span><span class="n">cfg</span><span class="p">,</span> <span class="s2">"head"</span><span class="p">)</span> </pre></div> </div> <p>Then in <code class="docutils literal notranslate"><span class="pre">env.py</span></code>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">run_migrations_online</span><span class="p">():</span> <span class="n">connectable</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="n">attributes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'connection'</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> <span class="k">if</span> <span class="n">connectable</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span> <span class="c1"># only create Engine if we don't have a Connection</span> <span class="c1"># from the outside</span> <span class="n">connectable</span> <span class="o">=</span> <span class="n">engine_from_config</span><span class="p">(</span> <span class="n">config</span><span class="o">.</span><span class="n">get_section</span><span class="p">(</span><span class="n">config</span><span class="o">.</span><span class="n">config_ini_section</span><span class="p">),</span> <span class="n">prefix</span><span class="o">=</span><span class="s1">'sqlalchemy.'</span><span class="p">,</span> <span class="n">poolclass</span><span class="o">=</span><span class="n">pool</span><span class="o">.</span><span class="n">NullPool</span><span class="p">)</span> <span class="k">with</span> <span class="n">connectable</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="n">connection</span><span class="o">=</span><span class="n">connection</span><span class="p">,</span> <span class="n">target_metadata</span><span class="o">=</span><span class="n">target_metadata</span> <span class="p">)</span> <span class="k">with</span> <span class="n">context</span><span class="o">.</span><span class="n">begin_transaction</span><span class="p">():</span> <span class="n">context</span><span class="o">.</span><span class="n">run_migrations</span><span class="p">()</span> <span class="k">else</span><span class="p">:</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="n">connection</span><span class="o">=</span><span class="n">connectable</span><span class="p">,</span> <span class="n">target_metadata</span><span class="o">=</span><span class="n">target_metadata</span> <span class="p">)</span> <span class="k">with</span> <span class="n">context</span><span class="o">.</span><span class="n">begin_transaction</span><span class="p">():</span> <span class="n">context</span><span class="o">.</span><span class="n">run_migrations</span><span class="p">()</span> </pre></div> </div> </section> <section id="replaceable-objects"> <span id="id1"></span><h2>Replaceable Objects<a class="headerlink" href="#replaceable-objects" title="Permalink to this headline">¶</a></h2> <p>This recipe proposes a hypothetical way of dealing with what we might call a <em>replaceable</em> schema object. A replaceable object is a schema object that needs to be created and dropped all at once. Examples of such objects include views, stored procedures, and triggers.</p> <div class="admonition seealso"> <p class="admonition-title">See also</p> <p>The Replaceable Object concept has been integrated by the <a class="reference external" href="https://github.com/olirice/alembic_utils">Alembic Utils</a> project, which provides autogenerate and migration support for PostgreSQL functions and views. See Alembic Utils at <a class="reference external" href="https://github.com/olirice/alembic_utils">https://github.com/olirice/alembic_utils</a> .</p> </div> <p>Replaceable objects present a problem in that in order to make incremental changes to them, we have to refer to the whole definition at once. If we need to add a new column to a view, for example, we have to drop it entirely and recreate it fresh with the extra column added, referring to the whole structure; but to make it even tougher, if we wish to support downgrade operarations in our migration scripts, we need to refer to the <em>previous</em> version of that construct fully, and we’d much rather not have to type out the whole definition in multiple places.</p> <p>This recipe proposes that we may refer to the older version of a replaceable construct by directly naming the migration version in which it was created, and having a migration refer to that previous file as migrations run. We will also demonstrate how to integrate this logic within the <a class="reference internal" href="api/operations.html#operation-plugins"><span class="std std-ref">Operation Plugins</span></a> feature introduced in Alembic 0.8. It may be very helpful to review this section first to get an overview of this API.</p> <section id="the-replaceable-object-structure"> <h3>The Replaceable Object Structure<a class="headerlink" href="#the-replaceable-object-structure" title="Permalink to this headline">¶</a></h3> <p>We first need to devise a simple format that represents the “CREATE XYZ” / “DROP XYZ” aspect of what it is we’re building. We will work with an object that represents a textual definition; while a SQL view is an object that we can define using a <a class="reference external" href="https://github.com/sqlalchemy/sqlalchemy/wiki/UsageRecipes/Views">table-metadata-like system</a>, this is not so much the case for things like stored procedures, where we pretty much need to have a full string definition written down somewhere. We’ll use a simple value object called <code class="docutils literal notranslate"><span class="pre">ReplaceableObject</span></code> that can represent any named set of SQL text to send to a “CREATE” statement of some kind:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">ReplaceableObject</span><span class="p">:</span> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">sqltext</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span> <span class="bp">self</span><span class="o">.</span><span class="n">sqltext</span> <span class="o">=</span> <span class="n">sqltext</span> </pre></div> </div> <p>Using this object in a migration script, assuming a Postgresql-style syntax, looks like:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">customer_view</span> <span class="o">=</span> <span class="n">ReplaceableObject</span><span class="p">(</span> <span class="s2">"customer_view"</span><span class="p">,</span> <span class="s2">"SELECT name, order_count FROM customer WHERE order_count > 0"</span> <span class="p">)</span> <span class="n">add_customer_sp</span> <span class="o">=</span> <span class="n">ReplaceableObject</span><span class="p">(</span> <span class="s2">"add_customer_sp(name varchar, order_count integer)"</span><span class="p">,</span> <span class="sd">"""</span> <span class="sd"> RETURNS integer AS $$</span> <span class="sd"> BEGIN</span> <span class="sd"> insert into customer (name, order_count)</span> <span class="sd"> VALUES (in_name, in_order_count);</span> <span class="sd"> END;</span> <span class="sd"> $$ LANGUAGE plpgsql;</span> <span class="sd"> """</span> <span class="p">)</span> </pre></div> </div> <p>The <code class="docutils literal notranslate"><span class="pre">ReplaceableObject</span></code> class is only one very simplistic way to do this. The structure of how we represent our schema objects is not too important for the purposes of this example; we can just as well put strings inside of tuples or dictionaries, as well as that we could define any kind of series of fields and class structures we want. The only important part is that below we will illustrate how organize the code that can consume the structure we create here.</p> </section> <section id="create-operations-for-the-target-objects"> <h3>Create Operations for the Target Objects<a class="headerlink" href="#create-operations-for-the-target-objects" title="Permalink to this headline">¶</a></h3> <p>We’ll use the <a class="reference internal" href="ops.html#alembic.operations.Operations" title="alembic.operations.Operations"><code class="xref py py-class docutils literal notranslate"><span class="pre">Operations</span></code></a> extension API to make new operations for create, drop, and replace of views and stored procedures. Using this API is also optional; we can just as well make any kind of Python function that we would invoke from our migration scripts. However, using this API gives us operations built directly into the Alembic <code class="docutils literal notranslate"><span class="pre">op.*</span></code> namespace very nicely.</p> <p>The most intricate class is below. This is the base of our “replaceable” operation, which includes not just a base operation for emitting CREATE and DROP instructions on a <code class="docutils literal notranslate"><span class="pre">ReplaceableObject</span></code>, it also assumes a certain model of “reversibility” which makes use of references to other migration files in order to refer to the “previous” version of an object:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">alembic.operations</span> <span class="kn">import</span> <span class="n">Operations</span><span class="p">,</span> <span class="n">MigrateOperation</span> <span class="k">class</span> <span class="nc">ReversibleOp</span><span class="p">(</span><span class="n">MigrateOperation</span><span class="p">):</span> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">target</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">target</span> <span class="o">=</span> <span class="n">target</span> <span class="nd">@classmethod</span> <span class="k">def</span> <span class="nf">invoke_for_target</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">operations</span><span class="p">,</span> <span class="n">target</span><span class="p">):</span> <span class="n">op</span> <span class="o">=</span> <span class="bp">cls</span><span class="p">(</span><span class="n">target</span><span class="p">)</span> <span class="k">return</span> <span class="n">operations</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="n">op</span><span class="p">)</span> <span class="k">def</span> <span class="nf">reverse</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">raise</span> <span class="ne">NotImplementedError</span><span class="p">()</span> <span class="nd">@classmethod</span> <span class="k">def</span> <span class="nf">_get_object_from_version</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">operations</span><span class="p">,</span> <span class="n">ident</span><span class="p">):</span> <span class="n">version</span><span class="p">,</span> <span class="n">objname</span> <span class="o">=</span> <span class="n">ident</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"."</span><span class="p">)</span> <span class="n">module</span> <span class="o">=</span> <span class="n">operations</span><span class="o">.</span><span class="n">get_context</span><span class="p">()</span><span class="o">.</span><span class="n">script</span><span class="o">.</span><span class="n">get_revision</span><span class="p">(</span><span class="n">version</span><span class="p">)</span><span class="o">.</span><span class="n">module</span> <span class="n">obj</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">module</span><span class="p">,</span> <span class="n">objname</span><span class="p">)</span> <span class="k">return</span> <span class="n">obj</span> <span class="nd">@classmethod</span> <span class="k">def</span> <span class="nf">replace</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">operations</span><span class="p">,</span> <span class="n">target</span><span class="p">,</span> <span class="n">replaces</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">replace_with</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span> <span class="k">if</span> <span class="n">replaces</span><span class="p">:</span> <span class="n">old_obj</span> <span class="o">=</span> <span class="bp">cls</span><span class="o">.</span><span class="n">_get_object_from_version</span><span class="p">(</span><span class="n">operations</span><span class="p">,</span> <span class="n">replaces</span><span class="p">)</span> <span class="n">drop_old</span> <span class="o">=</span> <span class="bp">cls</span><span class="p">(</span><span class="n">old_obj</span><span class="p">)</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span> <span class="n">create_new</span> <span class="o">=</span> <span class="bp">cls</span><span class="p">(</span><span class="n">target</span><span class="p">)</span> <span class="k">elif</span> <span class="n">replace_with</span><span class="p">:</span> <span class="n">old_obj</span> <span class="o">=</span> <span class="bp">cls</span><span class="o">.</span><span class="n">_get_object_from_version</span><span class="p">(</span><span class="n">operations</span><span class="p">,</span> <span class="n">replace_with</span><span class="p">)</span> <span class="n">drop_old</span> <span class="o">=</span> <span class="bp">cls</span><span class="p">(</span><span class="n">target</span><span class="p">)</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span> <span class="n">create_new</span> <span class="o">=</span> <span class="bp">cls</span><span class="p">(</span><span class="n">old_obj</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="k">raise</span> <span class="ne">TypeError</span><span class="p">(</span><span class="s2">"replaces or replace_with is required"</span><span class="p">)</span> <span class="n">operations</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="n">drop_old</span><span class="p">)</span> <span class="n">operations</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="n">create_new</span><span class="p">)</span> </pre></div> </div> <p>The workings of this class should become clear as we walk through the example. To create usable operations from this base, we will build a series of stub classes and use <a class="reference internal" href="ops.html#alembic.operations.Operations.register_operation" title="alembic.operations.Operations.register_operation"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Operations.register_operation()</span></code></a> to make them part of the <code class="docutils literal notranslate"><span class="pre">op.*</span></code> namespace:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@Operations</span><span class="o">.</span><span class="n">register_operation</span><span class="p">(</span><span class="s2">"create_view"</span><span class="p">,</span> <span class="s2">"invoke_for_target"</span><span class="p">)</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">register_operation</span><span class="p">(</span><span class="s2">"replace_view"</span><span class="p">,</span> <span class="s2">"replace"</span><span class="p">)</span> <span class="k">class</span> <span class="nc">CreateViewOp</span><span class="p">(</span><span class="n">ReversibleOp</span><span class="p">):</span> <span class="k">def</span> <span class="nf">reverse</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">return</span> <span class="n">DropViewOp</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">target</span><span class="p">)</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">register_operation</span><span class="p">(</span><span class="s2">"drop_view"</span><span class="p">,</span> <span class="s2">"invoke_for_target"</span><span class="p">)</span> <span class="k">class</span> <span class="nc">DropViewOp</span><span class="p">(</span><span class="n">ReversibleOp</span><span class="p">):</span> <span class="k">def</span> <span class="nf">reverse</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">return</span> <span class="n">CreateViewOp</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">target</span><span class="p">)</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">register_operation</span><span class="p">(</span><span class="s2">"create_sp"</span><span class="p">,</span> <span class="s2">"invoke_for_target"</span><span class="p">)</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">register_operation</span><span class="p">(</span><span class="s2">"replace_sp"</span><span class="p">,</span> <span class="s2">"replace"</span><span class="p">)</span> <span class="k">class</span> <span class="nc">CreateSPOp</span><span class="p">(</span><span class="n">ReversibleOp</span><span class="p">):</span> <span class="k">def</span> <span class="nf">reverse</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">return</span> <span class="n">DropSPOp</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">target</span><span class="p">)</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">register_operation</span><span class="p">(</span><span class="s2">"drop_sp"</span><span class="p">,</span> <span class="s2">"invoke_for_target"</span><span class="p">)</span> <span class="k">class</span> <span class="nc">DropSPOp</span><span class="p">(</span><span class="n">ReversibleOp</span><span class="p">):</span> <span class="k">def</span> <span class="nf">reverse</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">return</span> <span class="n">CreateSPOp</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">target</span><span class="p">)</span> </pre></div> </div> <p>To actually run the SQL like “CREATE VIEW” and “DROP SEQUENCE”, we’ll provide implementations using <a class="reference internal" href="ops.html#alembic.operations.Operations.implementation_for" title="alembic.operations.Operations.implementation_for"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Operations.implementation_for()</span></code></a> that run straight into <a class="reference internal" href="ops.html#alembic.operations.Operations.execute" title="alembic.operations.Operations.execute"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Operations.execute()</span></code></a>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@Operations</span><span class="o">.</span><span class="n">implementation_for</span><span class="p">(</span><span class="n">CreateViewOp</span><span class="p">)</span> <span class="k">def</span> <span class="nf">create_view</span><span class="p">(</span><span class="n">operations</span><span class="p">,</span> <span class="n">operation</span><span class="p">):</span> <span class="n">operations</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"CREATE VIEW </span><span class="si">%s</span><span class="s2"> AS </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span> <span class="n">operation</span><span class="o">.</span><span class="n">target</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">operation</span><span class="o">.</span><span class="n">target</span><span class="o">.</span><span class="n">sqltext</span> <span class="p">))</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">implementation_for</span><span class="p">(</span><span class="n">DropViewOp</span><span class="p">)</span> <span class="k">def</span> <span class="nf">drop_view</span><span class="p">(</span><span class="n">operations</span><span class="p">,</span> <span class="n">operation</span><span class="p">):</span> <span class="n">operations</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"DROP VIEW </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="n">operation</span><span class="o">.</span><span class="n">target</span><span class="o">.</span><span class="n">name</span><span class="p">)</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">implementation_for</span><span class="p">(</span><span class="n">CreateSPOp</span><span class="p">)</span> <span class="k">def</span> <span class="nf">create_sp</span><span class="p">(</span><span class="n">operations</span><span class="p">,</span> <span class="n">operation</span><span class="p">):</span> <span class="n">operations</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span> <span class="s2">"CREATE FUNCTION </span><span class="si">%s</span><span class="s2"> </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span> <span class="n">operation</span><span class="o">.</span><span class="n">target</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">operation</span><span class="o">.</span><span class="n">target</span><span class="o">.</span><span class="n">sqltext</span> <span class="p">)</span> <span class="p">)</span> <span class="nd">@Operations</span><span class="o">.</span><span class="n">implementation_for</span><span class="p">(</span><span class="n">DropSPOp</span><span class="p">)</span> <span class="k">def</span> <span class="nf">drop_sp</span><span class="p">(</span><span class="n">operations</span><span class="p">,</span> <span class="n">operation</span><span class="p">):</span> <span class="n">operations</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"DROP FUNCTION </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="n">operation</span><span class="o">.</span><span class="n">target</span><span class="o">.</span><span class="n">name</span><span class="p">)</span> </pre></div> </div> <p>All of the above code can be present anywhere within an application’s source tree; the only requirement is that when the <code class="docutils literal notranslate"><span class="pre">env.py</span></code> script is invoked, it includes imports that ultimately call upon these classes as well as the <a class="reference internal" href="ops.html#alembic.operations.Operations.register_operation" title="alembic.operations.Operations.register_operation"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Operations.register_operation()</span></code></a> and <a class="reference internal" href="ops.html#alembic.operations.Operations.implementation_for" title="alembic.operations.Operations.implementation_for"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Operations.implementation_for()</span></code></a> sequences.</p> </section> <section id="create-initial-migrations"> <h3>Create Initial Migrations<a class="headerlink" href="#create-initial-migrations" title="Permalink to this headline">¶</a></h3> <p>We can now illustrate how these objects look during use. For the first step, we’ll create a new migration to create a “customer” table:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ alembic revision -m "create table" </pre></div> </div> <p>We build the first revision as follows:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="sd">"""create table</span> <span class="sd">Revision ID: 3ab8b2dfb055</span> <span class="sd">Revises:</span> <span class="sd">Create Date: 2015-07-27 16:22:44.918507</span> <span class="sd">"""</span> <span class="c1"># revision identifiers, used by Alembic.</span> <span class="n">revision</span> <span class="o">=</span> <span class="s1">'3ab8b2dfb055'</span> <span class="n">down_revision</span> <span class="o">=</span> <span class="kc">None</span> <span class="n">branch_labels</span> <span class="o">=</span> <span class="kc">None</span> <span class="n">depends_on</span> <span class="o">=</span> <span class="kc">None</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">op</span> <span class="kn">import</span> <span class="nn">sqlalchemy</span> <span class="k">as</span> <span class="nn">sa</span> <span class="k">def</span> <span class="nf">upgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span> <span class="s2">"customer"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s1">'id'</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s1">'name'</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">String</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s1">'order_count'</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Integer</span><span class="p">),</span> <span class="p">)</span> <span class="k">def</span> <span class="nf">downgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_table</span><span class="p">(</span><span class="s1">'customer'</span><span class="p">)</span> </pre></div> </div> <p>For the second migration, we will create a view and a stored procedure which act upon this table:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ alembic revision -m "create views/sp" </pre></div> </div> <p>This migration will use the new directives:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="sd">"""create views/sp</span> <span class="sd">Revision ID: 28af9800143f</span> <span class="sd">Revises: 3ab8b2dfb055</span> <span class="sd">Create Date: 2015-07-27 16:24:03.589867</span> <span class="sd">"""</span> <span class="c1"># revision identifiers, used by Alembic.</span> <span class="n">revision</span> <span class="o">=</span> <span class="s1">'28af9800143f'</span> <span class="n">down_revision</span> <span class="o">=</span> <span class="s1">'3ab8b2dfb055'</span> <span class="n">branch_labels</span> <span class="o">=</span> <span class="kc">None</span> <span class="n">depends_on</span> <span class="o">=</span> <span class="kc">None</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">op</span> <span class="kn">import</span> <span class="nn">sqlalchemy</span> <span class="k">as</span> <span class="nn">sa</span> <span class="kn">from</span> <span class="nn">foo</span> <span class="kn">import</span> <span class="n">ReplaceableObject</span> <span class="n">customer_view</span> <span class="o">=</span> <span class="n">ReplaceableObject</span><span class="p">(</span> <span class="s2">"customer_view"</span><span class="p">,</span> <span class="s2">"SELECT name, order_count FROM customer WHERE order_count > 0"</span> <span class="p">)</span> <span class="n">add_customer_sp</span> <span class="o">=</span> <span class="n">ReplaceableObject</span><span class="p">(</span> <span class="s2">"add_customer_sp(name varchar, order_count integer)"</span><span class="p">,</span> <span class="sd">"""</span> <span class="sd"> RETURNS integer AS $$</span> <span class="sd"> BEGIN</span> <span class="sd"> insert into customer (name, order_count)</span> <span class="sd"> VALUES (in_name, in_order_count);</span> <span class="sd"> END;</span> <span class="sd"> $$ LANGUAGE plpgsql;</span> <span class="sd"> """</span> <span class="p">)</span> <span class="k">def</span> <span class="nf">upgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">create_view</span><span class="p">(</span><span class="n">customer_view</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">create_sp</span><span class="p">(</span><span class="n">add_customer_sp</span><span class="p">)</span> <span class="k">def</span> <span class="nf">downgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_view</span><span class="p">(</span><span class="n">customer_view</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_sp</span><span class="p">(</span><span class="n">add_customer_sp</span><span class="p">)</span> </pre></div> </div> <p>We see the use of our new <code class="docutils literal notranslate"><span class="pre">create_view()</span></code>, <code class="docutils literal notranslate"><span class="pre">create_sp()</span></code>, <code class="docutils literal notranslate"><span class="pre">drop_view()</span></code>, and <code class="docutils literal notranslate"><span class="pre">drop_sp()</span></code> directives. Running these to “head” we get the following (this includes an edited view of SQL emitted):</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ alembic upgrade 28af9800143 INFO [alembic.runtime.migration] Context impl PostgresqlImpl. INFO [alembic.runtime.migration] Will assume transactional DDL. INFO [sqlalchemy.engine.base.Engine] BEGIN (implicit) INFO [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s INFO [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'} INFO [sqlalchemy.engine.base.Engine] SELECT alembic_version.version_num FROM alembic_version INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s INFO [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'} INFO [alembic.runtime.migration] Running upgrade -> 3ab8b2dfb055, create table INFO [sqlalchemy.engine.base.Engine] CREATE TABLE customer ( id SERIAL NOT NULL, name VARCHAR, order_count INTEGER, PRIMARY KEY (id) ) INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] INSERT INTO alembic_version (version_num) VALUES ('3ab8b2dfb055') INFO [sqlalchemy.engine.base.Engine] {} INFO [alembic.runtime.migration] Running upgrade 3ab8b2dfb055 -> 28af9800143f, create views/sp INFO [sqlalchemy.engine.base.Engine] CREATE VIEW customer_view AS SELECT name, order_count FROM customer WHERE order_count > 0 INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] CREATE FUNCTION add_customer_sp(name varchar, order_count integer) RETURNS integer AS $$ BEGIN insert into customer (name, order_count) VALUES (in_name, in_order_count); END; $$ LANGUAGE plpgsql; INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='28af9800143f' WHERE alembic_version.version_num = '3ab8b2dfb055' INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] COMMIT </pre></div> </div> <p>We see that our CREATE TABLE proceeded as well as the CREATE VIEW and CREATE FUNCTION operations produced by our new directives.</p> </section> <section id="create-revision-migrations"> <h3>Create Revision Migrations<a class="headerlink" href="#create-revision-migrations" title="Permalink to this headline">¶</a></h3> <p>Finally, we can illustrate how we would “revise” these objects. Let’s consider we added a new column <code class="docutils literal notranslate"><span class="pre">email</span></code> to our <code class="docutils literal notranslate"><span class="pre">customer</span></code> table:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ alembic revision -m "add email col" </pre></div> </div> <p>The migration is:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="sd">"""add email col</span> <span class="sd">Revision ID: 191a2d20b025</span> <span class="sd">Revises: 28af9800143f</span> <span class="sd">Create Date: 2015-07-27 16:25:59.277326</span> <span class="sd">"""</span> <span class="c1"># revision identifiers, used by Alembic.</span> <span class="n">revision</span> <span class="o">=</span> <span class="s1">'191a2d20b025'</span> <span class="n">down_revision</span> <span class="o">=</span> <span class="s1">'28af9800143f'</span> <span class="n">branch_labels</span> <span class="o">=</span> <span class="kc">None</span> <span class="n">depends_on</span> <span class="o">=</span> <span class="kc">None</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">op</span> <span class="kn">import</span> <span class="nn">sqlalchemy</span> <span class="k">as</span> <span class="nn">sa</span> <span class="k">def</span> <span class="nf">upgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">add_column</span><span class="p">(</span><span class="s2">"customer"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s2">"email"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">String</span><span class="p">()))</span> <span class="k">def</span> <span class="nf">downgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_column</span><span class="p">(</span><span class="s2">"customer"</span><span class="p">,</span> <span class="s2">"email"</span><span class="p">)</span> </pre></div> </div> <p>We now need to recreate the <code class="docutils literal notranslate"><span class="pre">customer_view</span></code> view and the <code class="docutils literal notranslate"><span class="pre">add_customer_sp</span></code> function. To include downgrade capability, we will need to refer to the <strong>previous</strong> version of the construct; the <code class="docutils literal notranslate"><span class="pre">replace_view()</span></code> and <code class="docutils literal notranslate"><span class="pre">replace_sp()</span></code> operations we’ve created make this possible, by allowing us to refer to a specific, previous revision. the <code class="docutils literal notranslate"><span class="pre">replaces</span></code> and <code class="docutils literal notranslate"><span class="pre">replace_with</span></code> arguments accept a dot-separated string, which refers to a revision number and an object name, such as <code class="docutils literal notranslate"><span class="pre">"28af9800143f.customer_view"</span></code>. The <code class="docutils literal notranslate"><span class="pre">ReversibleOp</span></code> class makes use of the <a class="reference internal" href="ops.html#alembic.operations.Operations.get_context" title="alembic.operations.Operations.get_context"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Operations.get_context()</span></code></a> method to locate the version file we refer to:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ alembic revision -m "update views/sp" </pre></div> </div> <p>The migration:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="sd">"""update views/sp</span> <span class="sd">Revision ID: 199028bf9856</span> <span class="sd">Revises: 191a2d20b025</span> <span class="sd">Create Date: 2015-07-27 16:26:31.344504</span> <span class="sd">"""</span> <span class="c1"># revision identifiers, used by Alembic.</span> <span class="n">revision</span> <span class="o">=</span> <span class="s1">'199028bf9856'</span> <span class="n">down_revision</span> <span class="o">=</span> <span class="s1">'191a2d20b025'</span> <span class="n">branch_labels</span> <span class="o">=</span> <span class="kc">None</span> <span class="n">depends_on</span> <span class="o">=</span> <span class="kc">None</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">op</span> <span class="kn">import</span> <span class="nn">sqlalchemy</span> <span class="k">as</span> <span class="nn">sa</span> <span class="kn">from</span> <span class="nn">foo</span> <span class="kn">import</span> <span class="n">ReplaceableObject</span> <span class="n">customer_view</span> <span class="o">=</span> <span class="n">ReplaceableObject</span><span class="p">(</span> <span class="s2">"customer_view"</span><span class="p">,</span> <span class="s2">"SELECT name, order_count, email "</span> <span class="s2">"FROM customer WHERE order_count > 0"</span> <span class="p">)</span> <span class="n">add_customer_sp</span> <span class="o">=</span> <span class="n">ReplaceableObject</span><span class="p">(</span> <span class="s2">"add_customer_sp(name varchar, order_count integer, email varchar)"</span><span class="p">,</span> <span class="sd">"""</span> <span class="sd"> RETURNS integer AS $$</span> <span class="sd"> BEGIN</span> <span class="sd"> insert into customer (name, order_count, email)</span> <span class="sd"> VALUES (in_name, in_order_count, email);</span> <span class="sd"> END;</span> <span class="sd"> $$ LANGUAGE plpgsql;</span> <span class="sd"> """</span> <span class="p">)</span> <span class="k">def</span> <span class="nf">upgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">replace_view</span><span class="p">(</span><span class="n">customer_view</span><span class="p">,</span> <span class="n">replaces</span><span class="o">=</span><span class="s2">"28af9800143f.customer_view"</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">replace_sp</span><span class="p">(</span><span class="n">add_customer_sp</span><span class="p">,</span> <span class="n">replaces</span><span class="o">=</span><span class="s2">"28af9800143f.add_customer_sp"</span><span class="p">)</span> <span class="k">def</span> <span class="nf">downgrade</span><span class="p">():</span> <span class="n">op</span><span class="o">.</span><span class="n">replace_view</span><span class="p">(</span><span class="n">customer_view</span><span class="p">,</span> <span class="n">replace_with</span><span class="o">=</span><span class="s2">"28af9800143f.customer_view"</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">replace_sp</span><span class="p">(</span><span class="n">add_customer_sp</span><span class="p">,</span> <span class="n">replace_with</span><span class="o">=</span><span class="s2">"28af9800143f.add_customer_sp"</span><span class="p">)</span> </pre></div> </div> <p>Above, instead of using <code class="docutils literal notranslate"><span class="pre">create_view()</span></code>, <code class="docutils literal notranslate"><span class="pre">create_sp()</span></code>, <code class="docutils literal notranslate"><span class="pre">drop_view()</span></code>, and <code class="docutils literal notranslate"><span class="pre">drop_sp()</span></code> methods, we now use <code class="docutils literal notranslate"><span class="pre">replace_view()</span></code> and <code class="docutils literal notranslate"><span class="pre">replace_sp()</span></code>. The replace operation we’ve built always runs a DROP <em>and</em> a CREATE. Running an upgrade to head we see:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ alembic upgrade head INFO [alembic.runtime.migration] Context impl PostgresqlImpl. INFO [alembic.runtime.migration] Will assume transactional DDL. INFO [sqlalchemy.engine.base.Engine] BEGIN (implicit) INFO [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s INFO [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'} INFO [sqlalchemy.engine.base.Engine] SELECT alembic_version.version_num FROM alembic_version INFO [sqlalchemy.engine.base.Engine] {} INFO [alembic.runtime.migration] Running upgrade 28af9800143f -> 191a2d20b025, add email col INFO [sqlalchemy.engine.base.Engine] ALTER TABLE customer ADD COLUMN email VARCHAR INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='191a2d20b025' WHERE alembic_version.version_num = '28af9800143f' INFO [sqlalchemy.engine.base.Engine] {} INFO [alembic.runtime.migration] Running upgrade 191a2d20b025 -> 199028bf9856, update views/sp INFO [sqlalchemy.engine.base.Engine] DROP VIEW customer_view INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] CREATE VIEW customer_view AS SELECT name, order_count, email FROM customer WHERE order_count > 0 INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] DROP FUNCTION add_customer_sp(name varchar, order_count integer) INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] CREATE FUNCTION add_customer_sp(name varchar, order_count integer, email varchar) RETURNS integer AS $$ BEGIN insert into customer (name, order_count, email) VALUES (in_name, in_order_count, email); END; $$ LANGUAGE plpgsql; INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='199028bf9856' WHERE alembic_version.version_num = '191a2d20b025' INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] COMMIT </pre></div> </div> <p>After adding our new <code class="docutils literal notranslate"><span class="pre">email</span></code> column, we see that both <code class="docutils literal notranslate"><span class="pre">customer_view</span></code> and <code class="docutils literal notranslate"><span class="pre">add_customer_sp()</span></code> are dropped before the new version is created. If we downgrade back to the old version, we see the old version of these recreated again within the downgrade for this migration:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ alembic downgrade 28af9800143 INFO [alembic.runtime.migration] Context impl PostgresqlImpl. INFO [alembic.runtime.migration] Will assume transactional DDL. INFO [sqlalchemy.engine.base.Engine] BEGIN (implicit) INFO [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s INFO [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'} INFO [sqlalchemy.engine.base.Engine] SELECT alembic_version.version_num FROM alembic_version INFO [sqlalchemy.engine.base.Engine] {} INFO [alembic.runtime.migration] Running downgrade 199028bf9856 -> 191a2d20b025, update views/sp INFO [sqlalchemy.engine.base.Engine] DROP VIEW customer_view INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] CREATE VIEW customer_view AS SELECT name, order_count FROM customer WHERE order_count > 0 INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] DROP FUNCTION add_customer_sp(name varchar, order_count integer, email varchar) INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] CREATE FUNCTION add_customer_sp(name varchar, order_count integer) RETURNS integer AS $$ BEGIN insert into customer (name, order_count) VALUES (in_name, in_order_count); END; $$ LANGUAGE plpgsql; INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='191a2d20b025' WHERE alembic_version.version_num = '199028bf9856' INFO [sqlalchemy.engine.base.Engine] {} INFO [alembic.runtime.migration] Running downgrade 191a2d20b025 -> 28af9800143f, add email col INFO [sqlalchemy.engine.base.Engine] ALTER TABLE customer DROP COLUMN email INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='28af9800143f' WHERE alembic_version.version_num = '191a2d20b025' INFO [sqlalchemy.engine.base.Engine] {} INFO [sqlalchemy.engine.base.Engine] COMMIT </pre></div> </div> </section> </section> <section id="rudimental-schema-level-multi-tenancy-for-postgresql-databases"> <span id="cookbook-postgresql-multi-tenancy"></span><h2>Rudimental Schema-Level Multi Tenancy for PostgreSQL Databases<a class="headerlink" href="#rudimental-schema-level-multi-tenancy-for-postgresql-databases" title="Permalink to this headline">¶</a></h2> <p><strong>Multi tenancy</strong> refers to an application that accommodates for many clients simultaneously. Within the scope of a database migrations tool, multi-tenancy typically refers to the practice of maintaining multiple, identical databases where each database is assigned to one client.</p> <p>Alembic does not currently have explicit multi-tenant support; typically, the approach must involve running Alembic multiple times against different database URLs.</p> <p>One common approach to multi-tenancy, particularly on the PostgreSQL database, is to install tenants within <strong>individual PostgreSQL schemas</strong>. When using PostgreSQL’s schemas, a special variable <code class="docutils literal notranslate"><span class="pre">search_path</span></code> is offered that is intended to assist with targeting of different schemas.</p> <div class="admonition note"> <p class="admonition-title">Note</p> <p>SQLAlchemy includes a system of directing a common set of <code class="docutils literal notranslate"><span class="pre">Table</span></code> metadata to many schemas called <a class="reference external" href="https://docs.sqlalchemy.org/core/connections.html#translation-of-schema-names">schema_translate_map</a>. Alembic at the time of this writing lacks adequate support for this feature. The recipe below should be considered <strong>interim</strong> until Alembic has more first-class support for schema-level multi-tenancy.</p> </div> <p>The recipe below can be altered for flexibility. The primary purpose of this recipe is to illustrate how to point the Alembic process towards one PostgreSQL schema or another.</p> <ol class="arabic"> <li><p>The model metadata used as the target for autogenerate must not include any schema name for tables; the schema must be non-present or set to <code class="docutils literal notranslate"><span class="pre">None</span></code>. Otherwise, Alembic autogenerate will still attempt to compare and render tables in terms of this schema:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">A</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span> <span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'a'</span> <span class="nb">id</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="n">data</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">UnicodeText</span><span class="p">())</span> <span class="n">foo</span> <span class="o">=</span> <span class="n">Column</span><span class="p">(</span><span class="n">Integer</span><span class="p">)</span> <span class="n">__table_args__</span> <span class="o">=</span> <span class="p">{</span> <span class="s2">"schema"</span><span class="p">:</span> <span class="kc">None</span> <span class="p">}</span> </pre></div> </div> </li> <li><p>The <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.configure.params.include_schemas" title="alembic.runtime.environment.EnvironmentContext.configure"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">EnvironmentContext.configure.include_schemas</span></code></a> flag must also be False or not included.</p></li> <li><p>The “tenant” will be a schema name passed to Alembic using the “-x” flag. In <code class="docutils literal notranslate"><span class="pre">env.py</span></code> an approach like the following allows <code class="docutils literal notranslate"><span class="pre">-xtenant=some_schema</span></code> to be supported by making use of <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.get_x_argument" title="alembic.runtime.environment.EnvironmentContext.get_x_argument"><code class="xref py py-meth docutils literal notranslate"><span class="pre">EnvironmentContext.get_x_argument()</span></code></a>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">run_migrations_online</span><span class="p">():</span> <span class="n">connectable</span> <span class="o">=</span> <span class="n">engine_from_config</span><span class="p">(</span> <span class="n">config</span><span class="o">.</span><span class="n">get_section</span><span class="p">(</span><span class="n">config</span><span class="o">.</span><span class="n">config_ini_section</span><span class="p">),</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"sqlalchemy."</span><span class="p">,</span> <span class="n">poolclass</span><span class="o">=</span><span class="n">pool</span><span class="o">.</span><span class="n">NullPool</span><span class="p">,</span> <span class="p">)</span> <span class="n">current_tenant</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">get_x_argument</span><span class="p">(</span><span class="n">as_dictionary</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"tenant"</span><span class="p">)</span> <span class="k">with</span> <span class="n">connectable</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="c1"># set search path on the connection, which ensures that</span> <span class="c1"># PostgreSQL will emit all CREATE / ALTER / DROP statements</span> <span class="c1"># in terms of this schema by default</span> <span class="n">connection</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"set search_path to </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="n">current_tenant</span><span class="p">)</span> <span class="c1"># make use of non-supported SQLAlchemy attribute to ensure</span> <span class="c1"># the dialect reflects tables in terms of the current tenant name</span> <span class="n">connection</span><span class="o">.</span><span class="n">dialect</span><span class="o">.</span><span class="n">default_schema_name</span> <span class="o">=</span> <span class="n">current_tenant</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="n">connection</span><span class="o">=</span><span class="n">connection</span><span class="p">,</span> <span class="n">target_metadata</span><span class="o">=</span><span class="n">target_metadata</span><span class="p">,</span> <span class="p">)</span> <span class="k">with</span> <span class="n">context</span><span class="o">.</span><span class="n">begin_transaction</span><span class="p">():</span> <span class="n">context</span><span class="o">.</span><span class="n">run_migrations</span><span class="p">()</span> </pre></div> </div> <p>The current tenant is set using the PostgreSQL <code class="docutils literal notranslate"><span class="pre">search_path</span></code> variable on the connection. Note above we must employ a <strong>non-supported SQLAlchemy workaround</strong> at the moment which is to hardcode the SQLAlchemy dialect’s default schema name to our target schema.</p> <p>It is also important to note that the above changes <strong>remain on the connection permanently unless reversed explicitly</strong>. If the alembic application simply exits above, there is no issue. However if the application attempts to continue using the above connection for other purposes, it may be necessary to reset these variables back to the default, which for PostgreSQL is usually the name “public” however may be different based on configuration.</p> </li> <li><p>Alembic operations will now proceed in terms of whichever schema we pass on the command line. All logged SQL will show no schema, except for reflection operations which will make use of the <code class="docutils literal notranslate"><span class="pre">default_schema_name</span></code> attribute:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>[]$ alembic -x tenant=some_schema revision -m "rev1" --autogenerate </pre></div> </div> </li> <li><p>Since all schemas are to be maintained in sync, autogenerate should be run against only <strong>one</strong> schema, generating new Alembic migration files. Autogenerate migratin operations are then run against <strong>all</strong> schemas.</p></li> </ol> </section> <section id="don-t-generate-empty-migrations-with-autogenerate"> <span id="cookbook-no-empty-migrations"></span><h2>Don’t Generate Empty Migrations with Autogenerate<a class="headerlink" href="#don-t-generate-empty-migrations-with-autogenerate" title="Permalink to this headline">¶</a></h2> <p>A common request is to have the <code class="docutils literal notranslate"><span class="pre">alembic</span> <span class="pre">revision</span> <span class="pre">--autogenerate</span></code> command not actually generate a revision file if no changes to the schema is detected. Using the <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.configure.params.process_revision_directives" title="alembic.runtime.environment.EnvironmentContext.configure"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">EnvironmentContext.configure.process_revision_directives</span></code></a> hook, this is straightforward; place a <code class="docutils literal notranslate"><span class="pre">process_revision_directives</span></code> hook in <a class="reference internal" href="api/runtime.html#alembic.runtime.migration.MigrationContext.configure" title="alembic.runtime.migration.MigrationContext.configure"><code class="xref py py-meth docutils literal notranslate"><span class="pre">MigrationContext.configure()</span></code></a> which removes the single <a class="reference internal" href="api/operations.html#alembic.operations.ops.MigrationScript" title="alembic.operations.ops.MigrationScript"><code class="xref py py-class docutils literal notranslate"><span class="pre">MigrationScript</span></code></a> directive if it is empty of any operations:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">run_migrations_online</span><span class="p">():</span> <span class="c1"># ...</span> <span class="k">def</span> <span class="nf">process_revision_directives</span><span class="p">(</span><span class="n">context</span><span class="p">,</span> <span class="n">revision</span><span class="p">,</span> <span class="n">directives</span><span class="p">):</span> <span class="k">if</span> <span class="n">config</span><span class="o">.</span><span class="n">cmd_opts</span><span class="o">.</span><span class="n">autogenerate</span><span class="p">:</span> <span class="n">script</span> <span class="o">=</span> <span class="n">directives</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">if</span> <span class="n">script</span><span class="o">.</span><span class="n">upgrade_ops</span><span class="o">.</span><span class="n">is_empty</span><span class="p">():</span> <span class="n">directives</span><span class="p">[:]</span> <span class="o">=</span> <span class="p">[]</span> <span class="c1"># connectable = ...</span> <span class="k">with</span> <span class="n">connectable</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="n">connection</span><span class="o">=</span><span class="n">connection</span><span class="p">,</span> <span class="n">target_metadata</span><span class="o">=</span><span class="n">target_metadata</span><span class="p">,</span> <span class="n">process_revision_directives</span><span class="o">=</span><span class="n">process_revision_directives</span> <span class="p">)</span> <span class="k">with</span> <span class="n">context</span><span class="o">.</span><span class="n">begin_transaction</span><span class="p">():</span> <span class="n">context</span><span class="o">.</span><span class="n">run_migrations</span><span class="p">()</span> </pre></div> </div> </section> <section id="don-t-emit-drop-index-when-the-table-is-to-be-dropped-as-well"> <span id="cookbook-dont-emit-drop-index"></span><h2>Don’t emit DROP INDEX when the table is to be dropped as well<a class="headerlink" href="#don-t-emit-drop-index-when-the-table-is-to-be-dropped-as-well" title="Permalink to this headline">¶</a></h2> <p>MySQL may complain when dropping an index that is against a column that also has a foreign key constraint on it. If the table is to be dropped in any case, the DROP INDEX isn’t necessary. This recipe will process the set of autogenerate directives such that all <a class="reference internal" href="api/operations.html#alembic.operations.ops.DropIndexOp" title="alembic.operations.ops.DropIndexOp"><code class="xref py py-class docutils literal notranslate"><span class="pre">DropIndexOp</span></code></a> directives are removed against tables that themselves are to be dropped:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">run_migrations_online</span><span class="p">():</span> <span class="c1"># ...</span> <span class="kn">from</span> <span class="nn">alembic.operations</span> <span class="kn">import</span> <span class="n">ops</span> <span class="k">def</span> <span class="nf">process_revision_directives</span><span class="p">(</span><span class="n">context</span><span class="p">,</span> <span class="n">revision</span><span class="p">,</span> <span class="n">directives</span><span class="p">):</span> <span class="n">script</span> <span class="o">=</span> <span class="n">directives</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="c1"># process both "def upgrade()", "def downgrade()"</span> <span class="k">for</span> <span class="n">directive</span> <span class="ow">in</span> <span class="p">(</span><span class="n">script</span><span class="o">.</span><span class="n">upgrade_ops</span><span class="p">,</span> <span class="n">script</span><span class="o">.</span><span class="n">downgrade_ops</span><span class="p">):</span> <span class="c1"># make a set of tables that are being dropped within</span> <span class="c1"># the migration function</span> <span class="n">tables_dropped</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span> <span class="k">for</span> <span class="n">op</span> <span class="ow">in</span> <span class="n">directive</span><span class="o">.</span><span class="n">ops</span><span class="p">:</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">op</span><span class="p">,</span> <span class="n">ops</span><span class="o">.</span><span class="n">DropTableOp</span><span class="p">):</span> <span class="n">tables_dropped</span><span class="o">.</span><span class="n">add</span><span class="p">((</span><span class="n">op</span><span class="o">.</span><span class="n">table_name</span><span class="p">,</span> <span class="n">op</span><span class="o">.</span><span class="n">schema</span><span class="p">))</span> <span class="c1"># now rewrite the list of "ops" such that DropIndexOp</span> <span class="c1"># is removed for those tables. Needs a recursive function.</span> <span class="n">directive</span><span class="o">.</span><span class="n">ops</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span> <span class="n">_filter_drop_indexes</span><span class="p">(</span><span class="n">directive</span><span class="o">.</span><span class="n">ops</span><span class="p">,</span> <span class="n">tables_dropped</span><span class="p">)</span> <span class="p">)</span> <span class="k">def</span> <span class="nf">_filter_drop_indexes</span><span class="p">(</span><span class="n">directives</span><span class="p">,</span> <span class="n">tables_dropped</span><span class="p">):</span> <span class="c1"># given a set of (tablename, schemaname) to be dropped, filter</span> <span class="c1"># out DropIndexOp from the list of directives and yield the result.</span> <span class="k">for</span> <span class="n">directive</span> <span class="ow">in</span> <span class="n">directives</span><span class="p">:</span> <span class="c1"># ModifyTableOps is a container of ALTER TABLE types of</span> <span class="c1"># commands. process those in place recursively.</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">directive</span><span class="p">,</span> <span class="n">ops</span><span class="o">.</span><span class="n">ModifyTableOps</span><span class="p">)</span> <span class="ow">and</span> \ <span class="p">(</span><span class="n">directive</span><span class="o">.</span><span class="n">table_name</span><span class="p">,</span> <span class="n">directive</span><span class="o">.</span><span class="n">schema</span><span class="p">)</span> <span class="ow">in</span> <span class="n">tables_dropped</span><span class="p">:</span> <span class="n">directive</span><span class="o">.</span><span class="n">ops</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span> <span class="n">_filter_drop_indexes</span><span class="p">(</span><span class="n">directive</span><span class="o">.</span><span class="n">ops</span><span class="p">,</span> <span class="n">tables_dropped</span><span class="p">)</span> <span class="p">)</span> <span class="c1"># if we emptied out the directives, then skip the</span> <span class="c1"># container altogether.</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">directive</span><span class="o">.</span><span class="n">ops</span><span class="p">:</span> <span class="k">continue</span> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">directive</span><span class="p">,</span> <span class="n">ops</span><span class="o">.</span><span class="n">DropIndexOp</span><span class="p">)</span> <span class="ow">and</span> \ <span class="p">(</span><span class="n">directive</span><span class="o">.</span><span class="n">table_name</span><span class="p">,</span> <span class="n">directive</span><span class="o">.</span><span class="n">schema</span><span class="p">)</span> <span class="ow">in</span> <span class="n">tables_dropped</span><span class="p">:</span> <span class="c1"># we found a target DropIndexOp. keep looping</span> <span class="k">continue</span> <span class="c1"># otherwise if not filtered, yield out the directive</span> <span class="k">yield</span> <span class="n">directive</span> <span class="c1"># connectable = ...</span> <span class="k">with</span> <span class="n">connectable</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="n">connection</span><span class="o">=</span><span class="n">connection</span><span class="p">,</span> <span class="n">target_metadata</span><span class="o">=</span><span class="n">target_metadata</span><span class="p">,</span> <span class="n">process_revision_directives</span><span class="o">=</span><span class="n">process_revision_directives</span> <span class="p">)</span> <span class="k">with</span> <span class="n">context</span><span class="o">.</span><span class="n">begin_transaction</span><span class="p">():</span> <span class="n">context</span><span class="o">.</span><span class="n">run_migrations</span><span class="p">()</span> </pre></div> </div> <p>Whereas autogenerate, when dropping two tables with a foreign key and an index, would previously generate something like:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">downgrade</span><span class="p">():</span> <span class="c1"># ### commands auto generated by Alembic - please adjust! ###</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_index</span><span class="p">(</span><span class="n">op</span><span class="o">.</span><span class="n">f</span><span class="p">(</span><span class="s1">'ix_b_aid'</span><span class="p">),</span> <span class="n">table_name</span><span class="o">=</span><span class="s1">'b'</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_table</span><span class="p">(</span><span class="s1">'b'</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_table</span><span class="p">(</span><span class="s1">'a'</span><span class="p">)</span> <span class="c1"># ### end Alembic commands ###</span> </pre></div> </div> <p>With the above rewriter, it generates as:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">downgrade</span><span class="p">():</span> <span class="c1"># ### commands auto generated by Alembic - please adjust! ###</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_table</span><span class="p">(</span><span class="s1">'b'</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">drop_table</span><span class="p">(</span><span class="s1">'a'</span><span class="p">)</span> <span class="c1"># ### end Alembic commands ###</span> </pre></div> </div> </section> <section id="don-t-generate-any-drop-table-directives-with-autogenerate"> <h2>Don’t generate any DROP TABLE directives with autogenerate<a class="headerlink" href="#don-t-generate-any-drop-table-directives-with-autogenerate" title="Permalink to this headline">¶</a></h2> <p>When running autogenerate against a database that has existing tables outside of the application’s autogenerated metadata, it may be desirable to prevent autogenerate from considering any of those existing tables to be dropped. This will prevent autogenerate from detecting tables removed from the local metadata as well however this is only a small caveat.</p> <p>The most direct way to achieve this using the <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.configure.params.include_object" title="alembic.runtime.environment.EnvironmentContext.configure"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">EnvironmentContext.configure.include_object</span></code></a> hook. There is no need to hardcode a fixed “whitelist” of table names; the hook gives enough information in the given arguments to determine if a particular table name is not part of the local <code class="xref py py-class docutils literal notranslate"><span class="pre">MetaData</span></code> being autogenerated, by checking first that the type of object is <code class="docutils literal notranslate"><span class="pre">"table"</span></code>, then that <code class="docutils literal notranslate"><span class="pre">reflected</span></code> is <code class="docutils literal notranslate"><span class="pre">True</span></code>, indicating this table name is from the local database connection, not the <code class="xref py py-class docutils literal notranslate"><span class="pre">MetaData</span></code>, and finally that <code class="docutils literal notranslate"><span class="pre">compare_to</span></code> is <code class="docutils literal notranslate"><span class="pre">None</span></code>, indicating autogenerate is not comparing this <code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code> to any <code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code> in the local <code class="xref py py-class docutils literal notranslate"><span class="pre">MetaData</span></code> collection:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># in env.py</span> <span class="k">def</span> <span class="nf">include_object</span><span class="p">(</span><span class="nb">object</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">type_</span><span class="p">,</span> <span class="n">reflected</span><span class="p">,</span> <span class="n">compare_to</span><span class="p">):</span> <span class="k">if</span> <span class="n">type_</span> <span class="o">==</span> <span class="s2">"table"</span> <span class="ow">and</span> <span class="n">reflected</span> <span class="ow">and</span> <span class="n">compare_to</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span> <span class="k">return</span> <span class="kc">False</span> <span class="k">else</span><span class="p">:</span> <span class="k">return</span> <span class="kc">True</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="c1"># ...</span> <span class="n">include_object</span> <span class="o">=</span> <span class="n">include_object</span> <span class="p">)</span> </pre></div> </div> </section> <section id="apply-custom-sorting-to-table-columns-within-create-table"> <span id="cookbook-custom-sorting-create-table"></span><h2>Apply Custom Sorting to Table Columns within CREATE TABLE<a class="headerlink" href="#apply-custom-sorting-to-table-columns-within-create-table" title="Permalink to this headline">¶</a></h2> <p>This example illustrates use of the <a class="reference internal" href="api/autogenerate.html#alembic.autogenerate.rewriter.Rewriter" title="alembic.autogenerate.rewriter.Rewriter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Rewriter</span></code></a> object introduced at <a class="reference internal" href="api/autogenerate.html#autogen-rewriter"><span class="std std-ref">Fine-Grained Autogenerate Generation with Rewriters</span></a>. While the rewriter grants access to the individual <a class="reference internal" href="api/operations.html#alembic.operations.ops.MigrateOperation" title="alembic.operations.ops.MigrateOperation"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.MigrateOperation</span></code></a> objects, there are sometimes some special techniques required to get around some structural limitations that are present.</p> <p>One is when trying to reorganize the order of columns in a table within a <a class="reference internal" href="api/operations.html#alembic.operations.ops.CreateTableOp" title="alembic.operations.ops.CreateTableOp"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.CreateTableOp</span></code></a> directive. This directive, when generated by autogenerate, actually holds onto the original <code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code> object as the source of its information, so attempting to reorder the <code class="xref py py-attr docutils literal notranslate"><span class="pre">ops.CreateTableOp.columns</span></code> collection will usually have no effect. Instead, a new <a class="reference internal" href="api/operations.html#alembic.operations.ops.CreateTableOp" title="alembic.operations.ops.CreateTableOp"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.CreateTableOp</span></code></a> object may be constructed with the new ordering. However, a second issue is that the <code class="xref py py-class docutils literal notranslate"><span class="pre">Column</span></code> objects inside will already be associated with the <code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code> that is from the model being autogenerated, meaning they can’t be reassigned directly to a new <code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code>. To get around this, we can copy all the columns and constraints using methods like <code class="xref py py-meth docutils literal notranslate"><span class="pre">Column.copy()</span></code>.</p> <p>Below we use <a class="reference internal" href="api/autogenerate.html#alembic.autogenerate.rewriter.Rewriter" title="alembic.autogenerate.rewriter.Rewriter"><code class="xref py py-class docutils literal notranslate"><span class="pre">Rewriter</span></code></a> to create a new <a class="reference internal" href="api/operations.html#alembic.operations.ops.CreateTableOp" title="alembic.operations.ops.CreateTableOp"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.CreateTableOp</span></code></a> directive and to copy the <code class="xref py py-class docutils literal notranslate"><span class="pre">Column</span></code> objects from one into another, copying each column or constraint object and applying a new sorting scheme:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># in env.py</span> <span class="kn">from</span> <span class="nn">alembic.operations</span> <span class="kn">import</span> <span class="n">ops</span> <span class="kn">from</span> <span class="nn">alembic.autogenerate</span> <span class="kn">import</span> <span class="n">rewriter</span> <span class="n">writer</span> <span class="o">=</span> <span class="n">rewriter</span><span class="o">.</span><span class="n">Rewriter</span><span class="p">()</span> <span class="nd">@writer</span><span class="o">.</span><span class="n">rewrites</span><span class="p">(</span><span class="n">ops</span><span class="o">.</span><span class="n">CreateTableOp</span><span class="p">)</span> <span class="k">def</span> <span class="nf">order_columns</span><span class="p">(</span><span class="n">context</span><span class="p">,</span> <span class="n">revision</span><span class="p">,</span> <span class="n">op</span><span class="p">):</span> <span class="n">special_names</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="o">-</span><span class="mi">100</span><span class="p">,</span> <span class="s2">"created_at"</span><span class="p">:</span> <span class="mi">1001</span><span class="p">,</span> <span class="s2">"updated_at"</span><span class="p">:</span> <span class="mi">1002</span><span class="p">}</span> <span class="n">cols_by_key</span> <span class="o">=</span> <span class="p">[</span> <span class="p">(</span> <span class="n">special_names</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">col</span><span class="o">.</span><span class="n">key</span><span class="p">,</span> <span class="n">index</span><span class="p">)</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">col</span><span class="p">,</span> <span class="n">Column</span><span class="p">)</span> <span class="k">else</span> <span class="mi">2000</span><span class="p">,</span> <span class="n">col</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="p">)</span> <span class="k">for</span> <span class="n">index</span><span class="p">,</span> <span class="n">col</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">op</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span> <span class="p">]</span> <span class="n">columns</span> <span class="o">=</span> <span class="p">[</span> <span class="n">col</span> <span class="k">for</span> <span class="n">idx</span><span class="p">,</span> <span class="n">col</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">cols_by_key</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">entry</span><span class="p">:</span> <span class="n">entry</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="p">]</span> <span class="k">return</span> <span class="n">ops</span><span class="o">.</span><span class="n">CreateTableOp</span><span class="p">(</span> <span class="n">op</span><span class="o">.</span><span class="n">table_name</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">op</span><span class="o">.</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">op</span><span class="o">.</span><span class="n">kw</span><span class="p">)</span> <span class="c1"># ...</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="c1"># ...</span> <span class="n">process_revision_directives</span><span class="o">=</span><span class="n">writer</span> <span class="p">)</span> </pre></div> </div> <p>Above, when we apply the <code class="docutils literal notranslate"><span class="pre">writer</span></code> to a table such as:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Table</span><span class="p">(</span> <span class="s2">"my_table"</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"data"</span><span class="p">,</span> <span class="n">String</span><span class="p">(</span><span class="mi">50</span><span class="p">)),</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"created_at"</span><span class="p">,</span> <span class="n">DateTime</span><span class="p">),</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"id"</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"updated_at"</span><span class="p">,</span> <span class="n">DateTime</span><span class="p">),</span> <span class="n">UniqueConstraint</span><span class="p">(</span><span class="s2">"data"</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">"uq_data"</span><span class="p">)</span> <span class="p">)</span> </pre></div> </div> <p>This will render in the autogenerated file as:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">upgrade</span><span class="p">():</span> <span class="c1"># ### commands auto generated by Alembic - please adjust! ###</span> <span class="n">op</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span> <span class="s2">"my_table"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s2">"id"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Integer</span><span class="p">(),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s2">"data"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">String</span><span class="p">(</span><span class="n">length</span><span class="o">=</span><span class="mi">50</span><span class="p">),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s2">"created_at"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">DateTime</span><span class="p">(),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s2">"updated_at"</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">DateTime</span><span class="p">(),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">PrimaryKeyConstraint</span><span class="p">(</span><span class="s2">"id"</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">UniqueConstraint</span><span class="p">(</span><span class="s2">"data"</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">"uq_data"</span><span class="p">),</span> <span class="p">)</span> <span class="c1"># ### end Alembic commands ###</span> </pre></div> </div> </section> <section id="don-t-emit-create-table-statements-for-views"> <h2>Don’t emit CREATE TABLE statements for Views<a class="headerlink" href="#don-t-emit-create-table-statements-for-views" title="Permalink to this headline">¶</a></h2> <p>It is sometimes convenient to create <code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code> instances for views so that they can be queried using normal SQLAlchemy techniques. Unfortunately this causes Alembic to treat them as tables in need of creation and to generate spurious <code class="docutils literal notranslate"><span class="pre">create_table()</span></code> operations. This is easily fixable by flagging such Tables and using the <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.configure.params.include_object" title="alembic.runtime.environment.EnvironmentContext.configure"><code class="xref py py-paramref docutils literal notranslate"><span class="pre">include_object</span></code></a> hook to exclude them:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">my_view</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="s1">'my_view'</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">autoload</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">info</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">is_view</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span> <span class="c1"># Flag this as a view</span> </pre></div> </div> <p>Or, if you use declarative tables:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyView</span><span class="p">(</span><span class="n">Base</span><span class="p">):</span> <span class="n">__tablename__</span> <span class="o">=</span> <span class="s1">'my_view'</span> <span class="n">__table_args__</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'info'</span><span class="p">:</span> <span class="p">{</span><span class="s1">'is_view'</span><span class="p">:</span> <span class="kc">True</span><span class="p">}}</span> <span class="c1"># Flag this as a view</span> </pre></div> </div> <p>Then define <code class="docutils literal notranslate"><span class="pre">include_object</span></code> as:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">include_object</span><span class="p">(</span><span class="nb">object</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">type_</span><span class="p">,</span> <span class="n">reflected</span><span class="p">,</span> <span class="n">compare_to</span><span class="p">):</span> <span class="sd">"""</span> <span class="sd"> Exclude views from Alembic's consideration.</span> <span class="sd"> """</span> <span class="k">return</span> <span class="ow">not</span> <span class="nb">object</span><span class="o">.</span><span class="n">info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'is_view'</span><span class="p">,</span> <span class="kc">False</span><span class="p">)</span> </pre></div> </div> <p>Finally, in <code class="docutils literal notranslate"><span class="pre">env.py</span></code> pass your <code class="docutils literal notranslate"><span class="pre">include_object</span></code> as a keyword argument to <a class="reference internal" href="api/runtime.html#alembic.runtime.environment.EnvironmentContext.configure" title="alembic.runtime.environment.EnvironmentContext.configure"><code class="xref py py-meth docutils literal notranslate"><span class="pre">EnvironmentContext.configure()</span></code></a>.</p> </section> <section id="run-multiple-alembic-environments-from-one-ini-file"> <span id="multiple-environments"></span><h2>Run Multiple Alembic Environments from one .ini file<a class="headerlink" href="#run-multiple-alembic-environments-from-one-ini-file" title="Permalink to this headline">¶</a></h2> <p>Long before Alembic had the “multiple bases” feature described in <a class="reference internal" href="branches.html#multiple-bases"><span class="std std-ref">Working with Multiple Bases</span></a>, projects had a need to maintain more than one Alembic version history in a single project, where these version histories are completely independent of each other and each refer to their own alembic_version table, either across multiple databases, schemas, or namespaces. A simple approach was added to support this, the <code class="docutils literal notranslate"><span class="pre">--name</span></code> flag on the commandline.</p> <p>First, one would create an alembic.ini file of this form:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">DEFAULT</span><span class="p">]</span> <span class="c1"># all defaults shared between environments go here</span> <span class="n">sqlalchemy</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">//</span><span class="n">scott</span><span class="p">:</span><span class="n">tiger</span><span class="nd">@hostname</span><span class="o">/</span><span class="n">mydatabase</span> <span class="p">[</span><span class="n">schema1</span><span class="p">]</span> <span class="c1"># path to env.py and migration scripts for schema1</span> <span class="n">script_location</span> <span class="o">=</span> <span class="n">myproject</span><span class="o">/</span><span class="n">revisions</span><span class="o">/</span><span class="n">schema1</span> <span class="p">[</span><span class="n">schema2</span><span class="p">]</span> <span class="c1"># path to env.py and migration scripts for schema2</span> <span class="n">script_location</span> <span class="o">=</span> <span class="n">myproject</span><span class="o">/</span><span class="n">revisions</span><span class="o">/</span><span class="n">schema2</span> <span class="p">[</span><span class="n">schema3</span><span class="p">]</span> <span class="c1"># path to env.py and migration scripts for schema3</span> <span class="n">script_location</span> <span class="o">=</span> <span class="n">myproject</span><span class="o">/</span><span class="n">revisions</span><span class="o">/</span><span class="n">db2</span> <span class="c1"># this schema uses a different database URL as well</span> <span class="n">sqlalchemy</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">//</span><span class="n">scott</span><span class="p">:</span><span class="n">tiger</span><span class="nd">@hostname</span><span class="o">/</span><span class="n">myotherdatabase</span> </pre></div> </div> <p>Above, in the <code class="docutils literal notranslate"><span class="pre">[DEFAULT]</span></code> section we set up a default database URL. Then we create three sections corresponding to different revision lineages in our project. Each of these directories would have its own <code class="docutils literal notranslate"><span class="pre">env.py</span></code> and set of versioning files. Then when we run the <code class="docutils literal notranslate"><span class="pre">alembic</span></code> command, we simply give it the name of the configuration we want to use:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">alembic</span> <span class="o">--</span><span class="n">name</span> <span class="n">schema2</span> <span class="n">revision</span> <span class="o">-</span><span class="n">m</span> <span class="s2">"new rev for schema 2"</span> <span class="o">--</span><span class="n">autogenerate</span> </pre></div> </div> <p>Above, the <code class="docutils literal notranslate"><span class="pre">alembic</span></code> command makes use of the configuration in <code class="docutils literal notranslate"><span class="pre">[schema2]</span></code>, populated with defaults from the <code class="docutils literal notranslate"><span class="pre">[DEFAULT]</span></code> section.</p> <p>The above approach can be automated by creating a custom front-end to the Alembic commandline as well.</p> </section> <section id="print-python-code-to-generate-particular-database-tables"> <h2>Print Python Code to Generate Particular Database Tables<a class="headerlink" href="#print-python-code-to-generate-particular-database-tables" title="Permalink to this headline">¶</a></h2> <p>Suppose you have a database already, and want to generate some <code class="docutils literal notranslate"><span class="pre">op.create_table()</span></code> and other directives that you’d have in a migration file. How can we automate generating that code? Suppose the database schema looks like (assume MySQL):</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>CREATE TABLE IF NOT EXISTS `users` ( `id` int(11) NOT NULL, KEY `id` (`id`) ); CREATE TABLE IF NOT EXISTS `user_properties` ( `users_id` int(11) NOT NULL, `property_name` varchar(255) NOT NULL, `property_value` mediumtext NOT NULL, UNIQUE KEY `property_name_users_id` (`property_name`,`users_id`), KEY `users_id` (`users_id`), CONSTRAINT `user_properties_ibfk_1` FOREIGN KEY (`users_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8; </pre></div> </div> <p>Using <a class="reference internal" href="api/operations.html#alembic.operations.ops.UpgradeOps" title="alembic.operations.ops.UpgradeOps"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.UpgradeOps</span></code></a>, <a class="reference internal" href="api/operations.html#alembic.operations.ops.CreateTableOp" title="alembic.operations.ops.CreateTableOp"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.CreateTableOp</span></code></a>, and <a class="reference internal" href="api/operations.html#alembic.operations.ops.CreateIndexOp" title="alembic.operations.ops.CreateIndexOp"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.CreateIndexOp</span></code></a>, we create a migration file structure, using <code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code> objects that we get from SQLAlchemy reflection. The structure is passed to <a class="reference internal" href="api/autogenerate.html#alembic.autogenerate.render_python_code" title="alembic.autogenerate.render_python_code"><code class="xref py py-func docutils literal notranslate"><span class="pre">autogenerate.render_python_code()</span></code></a> to produce the Python code for a migration file:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">create_engine</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">MetaData</span><span class="p">,</span> <span class="n">Table</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">autogenerate</span> <span class="kn">from</span> <span class="nn">alembic.operations</span> <span class="kn">import</span> <span class="n">ops</span> <span class="n">e</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"mysql://scott:tiger@localhost/test"</span><span class="p">)</span> <span class="k">with</span> <span class="n">e</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">conn</span><span class="p">:</span> <span class="n">m</span> <span class="o">=</span> <span class="n">MetaData</span><span class="p">()</span> <span class="n">user_table</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="s1">'users'</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">autoload_with</span><span class="o">=</span><span class="n">conn</span><span class="p">)</span> <span class="n">user_property_table</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="s1">'user_properties'</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">autoload_with</span><span class="o">=</span><span class="n">conn</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="n">autogenerate</span><span class="o">.</span><span class="n">render_python_code</span><span class="p">(</span> <span class="n">ops</span><span class="o">.</span><span class="n">UpgradeOps</span><span class="p">(</span> <span class="n">ops</span><span class="o">=</span><span class="p">[</span> <span class="n">ops</span><span class="o">.</span><span class="n">CreateTableOp</span><span class="o">.</span><span class="n">from_table</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">m</span><span class="o">.</span><span class="n">tables</span><span class="o">.</span><span class="n">values</span><span class="p">()</span> <span class="p">]</span> <span class="o">+</span> <span class="p">[</span> <span class="n">ops</span><span class="o">.</span><span class="n">CreateIndexOp</span><span class="o">.</span><span class="n">from_index</span><span class="p">(</span><span class="n">idx</span><span class="p">)</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">m</span><span class="o">.</span><span class="n">tables</span><span class="o">.</span><span class="n">values</span><span class="p">()</span> <span class="k">for</span> <span class="n">idx</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">indexes</span> <span class="p">]</span> <span class="p">))</span> <span class="p">)</span> </pre></div> </div> <p>Output:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># ### commands auto generated by Alembic - please adjust! ###</span> <span class="n">op</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="s1">'users'</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s1">'id'</span><span class="p">,</span> <span class="n">mysql</span><span class="o">.</span><span class="n">INTEGER</span><span class="p">(</span><span class="n">display_width</span><span class="o">=</span><span class="mi">11</span><span class="p">),</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">mysql_default_charset</span><span class="o">=</span><span class="s1">'latin1'</span><span class="p">,</span> <span class="n">mysql_engine</span><span class="o">=</span><span class="s1">'InnoDB'</span> <span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="s1">'user_properties'</span><span class="p">,</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s1">'users_id'</span><span class="p">,</span> <span class="n">mysql</span><span class="o">.</span><span class="n">INTEGER</span><span class="p">(</span><span class="n">display_width</span><span class="o">=</span><span class="mi">11</span><span class="p">),</span> <span class="n">autoincrement</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s1">'property_name'</span><span class="p">,</span> <span class="n">mysql</span><span class="o">.</span><span class="n">VARCHAR</span><span class="p">(</span><span class="n">length</span><span class="o">=</span><span class="mi">255</span><span class="p">),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">Column</span><span class="p">(</span><span class="s1">'property_value'</span><span class="p">,</span> <span class="n">mysql</span><span class="o">.</span><span class="n">MEDIUMTEXT</span><span class="p">(),</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="n">sa</span><span class="o">.</span><span class="n">ForeignKeyConstraint</span><span class="p">([</span><span class="s1">'users_id'</span><span class="p">],</span> <span class="p">[</span><span class="s1">'users.id'</span><span class="p">],</span> <span class="n">name</span><span class="o">=</span><span class="s1">'user_properties_ibfk_1'</span><span class="p">,</span> <span class="n">ondelete</span><span class="o">=</span><span class="s1">'CASCADE'</span><span class="p">),</span> <span class="n">mysql_comment</span><span class="o">=</span><span class="s1">'user properties'</span><span class="p">,</span> <span class="n">mysql_default_charset</span><span class="o">=</span><span class="s1">'utf8'</span><span class="p">,</span> <span class="n">mysql_engine</span><span class="o">=</span><span class="s1">'InnoDB'</span> <span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">create_index</span><span class="p">(</span><span class="s1">'id'</span><span class="p">,</span> <span class="s1">'users'</span><span class="p">,</span> <span class="p">[</span><span class="s1">'id'</span><span class="p">],</span> <span class="n">unique</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">create_index</span><span class="p">(</span><span class="s1">'users_id'</span><span class="p">,</span> <span class="s1">'user_properties'</span><span class="p">,</span> <span class="p">[</span><span class="s1">'users_id'</span><span class="p">],</span> <span class="n">unique</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> <span class="n">op</span><span class="o">.</span><span class="n">create_index</span><span class="p">(</span><span class="s1">'property_name_users_id'</span><span class="p">,</span> <span class="s1">'user_properties'</span><span class="p">,</span> <span class="p">[</span><span class="s1">'property_name'</span><span class="p">,</span> <span class="s1">'users_id'</span><span class="p">],</span> <span class="n">unique</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="c1"># ### end Alembic commands ###</span> </pre></div> </div> </section> <section id="run-alembic-operation-objects-directly-as-in-from-autogenerate"> <h2>Run Alembic Operation Objects Directly (as in from autogenerate)<a class="headerlink" href="#run-alembic-operation-objects-directly-as-in-from-autogenerate" title="Permalink to this headline">¶</a></h2> <p>The <a class="reference internal" href="ops.html#alembic.operations.Operations" title="alembic.operations.Operations"><code class="xref py py-class docutils literal notranslate"><span class="pre">Operations</span></code></a> object has a method known as <a class="reference internal" href="ops.html#alembic.operations.Operations.invoke" title="alembic.operations.Operations.invoke"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Operations.invoke()</span></code></a> that will generically invoke a particular operation object. We can therefore use the <a class="reference internal" href="api/autogenerate.html#alembic.autogenerate.produce_migrations" title="alembic.autogenerate.produce_migrations"><code class="xref py py-func docutils literal notranslate"><span class="pre">autogenerate.produce_migrations()</span></code></a> function to run an autogenerate comparison, get back a <a class="reference internal" href="api/operations.html#alembic.operations.ops.MigrationScript" title="alembic.operations.ops.MigrationScript"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.MigrationScript</span></code></a> structure representing the changes, and with a little bit of insider information we can invoke them directly.</p> <p>The traversal through the <a class="reference internal" href="api/operations.html#alembic.operations.ops.MigrationScript" title="alembic.operations.ops.MigrationScript"><code class="xref py py-class docutils literal notranslate"><span class="pre">ops.MigrationScript</span></code></a> structure is as follows:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">use_batch</span> <span class="o">=</span> <span class="n">engine</span><span class="o">.</span><span class="n">name</span> <span class="o">==</span> <span class="s2">"sqlite"</span> <span class="n">stack</span> <span class="o">=</span> <span class="p">[</span><span class="n">migrations</span><span class="o">.</span><span class="n">upgrade_ops</span><span class="p">]</span> <span class="k">while</span> <span class="n">stack</span><span class="p">:</span> <span class="n">elem</span> <span class="o">=</span> <span class="n">stack</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">if</span> <span class="n">use_batch</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">elem</span><span class="p">,</span> <span class="n">ModifyTableOps</span><span class="p">):</span> <span class="k">with</span> <span class="n">operations</span><span class="o">.</span><span class="n">batch_alter_table</span><span class="p">(</span> <span class="n">elem</span><span class="o">.</span><span class="n">table_name</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">elem</span><span class="o">.</span><span class="n">schema</span> <span class="p">)</span> <span class="k">as</span> <span class="n">batch_ops</span><span class="p">:</span> <span class="k">for</span> <span class="n">table_elem</span> <span class="ow">in</span> <span class="n">elem</span><span class="o">.</span><span class="n">ops</span><span class="p">:</span> <span class="c1"># work around Alembic issue #753 (fixed in 1.5.0)</span> <span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">table_elem</span><span class="p">,</span> <span class="s2">"column"</span><span class="p">):</span> <span class="n">table_elem</span><span class="o">.</span><span class="n">column</span> <span class="o">=</span> <span class="n">table_elem</span><span class="o">.</span><span class="n">column</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> <span class="n">batch_ops</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="n">table_elem</span><span class="p">)</span> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">elem</span><span class="p">,</span> <span class="s2">"ops"</span><span class="p">):</span> <span class="n">stack</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">elem</span><span class="o">.</span><span class="n">ops</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="c1"># work around Alembic issue #753 (fixed in 1.5.0)</span> <span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">elem</span><span class="p">,</span> <span class="s2">"column"</span><span class="p">):</span> <span class="n">elem</span><span class="o">.</span><span class="n">column</span> <span class="o">=</span> <span class="n">elem</span><span class="o">.</span><span class="n">column</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> <span class="n">operations</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="n">elem</span><span class="p">)</span> </pre></div> </div> <p>Above, we detect elements that have a collection of operations by looking for the <code class="docutils literal notranslate"><span class="pre">.ops</span></code> attribute. A check for <a class="reference internal" href="api/operations.html#alembic.operations.ops.ModifyTableOps" title="alembic.operations.ops.ModifyTableOps"><code class="xref py py-class docutils literal notranslate"><span class="pre">ModifyTableOps</span></code></a> allows us to use a batch context if we are supporting that. Finally there’s a workaround for an Alembic issue that exists for SQLAlchemy 1.3.20 and greater combined with Alembic older than 1.5.</p> <p>A full example follows. The overall setup here is copied from the example at <a class="reference internal" href="api/autogenerate.html#alembic.autogenerate.compare_metadata" title="alembic.autogenerate.compare_metadata"><code class="xref py py-func docutils literal notranslate"><span class="pre">autogenerate.compare_metadata()</span></code></a>:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">Column</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">create_engine</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">Integer</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">MetaData</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">String</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">Table</span> <span class="kn">from</span> <span class="nn">alembic.autogenerate</span> <span class="kn">import</span> <span class="n">produce_migrations</span> <span class="kn">from</span> <span class="nn">alembic.migration</span> <span class="kn">import</span> <span class="n">MigrationContext</span> <span class="kn">from</span> <span class="nn">alembic.operations</span> <span class="kn">import</span> <span class="n">Operations</span> <span class="kn">from</span> <span class="nn">alembic.operations.ops</span> <span class="kn">import</span> <span class="n">ModifyTableOps</span> <span class="n">engine</span> <span class="o">=</span> <span class="n">create_engine</span><span class="p">(</span><span class="s2">"sqlite://"</span><span class="p">,</span> <span class="n">echo</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="k">with</span> <span class="n">engine</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">conn</span><span class="p">:</span> <span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span> <span class="sd">"""</span> <span class="sd"> create table foo (</span> <span class="sd"> id integer not null primary key,</span> <span class="sd"> old_data varchar(50),</span> <span class="sd"> x integer</span> <span class="sd"> )"""</span> <span class="p">)</span> <span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span> <span class="sd">"""</span> <span class="sd"> create table bar (</span> <span class="sd"> data varchar(50)</span> <span class="sd"> )"""</span> <span class="p">)</span> <span class="n">metadata</span> <span class="o">=</span> <span class="n">MetaData</span><span class="p">()</span> <span class="n">Table</span><span class="p">(</span> <span class="s2">"foo"</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"id"</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">primary_key</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"data"</span><span class="p">,</span> <span class="n">Integer</span><span class="p">),</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"x"</span><span class="p">,</span> <span class="n">Integer</span><span class="p">,</span> <span class="n">nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">),</span> <span class="p">)</span> <span class="n">Table</span><span class="p">(</span><span class="s2">"bat"</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">Column</span><span class="p">(</span><span class="s2">"info"</span><span class="p">,</span> <span class="n">String</span><span class="p">(</span><span class="mi">100</span><span class="p">)))</span> <span class="n">mc</span> <span class="o">=</span> <span class="n">MigrationContext</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span><span class="n">engine</span><span class="o">.</span><span class="n">connect</span><span class="p">())</span> <span class="n">migrations</span> <span class="o">=</span> <span class="n">produce_migrations</span><span class="p">(</span><span class="n">mc</span><span class="p">,</span> <span class="n">metadata</span><span class="p">)</span> <span class="n">operations</span> <span class="o">=</span> <span class="n">Operations</span><span class="p">(</span><span class="n">mc</span><span class="p">)</span> <span class="n">use_batch</span> <span class="o">=</span> <span class="n">engine</span><span class="o">.</span><span class="n">name</span> <span class="o">==</span> <span class="s2">"sqlite"</span> <span class="n">stack</span> <span class="o">=</span> <span class="p">[</span><span class="n">migrations</span><span class="o">.</span><span class="n">upgrade_ops</span><span class="p">]</span> <span class="k">while</span> <span class="n">stack</span><span class="p">:</span> <span class="n">elem</span> <span class="o">=</span> <span class="n">stack</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">if</span> <span class="n">use_batch</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">elem</span><span class="p">,</span> <span class="n">ModifyTableOps</span><span class="p">):</span> <span class="k">with</span> <span class="n">operations</span><span class="o">.</span><span class="n">batch_alter_table</span><span class="p">(</span> <span class="n">elem</span><span class="o">.</span><span class="n">table_name</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">elem</span><span class="o">.</span><span class="n">schema</span> <span class="p">)</span> <span class="k">as</span> <span class="n">batch_ops</span><span class="p">:</span> <span class="k">for</span> <span class="n">table_elem</span> <span class="ow">in</span> <span class="n">elem</span><span class="o">.</span><span class="n">ops</span><span class="p">:</span> <span class="c1"># work around Alembic issue #753 (fixed in 1.5.0)</span> <span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">table_elem</span><span class="p">,</span> <span class="s2">"column"</span><span class="p">):</span> <span class="n">table_elem</span><span class="o">.</span><span class="n">column</span> <span class="o">=</span> <span class="n">table_elem</span><span class="o">.</span><span class="n">column</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> <span class="n">batch_ops</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="n">table_elem</span><span class="p">)</span> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">elem</span><span class="p">,</span> <span class="s2">"ops"</span><span class="p">):</span> <span class="n">stack</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">elem</span><span class="o">.</span><span class="n">ops</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="c1"># work around Alembic issue #753 (fixed in 1.5.0)</span> <span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">elem</span><span class="p">,</span> <span class="s2">"column"</span><span class="p">):</span> <span class="n">elem</span><span class="o">.</span><span class="n">column</span> <span class="o">=</span> <span class="n">elem</span><span class="o">.</span><span class="n">column</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> <span class="n">operations</span><span class="o">.</span><span class="n">invoke</span><span class="p">(</span><span class="n">elem</span><span class="p">)</span> </pre></div> </div> </section> <section id="test-current-database-revision-is-at-head-s"> <h2>Test current database revision is at head(s)<a class="headerlink" href="#test-current-database-revision-is-at-head-s" title="Permalink to this headline">¶</a></h2> <p>A recipe to determine if a database schema is up to date in terms of applying Alembic migrations. May be useful for test or installation suites to determine if the target database is up to date. Makes use of the <a class="reference internal" href="api/runtime.html#alembic.runtime.migration.MigrationContext.get_current_heads" title="alembic.runtime.migration.MigrationContext.get_current_heads"><code class="xref py py-meth docutils literal notranslate"><span class="pre">MigrationContext.get_current_heads()</span></code></a> as well as <a class="reference internal" href="api/script.html#alembic.script.ScriptDirectory.get_heads" title="alembic.script.ScriptDirectory.get_heads"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ScriptDirectory.get_heads()</span></code></a> methods so that it accommodates for a branched revision tree:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">config</span><span class="p">,</span> <span class="n">script</span> <span class="kn">from</span> <span class="nn">alembic.runtime</span> <span class="kn">import</span> <span class="n">migration</span> <span class="kn">from</span> <span class="nn">sqlalchemy</span> <span class="kn">import</span> <span class="n">engine</span> <span class="k">def</span> <span class="nf">check_current_head</span><span class="p">(</span><span class="n">alembic_cfg</span><span class="p">,</span> <span class="n">connectable</span><span class="p">):</span> <span class="c1"># type: (config.Config, engine.Engine) -> bool</span> <span class="n">directory</span> <span class="o">=</span> <span class="n">script</span><span class="o">.</span><span class="n">ScriptDirectory</span><span class="o">.</span><span class="n">from_config</span><span class="p">(</span><span class="n">alembic_cfg</span><span class="p">)</span> <span class="k">with</span> <span class="n">connectable</span><span class="o">.</span><span class="n">begin</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="n">context</span> <span class="o">=</span> <span class="n">migration</span><span class="o">.</span><span class="n">MigrationContext</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span><span class="n">connection</span><span class="p">)</span> <span class="k">return</span> <span class="nb">set</span><span class="p">(</span><span class="n">context</span><span class="o">.</span><span class="n">get_current_heads</span><span class="p">())</span> <span class="o">==</span> <span class="nb">set</span><span class="p">(</span><span class="n">directory</span><span class="o">.</span><span class="n">get_heads</span><span class="p">())</span> <span class="n">e</span> <span class="o">=</span> <span class="n">engine</span><span class="o">.</span><span class="n">create_engine</span><span class="p">(</span><span class="s2">"mysql://scott:tiger@localhost/test"</span><span class="p">,</span> <span class="n">echo</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="n">cfg</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="n">Config</span><span class="p">(</span><span class="s2">"alembic.ini"</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="n">check_current_head</span><span class="p">(</span><span class="n">cfg</span><span class="p">,</span> <span class="n">e</span><span class="p">))</span> </pre></div> </div> <div class="admonition seealso"> <p class="admonition-title">See also</p> <p><a class="reference internal" href="api/runtime.html#alembic.runtime.migration.MigrationContext.get_current_heads" title="alembic.runtime.migration.MigrationContext.get_current_heads"><code class="xref py py-meth docutils literal notranslate"><span class="pre">MigrationContext.get_current_heads()</span></code></a></p> <p><a class="reference internal" href="api/script.html#alembic.script.ScriptDirectory.get_heads" title="alembic.script.ScriptDirectory.get_heads"><code class="xref py py-meth docutils literal notranslate"><span class="pre">ScriptDirectory.get_heads()</span></code></a></p> </div> </section> <section id="using-asyncio-with-alembic"> <span id="asyncio-recipe"></span><h2>Using Asyncio with Alembic<a class="headerlink" href="#using-asyncio-with-alembic" title="Permalink to this headline">¶</a></h2> <p>SQLAlchemy version 1.4 introduced experimental support for asyncio, allowing use of most of its interface from async applications. Alembic currently does not provide an async api directly, but it can use an use SQLAlchemy Async engine to run the migrations and autogenerate.</p> <p>New configurations can use the template “async” to bootstrap an environment which can be used with async DBAPI like asyncpg, running the command:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">alembic</span> <span class="n">init</span> <span class="o">-</span><span class="n">t</span> <span class="k">async</span> <span class="o"><</span><span class="n">script_directory_here</span><span class="o">></span> </pre></div> </div> <p>Existing configurations can be updated to use an async DBAPI by updating the <code class="docutils literal notranslate"><span class="pre">env.py</span></code> file that’s used by Alembic to start its operations. In particular only <code class="docutils literal notranslate"><span class="pre">run_migrations_online</span></code> will need to be updated to be something like the example below:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">asyncio</span> <span class="kn">from</span> <span class="nn">sqlalchemy.ext.asyncio</span> <span class="kn">import</span> <span class="n">async_engine_from_config</span> <span class="c1"># ... no change required to the rest of the code</span> <span class="k">def</span> <span class="nf">do_run_migrations</span><span class="p">(</span><span class="n">connection</span><span class="p">):</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span><span class="n">connection</span><span class="o">=</span><span class="n">connection</span><span class="p">,</span> <span class="n">target_metadata</span><span class="o">=</span><span class="n">target_metadata</span><span class="p">)</span> <span class="k">with</span> <span class="n">context</span><span class="o">.</span><span class="n">begin_transaction</span><span class="p">():</span> <span class="n">context</span><span class="o">.</span><span class="n">run_migrations</span><span class="p">()</span> <span class="k">async</span> <span class="k">def</span> <span class="nf">run_migrations_online</span><span class="p">():</span> <span class="sd">"""Run migrations in 'online' mode.</span> <span class="sd"> In this scenario we need to create an Engine</span> <span class="sd"> and associate a connection with the context.</span> <span class="sd"> """</span> <span class="n">connectable</span> <span class="o">=</span> <span class="n">async_engine_from_config</span><span class="p">(</span> <span class="n">config</span><span class="o">.</span><span class="n">get_section</span><span class="p">(</span><span class="n">config</span><span class="o">.</span><span class="n">config_ini_section</span><span class="p">),</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"sqlalchemy."</span><span class="p">,</span> <span class="n">poolclass</span><span class="o">=</span><span class="n">pool</span><span class="o">.</span><span class="n">NullPool</span><span class="p">,</span> <span class="p">)</span> <span class="k">async</span> <span class="k">with</span> <span class="n">connectable</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="k">await</span> <span class="n">connection</span><span class="o">.</span><span class="n">run_sync</span><span class="p">(</span><span class="n">do_run_migrations</span><span class="p">)</span> <span class="k">await</span> <span class="n">connectable</span><span class="o">.</span><span class="n">dispose</span><span class="p">()</span> <span class="k">if</span> <span class="n">context</span><span class="o">.</span><span class="n">is_offline_mode</span><span class="p">():</span> <span class="n">run_migrations_offline</span><span class="p">()</span> <span class="k">else</span><span class="p">:</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">run_migrations_online</span><span class="p">())</span> </pre></div> </div> <p>An async application can also interact with the Alembic api directly by using the SQLAlchemy <code class="docutils literal notranslate"><span class="pre">run_sync</span></code> method to adapt the non-async api of Alembic to an async consumer.</p> <section id="programmatic-api-use-connection-sharing-with-asyncio"> <span id="connection-sharing-plus-asyncio"></span><h3>Programmatic API use (connection sharing) With Asyncio<a class="headerlink" href="#programmatic-api-use-connection-sharing-with-asyncio" title="Permalink to this headline">¶</a></h3> <p>Combining the examples of <a class="reference internal" href="#connection-sharing"><span class="std std-ref">Sharing a Connection across one or more programmatic migration commands</span></a> with <a class="reference internal" href="#asyncio-recipe"><span class="std std-ref">Using Asyncio with Alembic</span></a> together, and <code class="docutils literal notranslate"><span class="pre">env.py</span></code> as follows works:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">asyncio</span> <span class="kn">from</span> <span class="nn">sqlalchemy.ext.asyncio</span> <span class="kn">import</span> <span class="n">async_engine_from_config</span> <span class="c1"># ... no change required to the rest of the code</span> <span class="k">def</span> <span class="nf">do_run_migrations</span><span class="p">(</span><span class="n">connection</span><span class="p">):</span> <span class="n">context</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span><span class="n">connection</span><span class="o">=</span><span class="n">connection</span><span class="p">,</span> <span class="n">target_metadata</span><span class="o">=</span><span class="n">target_metadata</span><span class="p">)</span> <span class="k">with</span> <span class="n">context</span><span class="o">.</span><span class="n">begin_transaction</span><span class="p">():</span> <span class="n">context</span><span class="o">.</span><span class="n">run_migrations</span><span class="p">()</span> <span class="k">async</span> <span class="k">def</span> <span class="nf">run_async_migrations</span><span class="p">():</span> <span class="n">connectable</span> <span class="o">=</span> <span class="n">async_engine_from_config</span><span class="p">(</span> <span class="n">config</span><span class="o">.</span><span class="n">get_section</span><span class="p">(</span><span class="n">config</span><span class="o">.</span><span class="n">config_ini_section</span><span class="p">),</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"sqlalchemy."</span><span class="p">,</span> <span class="n">poolclass</span><span class="o">=</span><span class="n">pool</span><span class="o">.</span><span class="n">NullPool</span><span class="p">,</span> <span class="p">)</span> <span class="k">async</span> <span class="k">with</span> <span class="n">connectable</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">as</span> <span class="n">connection</span><span class="p">:</span> <span class="k">await</span> <span class="n">connection</span><span class="o">.</span><span class="n">run_sync</span><span class="p">(</span><span class="n">do_run_migrations</span><span class="p">)</span> <span class="k">await</span> <span class="n">connectable</span><span class="o">.</span><span class="n">dispose</span><span class="p">()</span> <span class="k">def</span> <span class="nf">run_migrations_online</span><span class="p">():</span> <span class="sd">"""Run migrations in 'online' mode.</span> <span class="sd"> In this scenario we need to create an Engine</span> <span class="sd"> and associate a connection with the context.</span> <span class="sd"> """</span> <span class="n">connectable</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="n">attributes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"connection"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> <span class="k">if</span> <span class="n">connectable</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">run_async_migrations</span><span class="p">())</span> <span class="k">else</span><span class="p">:</span> <span class="n">do_run_migrations</span><span class="p">(</span><span class="n">connectable</span><span class="p">)</span> </pre></div> </div> <p>Above, using an asyncio database URL in <code class="docutils literal notranslate"><span class="pre">alembic.ini</span></code> one can run commands such as <code class="docutils literal notranslate"><span class="pre">alembic</span> <span class="pre">upgrade</span></code> from the command line. Programmatically, the same <code class="docutils literal notranslate"><span class="pre">env.py</span></code> file can be invoked using asyncio as:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">asyncio</span> <span class="kn">from</span> <span class="nn">sqlalchemy.ext.asyncio</span> <span class="kn">import</span> <span class="n">create_async_engine</span> <span class="kn">from</span> <span class="nn">alembic</span> <span class="kn">import</span> <span class="n">command</span><span class="p">,</span> <span class="n">config</span> <span class="k">def</span> <span class="nf">run_upgrade</span><span class="p">(</span><span class="n">connection</span><span class="p">,</span> <span class="n">cfg</span><span class="p">):</span> <span class="n">cfg</span><span class="o">.</span><span class="n">attributes</span><span class="p">[</span><span class="s2">"connection"</span><span class="p">]</span> <span class="o">=</span> <span class="n">connection</span> <span class="n">command</span><span class="o">.</span><span class="n">upgrade</span><span class="p">(</span><span class="n">cfg</span><span class="p">,</span> <span class="s2">"head"</span><span class="p">)</span> <span class="k">async</span> <span class="k">def</span> <span class="nf">run_async_upgrade</span><span class="p">():</span> <span class="n">async_engine</span> <span class="o">=</span> <span class="n">create_async_engine</span><span class="p">(</span><span class="s2">"sqlite+aiosqlite://"</span><span class="p">,</span> <span class="n">echo</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="k">async</span> <span class="k">with</span> <span class="n">async_engine</span><span class="o">.</span><span class="n">begin</span><span class="p">()</span> <span class="k">as</span> <span class="n">conn</span><span class="p">:</span> <span class="k">await</span> <span class="n">conn</span><span class="o">.</span><span class="n">run_sync</span><span class="p">(</span><span class="n">run_upgrade</span><span class="p">,</span> <span class="n">config</span><span class="o">.</span><span class="n">Config</span><span class="p">(</span><span class="s2">"alembic.ini"</span><span class="p">))</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">run_async_upgrade</span><span class="p">())</span> </pre></div> </div> </section> </section> </section> <div class="clearer"></div> </div> </div> </div> <div class="sphinxsidebar" role="navigation" aria-label="main navigation"> <div class="sphinxsidebarwrapper"> <div> <h3><a href="index.html">Table of Contents</a></h3> <ul> <li><a class="reference internal" href="#">Cookbook</a><ul> <li><a class="reference internal" href="#building-an-up-to-date-database-from-scratch">Building an Up to Date Database from Scratch</a></li> <li><a class="reference internal" href="#conditional-migration-elements">Conditional Migration Elements</a></li> <li><a class="reference internal" href="#sharing-a-connection-across-one-or-more-programmatic-migration-commands">Sharing a Connection across one or more programmatic migration commands</a></li> <li><a class="reference internal" href="#replaceable-objects">Replaceable Objects</a><ul> <li><a class="reference internal" href="#the-replaceable-object-structure">The Replaceable Object Structure</a></li> <li><a class="reference internal" href="#create-operations-for-the-target-objects">Create Operations for the Target Objects</a></li> <li><a class="reference internal" href="#create-initial-migrations">Create Initial Migrations</a></li> <li><a class="reference internal" href="#create-revision-migrations">Create Revision Migrations</a></li> </ul> </li> <li><a class="reference internal" href="#rudimental-schema-level-multi-tenancy-for-postgresql-databases">Rudimental Schema-Level Multi Tenancy for PostgreSQL Databases</a></li> <li><a class="reference internal" href="#don-t-generate-empty-migrations-with-autogenerate">Don’t Generate Empty Migrations with Autogenerate</a></li> <li><a class="reference internal" href="#don-t-emit-drop-index-when-the-table-is-to-be-dropped-as-well">Don’t emit DROP INDEX when the table is to be dropped as well</a></li> <li><a class="reference internal" href="#don-t-generate-any-drop-table-directives-with-autogenerate">Don’t generate any DROP TABLE directives with autogenerate</a></li> <li><a class="reference internal" href="#apply-custom-sorting-to-table-columns-within-create-table">Apply Custom Sorting to Table Columns within CREATE TABLE</a></li> <li><a class="reference internal" href="#don-t-emit-create-table-statements-for-views">Don’t emit CREATE TABLE statements for Views</a></li> <li><a class="reference internal" href="#run-multiple-alembic-environments-from-one-ini-file">Run Multiple Alembic Environments from one .ini file</a></li> <li><a class="reference internal" href="#print-python-code-to-generate-particular-database-tables">Print Python Code to Generate Particular Database Tables</a></li> <li><a class="reference internal" href="#run-alembic-operation-objects-directly-as-in-from-autogenerate">Run Alembic Operation Objects Directly (as in from autogenerate)</a></li> <li><a class="reference internal" href="#test-current-database-revision-is-at-head-s">Test current database revision is at head(s)</a></li> <li><a class="reference internal" href="#using-asyncio-with-alembic">Using Asyncio with Alembic</a><ul> <li><a class="reference internal" href="#programmatic-api-use-connection-sharing-with-asyncio">Programmatic API use (connection sharing) With Asyncio</a></li> </ul> </li> </ul> </li> </ul> </div> <div id="searchbox" style="display: none" role="search"> <h3 id="searchlabel">Quick search</h3> <div class="searchformwrapper"> <form class="search" action="search.html" method="get"> <input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/> <input type="submit" value="Go" /> </form> </div> </div> <script>$('#searchbox').show(0);</script> <div> <h4>Previous topic</h4> <p class="topless"><a href="ops.html" title="previous chapter">Operation Reference</a></p> </div> <div> <h4>Next topic</h4> <p class="topless"><a href="api/index.html" title="next chapter">API Details</a></p> </div> </div> </div> <div class="clearer"></div> </div> <div class="related" role="navigation" aria-label="related navigation"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="genindex.html" title="General Index" >index</a></li> <li class="right" > <a href="py-modindex.html" title="Python Module Index" >modules</a> |</li> <li class="right" > <a href="api/index.html" title="API Details" >next</a> |</li> <li class="right" > <a href="ops.html" title="Operation Reference" >previous</a> |</li> <li class="nav-item nav-item-0"><a href="index.html">Alembic 1.8.1 documentation</a> »</li> <li class="nav-item nav-item-this"><a href="">Cookbook</a></li> </ul> </div> <div class="footer" role="contentinfo"> © Copyright 2010-2022, Mike Bayer. Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 4.5.0. </div> </body> </html>