Skip to content

Commit

Permalink
doc update for tag
Browse files Browse the repository at this point in the history
  • Loading branch information
deltars committed Sep 26, 2023
1 parent 909c77b commit 5eff692
Show file tree
Hide file tree
Showing 10 changed files with 153 additions and 45 deletions.
21 changes: 21 additions & 0 deletions python/_sources/usage.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -508,3 +508,24 @@ the method will raise an error.
This method could also be used to insert a new partition if one doesn't already
exist, making this operation idempotent.


Restoring tables
~~~~~~~~~~~~~~~~

.. py:currentmodule:: deltalake.table
Restoring a table will restore delta table to a specified version or datetime. This
operation compares the current state of the delta table with the state to be restored.
And add those missing files into the AddFile actions and add redundant files into
RemoveFile actions. Then commit into a new version.


Use :meth:`DeltaTable.restore` to perform the restore operation. Note that if any other
concurrent operation was performed on the table, restore will fail.

.. code-block:: python
>>> dt = DeltaTable("../rust/tests/data/simple_table")
>>> dt.restore(1)
{'numRemovedFile': 5, 'numRestoredFile': 22}
131 changes: 96 additions & 35 deletions python/api_reference.html

Large diffs are not rendered by default.

13 changes: 12 additions & 1 deletion python/genindex.html
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
delta-rs
</a>
<div class="version">
0.10.1
0.11.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
Expand Down Expand Up @@ -89,6 +89,7 @@ <h1 id="index">Index</h1>
| <a href="#N"><strong>N</strong></a>
| <a href="#O"><strong>O</strong></a>
| <a href="#P"><strong>P</strong></a>
| <a href="#R"><strong>R</strong></a>
| <a href="#S"><strong>S</strong></a>
| <a href="#T"><strong>T</strong></a>
| <a href="#U"><strong>U</strong></a>
Expand Down Expand Up @@ -348,13 +349,23 @@ <h2 id="P">P</h2>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="api_reference.html#deltalake.table.DeltaTable.protocol">protocol() (deltalake.table.DeltaTable method)</a>
</li>
<li><a href="api_reference.html#deltalake.table.ProtocolVersions">ProtocolVersions (class in deltalake.table)</a>
</li>
<li><a href="api_reference.html#deltalake.table.DeltaTable.pyarrow_schema">pyarrow_schema() (deltalake.table.DeltaTable method)</a>
</li>
</ul></td>
</tr></table>

<h2 id="R">R</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="api_reference.html#deltalake.table.DeltaTable.restore">restore() (deltalake.table.DeltaTable method)</a>
</li>
</ul></td>
</tr></table>

<h2 id="S">S</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
Expand Down
2 changes: 1 addition & 1 deletion python/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
delta-rs
</a>
<div class="version">
0.10.1
0.11.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
Expand Down
2 changes: 1 addition & 1 deletion python/installation.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
delta-rs
</a>
<div class="version">
0.10.1
0.11.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
Expand Down
Binary file modified python/objects.inv
Binary file not shown.
2 changes: 1 addition & 1 deletion python/py-modindex.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
delta-rs
</a>
<div class="version">
0.10.1
0.11.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
Expand Down
2 changes: 1 addition & 1 deletion python/search.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
delta-rs
</a>
<div class="version">
0.10.1
0.11.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="#" method="get">
Expand Down
2 changes: 1 addition & 1 deletion python/searchindex.js

Large diffs are not rendered by default.

23 changes: 19 additions & 4 deletions python/usage.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
delta-rs
</a>
<div class="version">
0.10.1
0.11.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
Expand Down Expand Up @@ -67,6 +67,7 @@
</li>
<li class="toctree-l2"><a class="reference internal" href="#writing-delta-tables">Writing Delta Tables</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#overwriting-a-partition">Overwriting a partition</a></li>
<li class="toctree-l3"><a class="reference internal" href="#restoring-tables">Restoring tables</a></li>
</ul>
</li>
</ul>
Expand Down Expand Up @@ -181,9 +182,9 @@ <h3>Custom Storage Backends<a class="headerlink" href="#custom-storage-backends"
<p>While delta always needs its internal storage backend to work and be properly configured, in order to manage the delta log,
it may sometime be advantageous - and is common practice in the arrow world - to customize the storage interface used for
reading the bulk data.</p>
<p><code class="docutils literal notranslate"><span class="pre">deltalake</span></code> will work with any storage compliant with <a class="reference external" href="https://arrow.apache.org/docs/python/generated/pyarrow.fs.FileSystem.html#pyarrow.fs.FileSystem" title="(in Apache Arrow v12.0.1)"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyarrow.fs.FileSystem</span></code></a>, however the root of the filesystem has
<p><code class="docutils literal notranslate"><span class="pre">deltalake</span></code> will work with any storage compliant with <a class="reference external" href="https://arrow.apache.org/docs/python/generated/pyarrow.fs.FileSystem.html#pyarrow.fs.FileSystem" title="(in Apache Arrow v13.0.0)"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyarrow.fs.FileSystem</span></code></a>, however the root of the filesystem has
to be adjusted to point at the root of the Delta table. We can achieve this by wrapping the custom filesystem into
a <a class="reference external" href="https://arrow.apache.org/docs/python/generated/pyarrow.fs.SubTreeFileSystem.html#pyarrow.fs.SubTreeFileSystem" title="(in Apache Arrow v12.0.1)"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyarrow.fs.SubTreeFileSystem</span></code></a>.</p>
a <a class="reference external" href="https://arrow.apache.org/docs/python/generated/pyarrow.fs.SubTreeFileSystem.html#pyarrow.fs.SubTreeFileSystem" title="(in Apache Arrow v13.0.0)"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyarrow.fs.SubTreeFileSystem</span></code></a>.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pyarrow.fs</span> <span class="k">as</span> <span class="nn">fs</span>
<span class="kn">from</span> <span class="nn">deltalake</span> <span class="kn">import</span> <span class="n">DeltaTable</span>

Expand Down Expand Up @@ -499,7 +500,7 @@ <h2>Writing Delta Tables<a class="headerlink" href="#writing-delta-tables" title
<div class="admonition note">
<p class="admonition-title">Note</p>
<p><a class="reference internal" href="api_reference.html#deltalake.write_deltalake" title="deltalake.write_deltalake"><code class="xref py py-func docutils literal notranslate"><span class="pre">write_deltalake()</span></code></a> accepts a Pandas DataFrame, but will convert it to
a Arrow table before writing. See caveats in <a class="reference external" href="https://arrow.apache.org/docs/python/pandas.html" title="(in Apache Arrow v12.0.1)"><span>Pandas Integration</span></a>.</p>
a Arrow table before writing. See caveats in <a class="reference external" href="https://arrow.apache.org/docs/python/pandas.html" title="(in Apache Arrow v13.0.0)"><span>Pandas Integration</span></a>.</p>
</div>
<p>By default, writes create a new table and error if it already exists. This is
controlled by the <code class="docutils literal notranslate"><span class="pre">mode</span></code> parameter, which mirrors the behavior of Spark’s
Expand Down Expand Up @@ -537,6 +538,20 @@ <h3>Overwriting a partition<a class="headerlink" href="#overwriting-a-partition"
<p>This method could also be used to insert a new partition if one doesn’t already
exist, making this operation idempotent.</p>
</section>
<section id="restoring-tables">
<h3>Restoring tables<a class="headerlink" href="#restoring-tables" title="Permalink to this headline"></a></h3>
<p>Restoring a table will restore delta table to a specified version or datetime. This
operation compares the current state of the delta table with the state to be restored.
And add those missing files into the AddFile actions and add redundant files into
RemoveFile actions. Then commit into a new version.</p>
<p>Use <a class="reference internal" href="api_reference.html#deltalake.table.DeltaTable.restore" title="deltalake.table.DeltaTable.restore"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DeltaTable.restore()</span></code></a> to perform the restore operation. Note that if any other
concurrent operation was performed on the table, restore will fail.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">dt</span> <span class="o">=</span> <span class="n">DeltaTable</span><span class="p">(</span><span class="s2">&quot;../rust/tests/data/simple_table&quot;</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">dt</span><span class="o">.</span><span class="n">restore</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="go">{&#39;numRemovedFile&#39;: 5, &#39;numRestoredFile&#39;: 22}</span>
</pre></div>
</div>
</section>
</section>
</section>

Expand Down

0 comments on commit 5eff692

Please sign in to comment.