build based on c23ad00

dev/finding_adversarial_examples/single_image/index.html

@@ -13,5 +13,5 @@
     tightening_options,
     solve_if_predicted_in_targeted
 )
-</code></pre><p>Perturbs <code>input</code> such that the network <code>nn</code> classifies the perturbed image in one of the categories identified by the indexes in <code>target_selection</code>.</p><p>IMPORTANT: </p><ol><li><code>target_selection</code> can include the correct label for <code>input</code>.</li><li>It is possible (particularly with the <code>closest</code> objective) to see &#39;ties&#39; – that is, the perturbed input produces an output with two logits (one corresponding to a target category, and one corresponding to a non-target category) taking on the same maximal value. See the formal definition below for more; in particular, note that &#39;≥&#39; sign.</li></ol><p><code>optimizer</code> is used to build and solve the MIP problem.</p><p>The output dictionary has keys <code>:Model, :PerturbationFamily, :TargetIndexes, :SolveStatus, :Perturbation, :PerturbedInput, :Output</code>. See the <a href="https://nbviewer.jupyter.org/github/vtjeng/MIPVerify.jl/blob/master/examples/03_interpreting_the_output_of_find_adversarial_example.ipynb">tutorial</a> on what individual dictionary entries correspond to.</p><p><em>Formal Definition</em>: If there are a total of <code>n</code> categories, the (perturbed) output vector <code>y=d[:Output]=d[:PerturbedInput] |&gt; nn</code> has length <code>n</code>. If <code>:SolveStatus</code> is feasible, we guarantee that <code>y[j] - y[i] ≥ 0</code> for some <code>j ∈ target_selection</code> and for all <code>i ∉ target_selection</code>.</p><p><strong>Keyword Arguments:</strong></p><ul><li><code>invert_target_selection</code>: Defaults to <code>false</code>. If <code>true</code>, sets <code>target_selection</code> to be its   complement.</li><li><code>pp</code>: Defaults to <code>UnrestrictedPerturbationFamily()</code>. Determines the search space for adversarial examples.</li><li><code>norm_order</code>: Defaults to <code>1</code>. Determines the distance norm used to determine the distance from   the perturbed image to the original. Allowed options are <code>1</code> and <code>Inf</code>, and <code>2</code> if the   <code>optimizer</code> can solve MIQPs.</li><li><code>adversarial_example_objective</code>: Defaults to <code>closest</code>. Allowed options are <code>closest</code> or <code>worst</code>.<ul><li><code>closest</code> finds the closest adversarial example, as measured by the <code>norm_order</code> norm.</li><li><code>worst</code> finds the adversarial example with the <em>largest</em> gap between <code>max(y[j)</code> for <code>j ∈ target_selection</code> and <code>max(y[i])</code> for all <code>i ∉ target_selection</code>.</li></ul></li><li><code>tightening_algorithm</code>: Defaults to <code>mip</code>. Determines how we determine the upper and lower bounds   on input to each nonlinear unit. Allowed options are <code>interval_arithmetic</code>, <code>lp</code>, <code>mip</code>.<ul><li><code>interval_arithmetic</code> looks at the bounds on the output to the previous layer.</li><li><code>lp</code> solves an <code>lp</code> corresponding to the <code>mip</code> formulation, but with any integer constraints relaxed.</li><li><code>mip</code> solves the full <code>mip</code> formulation.</li></ul></li><li><code>tightening_options</code>: Solver-specific options passed to optimizer when used to determine upper   and lower bounds for input to nonlinear units. Note that these are only used if the   <code>tightening_algorithm</code> is <code>lp</code> or <code>mip</code> (no solver is used when <code>interval_arithmetic</code> is used   to compute the bounds). Defaults for Gurobi and HiGHS to a time limit of 20s per solve, with   output suppressed.</li><li><code>solve_if_predicted_in_targeted</code>: Defaults to <code>true</code>. The prediction that <code>nn</code> makes for the   unperturbed <code>input</code> can be determined efficiently. If the predicted index is one of the indexes   in <code>target_selection</code>, we can skip the relatively costly process of building the model for the   MIP problem since we already have an &quot;adversarial example&quot; –- namely, the input itself. We   continue build the model and solve the (trivial) MIP problem if and only if   <code>solve_if_predicted_in_targeted</code> is <code>true</code>.</li></ul></div><a class="docs-sourcelink" target="_blank" href="https://github.com/vtjeng/MIPVerify.jl/blob/2810656489c30208afa54898eb27898e550ea5a0/src/MIPVerify.jl#L42">source</a></section></article><article class="docstring"><header><a class="docstring-binding" id="MIPVerify.frac_correct-Tuple{NeuralNet, MIPVerify.LabelledDataset, Integer}" href="#MIPVerify.frac_correct-Tuple{NeuralNet, MIPVerify.LabelledDataset, Integer}"><code>MIPVerify.frac_correct</code></a> — <span class="docstring-category">Method</span></header><section><div><pre><code class="language-julia hljs">frac_correct(nn, dataset, num_samples)
-</code></pre><p>Returns the fraction of items the neural network correctly classifies of the first <code>num_samples</code> of the provided <code>dataset</code>. If there are fewer than <code>num_samples</code> items, we use all of the available samples.</p><p><strong>Named Arguments:</strong></p><ul><li><code>nn::NeuralNet</code>: The parameters of the neural network.</li><li><code>dataset::LabelledDataset</code>:</li><li><code>num_samples::Integer</code>: Number of samples to use.</li></ul></div><a class="docs-sourcelink" target="_blank" href="https://github.com/vtjeng/MIPVerify.jl/blob/2810656489c30208afa54898eb27898e550ea5a0/src/MIPVerify.jl#L183">source</a></section></article></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../../tutorials/">« Tutorials</a><a class="docs-footer-nextpage" href="../batch_processing/">Batch Processing »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Wednesday 13 March 2024 08:33">Wednesday 13 March 2024</span>. Using Julia version 1.10.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+</code></pre><p>Perturbs <code>input</code> such that the network <code>nn</code> classifies the perturbed image in one of the categories identified by the indexes in <code>target_selection</code>.</p><p>IMPORTANT: </p><ol><li><code>target_selection</code> can include the correct label for <code>input</code>.</li><li>It is possible (particularly with the <code>closest</code> objective) to see &#39;ties&#39; – that is, the perturbed input produces an output with two logits (one corresponding to a target category, and one corresponding to a non-target category) taking on the same maximal value. See the formal definition below for more; in particular, note that &#39;≥&#39; sign.</li></ol><p><code>optimizer</code> is used to build and solve the MIP problem.</p><p>The output dictionary has keys <code>:Model, :PerturbationFamily, :TargetIndexes, :SolveStatus, :Perturbation, :PerturbedInput, :Output</code>. See the <a href="https://nbviewer.jupyter.org/github/vtjeng/MIPVerify.jl/blob/master/examples/03_interpreting_the_output_of_find_adversarial_example.ipynb">tutorial</a> on what individual dictionary entries correspond to.</p><p><em>Formal Definition</em>: If there are a total of <code>n</code> categories, the (perturbed) output vector <code>y=d[:Output]=d[:PerturbedInput] |&gt; nn</code> has length <code>n</code>. If <code>:SolveStatus</code> is feasible, we guarantee that <code>y[j] - y[i] ≥ 0</code> for some <code>j ∈ target_selection</code> and for all <code>i ∉ target_selection</code>.</p><p><strong>Keyword Arguments:</strong></p><ul><li><code>invert_target_selection</code>: Defaults to <code>false</code>. If <code>true</code>, sets <code>target_selection</code> to be its   complement.</li><li><code>pp</code>: Defaults to <code>UnrestrictedPerturbationFamily()</code>. Determines the search space for adversarial examples.</li><li><code>norm_order</code>: Defaults to <code>1</code>. Determines the distance norm used to determine the distance from   the perturbed image to the original. Allowed options are <code>1</code> and <code>Inf</code>, and <code>2</code> if the   <code>optimizer</code> can solve MIQPs.</li><li><code>adversarial_example_objective</code>: Defaults to <code>closest</code>. Allowed options are <code>closest</code> or <code>worst</code>.<ul><li><code>closest</code> finds the closest adversarial example, as measured by the <code>norm_order</code> norm.</li><li><code>worst</code> finds the adversarial example with the <em>largest</em> gap between <code>max(y[j)</code> for <code>j ∈ target_selection</code> and <code>max(y[i])</code> for all <code>i ∉ target_selection</code>.</li></ul></li><li><code>tightening_algorithm</code>: Defaults to <code>mip</code>. Determines how we determine the upper and lower bounds   on input to each nonlinear unit. Allowed options are <code>interval_arithmetic</code>, <code>lp</code>, <code>mip</code>.<ul><li><code>interval_arithmetic</code> looks at the bounds on the output to the previous layer.</li><li><code>lp</code> solves an <code>lp</code> corresponding to the <code>mip</code> formulation, but with any integer constraints relaxed.</li><li><code>mip</code> solves the full <code>mip</code> formulation.</li></ul></li><li><code>tightening_options</code>: Solver-specific options passed to optimizer when used to determine upper   and lower bounds for input to nonlinear units. Note that these are only used if the   <code>tightening_algorithm</code> is <code>lp</code> or <code>mip</code> (no solver is used when <code>interval_arithmetic</code> is used   to compute the bounds). Defaults for Gurobi and HiGHS to a time limit of 20s per solve, with   output suppressed.</li><li><code>solve_if_predicted_in_targeted</code>: Defaults to <code>true</code>. The prediction that <code>nn</code> makes for the   unperturbed <code>input</code> can be determined efficiently. If the predicted index is one of the indexes   in <code>target_selection</code>, we can skip the relatively costly process of building the model for the   MIP problem since we already have an &quot;adversarial example&quot; –- namely, the input itself. We   continue build the model and solve the (trivial) MIP problem if and only if   <code>solve_if_predicted_in_targeted</code> is <code>true</code>.</li></ul></div><a class="docs-sourcelink" target="_blank" href="https://github.com/vtjeng/MIPVerify.jl/blob/c23ad0008b5413740564a533fb50cf32fa77b393/src/MIPVerify.jl#L42">source</a></section></article><article class="docstring"><header><a class="docstring-binding" id="MIPVerify.frac_correct-Tuple{NeuralNet, MIPVerify.LabelledDataset, Integer}" href="#MIPVerify.frac_correct-Tuple{NeuralNet, MIPVerify.LabelledDataset, Integer}"><code>MIPVerify.frac_correct</code></a> — <span class="docstring-category">Method</span></header><section><div><pre><code class="language-julia hljs">frac_correct(nn, dataset, num_samples)
+</code></pre><p>Returns the fraction of items the neural network correctly classifies of the first <code>num_samples</code> of the provided <code>dataset</code>. If there are fewer than <code>num_samples</code> items, we use all of the available samples.</p><p><strong>Named Arguments:</strong></p><ul><li><code>nn::NeuralNet</code>: The parameters of the neural network.</li><li><code>dataset::LabelledDataset</code>:</li><li><code>num_samples::Integer</code>: Number of samples to use.</li></ul></div><a class="docs-sourcelink" target="_blank" href="https://github.com/vtjeng/MIPVerify.jl/blob/c23ad0008b5413740564a533fb50cf32fa77b393/src/MIPVerify.jl#L183">source</a></section></article></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../../tutorials/">« Tutorials</a><a class="docs-footer-nextpage" href="../batch_processing/">Batch Processing »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Friday 15 March 2024 05:23">Friday 15 March 2024</span>. Using Julia version 1.10.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

dev/index.html

@@ -28,4 +28,4 @@
 No variables, no constraints: Error During Test
   Got an exception of type ErrorException outside of a @test
   Invalid Gurobi license
-  ...</code></pre><p><strong>FIX:</strong> The error message indicates that you have not installed your Gurobi license. If it has been installed, the license is saved as a file <code>gurobi.lic</code>, typically in either the <code>/home/ubuntu</code> or <code>opt/gurobi</code> directories.</p><h2 id="Getting-Started"><a class="docs-heading-anchor" href="#Getting-Started">Getting Started</a><a id="Getting-Started-1"></a><a class="docs-heading-anchor-permalink" href="#Getting-Started" title="Permalink"></a></h2><p>The best way to get started is to follow our <a href="https://nbviewer.jupyter.org/github/vtjeng/MIPVerify.jl/blob/master/examples/00_quickstart.ipynb">quickstart tutorial</a>, which demonstrates how to find adversarial examples for a pre-trained example network on the MNIST dataset. Once you&#39;re done with that, you can explore our other tutorials depending on your needs.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="tutorials/">Tutorials »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Wednesday 13 March 2024 08:33">Wednesday 13 March 2024</span>. Using Julia version 1.10.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+  ...</code></pre><p><strong>FIX:</strong> The error message indicates that you have not installed your Gurobi license. If it has been installed, the license is saved as a file <code>gurobi.lic</code>, typically in either the <code>/home/ubuntu</code> or <code>opt/gurobi</code> directories.</p><h2 id="Getting-Started"><a class="docs-heading-anchor" href="#Getting-Started">Getting Started</a><a id="Getting-Started-1"></a><a class="docs-heading-anchor-permalink" href="#Getting-Started" title="Permalink"></a></h2><p>The best way to get started is to follow our <a href="https://nbviewer.jupyter.org/github/vtjeng/MIPVerify.jl/blob/master/examples/00_quickstart.ipynb">quickstart tutorial</a>, which demonstrates how to find adversarial examples for a pre-trained example network on the MNIST dataset. Once you&#39;re done with that, you can explore our other tutorials depending on your needs.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="tutorials/">Tutorials »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Friday 15 March 2024 05:23">Friday 15 March 2024</span>. Using Julia version 1.10.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>