Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert tutorial links to latest #377

Merged
merged 1 commit into from
Sep 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions tutorials/arm_repertoire.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"source": [
"# Learning a Repertoire of Robot Arm Configurations\n",
"\n",
"_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/stable/tutorials.html) for the list of all tutorials and the order in which they should be read._\n",
"_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/latest/tutorials.html) for the list of all tutorials and the order in which they should be read._\n",
"\n",
"In robotic manipulation, [inverse kinematics](https://en.wikipedia.org/wiki/Inverse_kinematics) involves figuring out how to configure the joints of an arm such that the end effector is at a certain position. For instance, in order to pick up a cup, a robot must move its gripper to the cup's position, and in order to catch a ball, a robot must move its hand to where it predicts the ball will be.\n",
"\n",
Expand Down Expand Up @@ -127,9 +127,9 @@
"\n",
"We will use CMA-ME, with the following pyribs components, to search for arm configurations:\n",
"\n",
"- [`CVTArchive`](https://docs.pyribs.org/en/stable/api/ribs.archives.CVTArchive.html): This archive uses a [Centroidal Voronoi Tesselation (CVT)](https://en.wikipedia.org/wiki/Centroidal_Voronoi_tessellation) to divide the measure space into evenly sized cells. It is typically used for high-dimensional measure spaces where the curse of dimensionality prevents one from using [`GridArchive`](https://docs.pyribs.org/en/stable/api/ribs.archives.GridArchive.html), but it works perfectly fine for lower dimensions too.\n",
"- [`EvolutionStartegyEmitter`](https://docs.pyribs.org/en/stable/api/ribs.emitters.EvolutionStrategyEmitter.html): This emitter is used to create the improvement emitter, which originated in the work of [Fontaine et al., 2020](https://arxiv.org/abs/1912.02400). It uses CMA-ES to search for solutions that improve the archive.\n",
"- [`Scheduler`](https://docs.pyribs.org/en/stable/api/ribs.schedulers.Scheduler.html): Binds all the components together and controls how the archive and emitters interact.\n",
"- [`CVTArchive`](https://docs.pyribs.org/en/latest/api/ribs.archives.CVTArchive.html): This archive uses a [Centroidal Voronoi Tesselation (CVT)](https://en.wikipedia.org/wiki/Centroidal_Voronoi_tessellation) to divide the measure space into evenly sized cells. It is typically used for high-dimensional measure spaces where the curse of dimensionality prevents one from using [`GridArchive`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html), but it works perfectly fine for lower dimensions too.\n",
"- [`EvolutionStartegyEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.EvolutionStrategyEmitter.html): This emitter is used to create the improvement emitter, which originated in the work of [Fontaine et al., 2020](https://arxiv.org/abs/1912.02400). It uses CMA-ES to search for solutions that improve the archive.\n",
"- [`Scheduler`](https://docs.pyribs.org/en/latest/api/ribs.schedulers.Scheduler.html): Binds all the components together and controls how the archive and emitters interact.\n",
"\n",
"First, let's create the archive. This line may take a minute or two to run because it initializes the archive, and initializing `CVTArchive` involves using [k-means clustering (Lloyd's algorithm)](https://scikit-learn.org/stable/modules/clustering.html#k-means) to generate the CVT."
]
Expand Down Expand Up @@ -304,7 +304,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Using [`cvt_archive_heatmap`](https://docs.pyribs.org/en/stable/api/ribs.visualize.cvt_archive_heatmap.html) from [`ribs.visualize`](https://docs.pyribs.org/en/stable/api/ribs.visualize.html), we can also plot a heatmap showing all the positions for which we found an arm configuration, as well as the objective value of each configuration (higher is better). As we can see, CMA-ME found a solution for most of the possible positions (the arm can reach anywhere within a circle of radius 12 around its base), but there are still a few gaps."
"Using [`cvt_archive_heatmap`](https://docs.pyribs.org/en/latest/api/ribs.visualize.cvt_archive_heatmap.html) from [`ribs.visualize`](https://docs.pyribs.org/en/latest/api/ribs.visualize.html), we can also plot a heatmap showing all the positions for which we found an arm configuration, as well as the objective value of each configuration (higher is better). As we can see, CMA-ME found a solution for most of the possible positions (the arm can reach anywhere within a circle of radius 12 around its base), but there are still a few gaps."
]
},
{
Expand Down Expand Up @@ -395,7 +395,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We can retrieve `n` random solutions from the archive with [`sample_elites(n)`](https://docs.pyribs.org/en/stable/api/ribs.archives.CVTArchive.html#ribs.archives.CVTArchive.sample_elites). As our objective was to find configurations where the joint angles had small standard deviation from each other, it makes sense that all of our arms look \"smooth\" when visualized, as the joint angles are close to each other."
"We can retrieve `n` random solutions from the archive with [`sample_elites(n)`](https://docs.pyribs.org/en/latest/api/ribs.archives.CVTArchive.html#ribs.archives.CVTArchive.sample_elites). As our objective was to find configurations where the joint angles had small standard deviation from each other, it makes sense that all of our arms look \"smooth\" when visualized, as the joint angles are close to each other."
]
},
{
Expand Down Expand Up @@ -427,9 +427,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also retrieve solutions that reached specific positions with [`retrieve_single`](https://docs.pyribs.org/en/stable/api/ribs.archives.CVTArchive.html#ribs.archives.CVTArchive.retrieve_single). This method will check whether the archive cell with the specified measures contains a solution, and if so, it will return that solution. It looks like there is an arm configuration that can reach position (0,0), so let's see what that looks like.\n",
"We can also retrieve solutions that reached specific positions with [`retrieve_single`](https://docs.pyribs.org/en/latest/api/ribs.archives.CVTArchive.html#ribs.archives.CVTArchive.retrieve_single). This method will check whether the archive cell with the specified measures contains a solution, and if so, it will return that solution. It looks like there is an arm configuration that can reach position (0,0), so let's see what that looks like.\n",
"\n",
"If you want to query a batch of solutions, you can similarly use [`retrieve`](https://docs.pyribs.org/en/stable/api/ribs.archives.CVTArchive.html#ribs.archives.CVTArchive.retrieve)."
"If you want to query a batch of solutions, you can similarly use [`retrieve`](https://docs.pyribs.org/en/latest/api/ribs.archives.CVTArchive.html#ribs.archives.CVTArchive.retrieve)."
]
},
{
Expand Down Expand Up @@ -472,7 +472,7 @@
"\n",
"- Increasing the number of joints (increase `dof` under the Quality Diversity Algorithm Setup section)\n",
"- Using different link lengths (set `link_lengths` in the same section as `dof`)\n",
"- Using different types of emitters, like the `EvolutionStrategyEmitter` with random direction ranking or the [`IsoLineEmitter`](https://docs.pyribs.org/en/stable/api/ribs.emitters.IsoLineEmitter.html)"
"- Using different types of emitters, like the `EvolutionStrategyEmitter` with random direction ranking or the [`IsoLineEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.IsoLineEmitter.html)"
]
},
{
Expand Down
16 changes: 8 additions & 8 deletions tutorials/cma_mae.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
"source": [
"# Upgrading CMA-ME to CMA-MAE on the Sphere Benchmark\n",
"\n",
"_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/stable/tutorials.html) for the list of all tutorials and the order in which they should be read._\n",
"_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/latest/tutorials.html) for the list of all tutorials and the order in which they should be read._\n",
"\n",
"In the [previous tutorial](https://docs.pyribs.org/en/stable/tutorials/lunar_lander.html), we showed how to implement the CMA-ME algorithm in pyribs to tackle the lunar lander problem. CMA-ME enabled us to search for a diverse collection of high-performing lunar lander agents, including ones which landed like a space shuttle:"
"In the [previous tutorial](https://docs.pyribs.org/en/latest/tutorials/lunar_lander.html), we showed how to implement the CMA-ME algorithm in pyribs to tackle the lunar lander problem. CMA-ME enabled us to search for a diverse collection of high-performing lunar lander agents, including ones which landed like a space shuttle:"
]
},
{
Expand Down Expand Up @@ -231,7 +231,7 @@
"source": [
"### GridArchive\n",
"\n",
"First, we will create the [`GridArchive`](https://docs.pyribs.org/en/stable/api/ribs.archives.GridArchive.html). The archive is 100x100 and stores 100-dimensional solutions. The ranges of the archive are determined by the maximum outputs of the measure function. Since the $clip(\\theta_i)$ function has a maximum output of 5.12, and each component of the output is the sum of $\\frac{n}{2}$ clipped components, the bounds of the measure space are $\\pm 5.12 * \\frac{n}{2}$.\n",
"First, we will create the [`GridArchive`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html). The archive is 100x100 and stores 100-dimensional solutions. The ranges of the archive are determined by the maximum outputs of the measure function. Since the $clip(\\theta_i)$ function has a maximum output of 5.12, and each component of the output is the sum of $\\frac{n}{2}$ clipped components, the bounds of the measure space are $\\pm 5.12 * \\frac{n}{2}$.\n",
"\n",
"Next, the key difference from CMA-ME is that this archive takes in the `learning_rate` parameter ($\\alpha$) which controls how quickly the threshold in each cell is updated. We set this to 0.01 from the CMA-MAE paper. The second difference is that this archive takes in a `threshold_min` parameter ($min_f$) which is the starting threshold for each cell. This threshold is typically the minimum objective in the problem, hence we choose 0.0.\n",
"\n",
Expand Down Expand Up @@ -292,7 +292,7 @@
"source": [
"### EvolutionStrategyEmitter\n",
"\n",
"Next, we set up 15 instances of the [`EvolutionStrategyEmitter`](https://docs.pyribs.org/en/stable/api/ribs.emitters.EvolutionStrategyEmitter.html). The key difference from CMA-ME's `EvolutionStrategyEmitter` is that these emitters use _improvement ranking_ rather than _two-stage improvement ranking_. Two-stage improvement ranking (see the [Lunar Lander tutorial](https://docs.pyribs.org/en/stable/tutorials/lunar_lander.html)) considers the \"status\" and \"value\" of each solution, such that solutions which introduced new cells in the archive come first, followed by solutions which improved existing cells and solutions which were not added. In contrast, improvement ranking considers only the improvement values $\\Delta = f(\\theta') - t_e$, i.e., the difference between the objective values of incumbent solutions and the threshold of their corresponding cells in the archive.\n",
"Next, we set up 15 instances of the [`EvolutionStrategyEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.EvolutionStrategyEmitter.html). The key difference from CMA-ME's `EvolutionStrategyEmitter` is that these emitters use _improvement ranking_ rather than _two-stage improvement ranking_. Two-stage improvement ranking (see the [Lunar Lander tutorial](https://docs.pyribs.org/en/latest/tutorials/lunar_lander.html)) considers the \"status\" and \"value\" of each solution, such that solutions which introduced new cells in the archive come first, followed by solutions which improved existing cells and solutions which were not added. In contrast, improvement ranking considers only the improvement values $\\Delta = f(\\theta') - t_e$, i.e., the difference between the objective values of incumbent solutions and the threshold of their corresponding cells in the archive.\n",
"\n",
"There are further differences which set these emitters apart from the ones which are used in CMA-ME. First, the CMA-MAE emitters below use the \"mu\" `selection_rule`, which affects which solutions the emitter uses as parents for updating CMA-ES. The default \"filter\" selection (used in CMA-ME) uses all solutions which added a new cell or improved the archive, while \"mu\" selects the top half of all the generated solutions as parents. Second, the CMA-MAE emitters use a \"basic\" restart rule, which restarts the emitter according to the convergence rules of CMA-ES. The default \"no_improvement\" restart rule restarts the emitter when none of its generated solutions are inserted into the archive."
]
Expand Down Expand Up @@ -330,7 +330,7 @@
"source": [
"### Scheduler\n",
"\n",
"Finally, the [`Scheduler`](https://docs.pyribs.org/en/stable/api/ribs.schedulers.Scheduler.html) controls how the emitters interact with the archive and result archive. On every iteration, the scheduler calls the emitters to generate solutions. After the user evaluates these generated solutions, the scheduler inserts the solutions into both the archive and result archive. It then passes feedback from the archive (but not the result archive) to the emitters. In this manner, the emitters only interact with the archive, but the result archive stores all the best solutions found by CMA-MAE."
"Finally, the [`Scheduler`](https://docs.pyribs.org/en/latest/api/ribs.schedulers.Scheduler.html) controls how the emitters interact with the archive and result archive. On every iteration, the scheduler calls the emitters to generate solutions. After the user evaluates these generated solutions, the scheduler inserts the solutions into both the archive and result archive. It then passes feedback from the archive (but not the result archive) to the emitters. In this manner, the emitters only interact with the archive, but the result archive stores all the best solutions found by CMA-MAE."
]
},
{
Expand Down Expand Up @@ -388,7 +388,7 @@
"Supporting this behavior introduces two additional considerations:\n",
"\n",
"1. When we perform batch addition via the `add()` method on an archive with CMA-MAE, we apply the CMA-MAE batch addition rule in Appendix H of [Fontaine 2022](https://arxiv.org/abs/2205.10752). However, when using CMA-ME settings, we update the threshold by taking the maximum objective value to maintain consistency with the original algorithm.\n",
"2. When using the CMA-ME settings, using a regular `ImprovementRanker` (i.e. `ranker=\"imp\"`) is not advisable, as CMA-ME is only designed to work with two-stage improvement ranking (it is okay to use non-two stage versions of [other rankers](https://docs.pyribs.org/en/stable/api/ribs.emitters.rankers.html), however)."
"2. When using the CMA-ME settings, using a regular `ImprovementRanker` (i.e. `ranker=\"imp\"`) is not advisable, as CMA-ME is only designed to work with two-stage improvement ranking (it is okay to use non-two stage versions of [other rankers](https://docs.pyribs.org/en/latest/api/ribs.emitters.rankers.html), however)."
]
},
{
Expand Down Expand Up @@ -467,7 +467,7 @@
"source": [
"## Visualization\n",
"\n",
"Now we visualize the result archive with [`grid_archive_heatmap`](https://docs.pyribs.org/en/stable/api/ribs.visualize.grid_archive_heatmap.html) from the [`ribs.visualize`](https://docs.pyribs.org/en/stable/api/ribs.visualize.html) module (as mentioned previously, the archive used in the main algorithm does not always hold the best solutions, while the result archive does)."
"Now we visualize the result archive with [`grid_archive_heatmap`](https://docs.pyribs.org/en/latest/api/ribs.visualize.grid_archive_heatmap.html) from the [`ribs.visualize`](https://docs.pyribs.org/en/latest/api/ribs.visualize.html) module (as mentioned previously, the archive used in the main algorithm does not always hold the best solutions, while the result archive does)."
]
},
{
Expand Down Expand Up @@ -829,7 +829,7 @@
"\n",
"In this tutorial, we introduced CMA-MAE. We showed how it is implemented in pyribs, and we compared this implementation to that of CMA-ME. Then, we demonstrated CMA-MAE on the sphere linear projection benchmark, and we concluded by seeing how different settings of the learning rate $\\alpha$ affected is performance. Overall, CMA-MAE has strong theoretical and empirical properties which enable it to excel at QD problems like this one. For these reasons, we recommend CMA-MAE over CMA-ME for most QD optimization problems.\n",
"\n",
"_For implementations of a wide variety of algorithms on the sphere benchmark, take a look at our [sphere example](https://docs.pyribs.org/en/stable/examples/sphere.html)._"
"_For implementations of a wide variety of algorithms on the sphere benchmark, take a look at our [sphere example](https://docs.pyribs.org/en/latest/examples/sphere.html)._"
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions tutorials/fooling_mnist.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"source": [
"# Generating Images to Fool an MNIST Classifier\n",
"\n",
"_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/stable/tutorials.html) for the list of all tutorials and the order in which they should be read._\n",
"_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/latest/tutorials.html) for the list of all tutorials and the order in which they should be read._\n",
"\n",
"Despite their high performance on classification tasks such as MNIST, neural networks like the [LeNet-5](https://en.wikipedia.org/wiki/LeNet) have a weakness: they are easy to fool. Namely, given images like the ones below, a classifier may confidently believe that it is seeing certain digits, even though the images look like random noise to humans. Naturally, this phenomenon raises some concerns, especially when the network in question is used in a safety-critical system like a self-driving car. Given such unrecognizable input, one would hope that the network at least has low confidence in its prediction.\n",
"\n",
Expand Down Expand Up @@ -165,7 +165,7 @@
"\n",
"Our classifier outputs a log probability vector with its belief that it is seeing each digit. Thus, our objective for each digit is to maximize the probability that the classifier assigns to the image associated with it. For instance, for digit 5, we want to generate an image that makes the classifier believe with high probability that it is seeing a 5.\n",
"\n",
"In pyribs, we implement MAP-Elites with a [`GridArchive`](https://docs.pyribs.org/en/stable/api/ribs.archives.GridArchive.html) and a [`GaussianEmitter`](https://docs.pyribs.org/en/stable/api/ribs.emitters.GaussianEmitter.html). Below, we start by constructing the `GridArchive`. The archive has 10 cells and a range of (0,10). Since `GridArchive` was originally designed for continuous spaces, it does not directly support discrete spaces, but by using these settings, we have a cell for each digit from 0 to 9."
"In pyribs, we implement MAP-Elites with a [`GridArchive`](https://docs.pyribs.org/en/latest/api/ribs.archives.GridArchive.html) and a [`GaussianEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.GaussianEmitter.html). Below, we start by constructing the `GridArchive`. The archive has 10 cells and a range of (0,10). Since `GridArchive` was originally designed for continuous spaces, it does not directly support discrete spaces, but by using these settings, we have a cell for each digit from 0 to 9."
]
},
{
Expand Down
Loading