You can then write methods looking for implicit API instances, like
defdisplayTimer()(implicit kernel: almond.api.JupyterApi): Unit = {
+ val count = 5
+ val id = java.util.UUID.randomUUID().toString
+ kernel.publish.html(s"<b>$count</b>", id)
+ for (i <- (0 until count).reverse) {
+ Thread.sleep(1000L)
+ kernel.publish.updateHtml(s"<b>$i</b>", id)
+ }
+ Thread.sleep(200L)
+ kernel.publish.updateHtml(Character.toChars(0x1f981).mkString, id)
+}
+
Users can then load the library defining this method, and call it themselves from a notebook. The library can interact with the front-end, without overhead for users.
A library defining this method, along with instructions to set up the library and a notebook using it, are available in this repository.
You can then write methods looking for implicit API instances, like
defdisplayTimer()(implicit kernel: almond.api.JupyterApi): Unit = {
+ val count = 5
+ val id = java.util.UUID.randomUUID().toString
+ kernel.publish.html(s"<b>$count</b>", id)
+ for (i <- (0 until count).reverse) {
+ Thread.sleep(1000L)
+ kernel.publish.updateHtml(s"<b>$i</b>", id)
+ }
+ Thread.sleep(200L)
+ kernel.publish.updateHtml(Character.toChars(0x1f981).mkString, id)
+}
+
Users can then load the library defining this method, and call it themselves from a notebook. The library can interact with the front-end, without overhead for users.
A library defining this method, along with instructions to set up the library and a notebook using it, are available in this repository.
The Ammonite API consists in instances of several classes, that are created by Ammonite or almond, and allow you to interact with the REPL. These instances are accessible either by their name, from the REPL, like interp, or via implicits, like implicitly[ammonite.interp.api.InterpAPI].
interp.load.ivy(
+ // replace with linux-gpu-x86_64 on linux with nvidia gpu or with darwin-cpu-x86_64 on macOS
+ ("org.platanios" %% "tensorflow" % "0.4.1").withClassifier("linux-cpu-x86_64")
+)
+
The dependencies can then be used in the cell right after the one calling interp.load.ivy.
Note that in the case of simple dependencies, when directly entering code in a notebook, the following syntax is preferred and allows to use the dependency in the current cell rather than the next one:
The Ammonite API consists in instances of several classes, that are created by Ammonite or almond, and allow you to interact with the REPL. These instances are accessible either by their name, from the REPL, like interp, or via implicits, like implicitly[ammonite.interp.api.InterpAPI].
interp.load.ivy(
+ // replace with linux-gpu-x86_64 on linux with nvidia gpu or with darwin-cpu-x86_64 on macOS
+ ("org.platanios" %% "tensorflow" % "0.4.1").withClassifier("linux-cpu-x86_64")
+)
+
The dependencies can then be used in the cell right after the one calling interp.load.ivy.
Note that in the case of simple dependencies, when directly entering code in a notebook, the following syntax is preferred and allows to use the dependency in the current cell rather than the next one:
The Almond Jupyter API can be accessed via an instance of almond.api.JupyterApi. Such an instance is created by almond upon start-up. This instance accessible via the kernel variable and in the implicit scope via e.g. implicitly[almond.api.JupyterApi].
// These can be used to display things straightaway
+Html("<b>Bold</b>")
+
// A handle can also be retained, to later update or clear things
+val handle = Markdown("""
+# title
+## section
+text
+""")
+
+// can be updated in later cells
+// (this updates the previous display)
+handle.withContent("""
+# updated title
+## new section
+_content_
+""").update()
+
+// can be later cleared
+handle.clear()
+
Comm messages are part of the Jupyter messaging protocol. They allow the exchange of arbitrary messages between code running in the front-end (typically JavaScript code) and kernels.
The comm API can be used to receive messages, or send them.
kernel.comm.receiver allows to register a target to receive messages from the front-end, like
val id = java.util.UUID.randomUUID().toString
+kernel.publish.html("Waiting", id)
+
+kernel.comm.receiver("A") { data =>
+ // received message `data` from front-end
+ kernel.publish.updateHtml(s"<code>$data</code>", id)
+}
+
The most generic is display, accepting a almond.api.DisplayData.
kernel.publish.display(
+ almond.interpreter.api.DisplayData(
+ Map(
+ // if we set up an extension for application/myapp+json, first element should be picked
+ "application/myapp+json" -> """{"a": "A"}""",
+ // else, text/html should be displayed
+ "text/html" -> "<b>A</b>"
+ )
+ )
+)
+
OutputHandler also has helper methods to push HTML straightaway.
for (i <- 1 to 10) {
+ kernel.publish.html(s"Got item <b>#$i</b>")
+ Thread.sleep((200.0 + 200.0 * scala.util.Random.nextGaussian).toLong max 0L)
+}
+
The Almond Jupyter API can be accessed via an instance of almond.api.JupyterApi. Such an instance is created by almond upon start-up. This instance accessible via the kernel variable and in the implicit scope via e.g. implicitly[almond.api.JupyterApi].
// These can be used to display things straightaway
+Html("<b>Bold</b>")
+
// A handle can also be retained, to later update or clear things
+val handle = Markdown("""
+# title
+## section
+text
+""")
+
+// can be updated in later cells
+// (this updates the previous display)
+handle.withContent("""
+# updated title
+## new section
+_content_
+""").update()
+
+// can be later cleared
+handle.clear()
+
Comm messages are part of the Jupyter messaging protocol. They allow the exchange of arbitrary messages between code running in the front-end (typically JavaScript code) and kernels.
The comm API can be used to receive messages, or send them.
kernel.comm.receiver allows to register a target to receive messages from the front-end, like
val id = java.util.UUID.randomUUID().toString
+kernel.publish.html("Waiting", id)
+
+kernel.comm.receiver("A") { data =>
+ // received message `data` from front-end
+ kernel.publish.updateHtml(s"<code>$data</code>", id)
+}
+
The most generic is display, accepting a almond.api.DisplayData.
kernel.publish.display(
+ almond.interpreter.api.DisplayData(
+ Map(
+ // if we set up an extension for application/myapp+json, first element should be picked
+ "application/myapp+json" -> """{"a": "A"}""",
+ // else, text/html should be displayed
+ "text/html" -> "<b>A</b>"
+ )
+ )
+)
+
OutputHandler also has helper methods to push HTML straightaway.
for (i <- 1 to 10) {
+ kernel.publish.html(s"Got item <b>#$i</b>")
+ Thread.sleep((200.0 + 200.0 * scala.util.Random.nextGaussian).toLong max 0L)
+}
+
Note that case-app can be replaced by the command-line parsing library of your choice.
The demo kernel mainly consists in EchoInterpreter, an implementation of almond.interpreter.Interpreter, and in EchoKernel, a small application that either installs the kernel, or actually runs it when it is launched by Jupyter.
When implementing a custom kernel, you may want to re-use almost as is EchoKernel, instantiating your own Interpreter instead of EchoInterpreter. You may optionally handle command-line arguments differently, or add extra command-line options to pass to your Interpreter implementation.
For the kernel installation to work out-of-the-box, the kernel should to be launched via coursier in one way or another. (It is possible nonetheless to rely on e.g. uber JARs, by tweaking the arg or command fields of InstallOptions). To test your kernel locally, publish it locally via a publishLocal from sbt. Note its organization, module name, and version. Then create a launcher for your kernel with
Note that case-app can be replaced by the command-line parsing library of your choice.
The demo kernel mainly consists in EchoInterpreter, an implementation of almond.interpreter.Interpreter, and in EchoKernel, a small application that either installs the kernel, or actually runs it when it is launched by Jupyter.
When implementing a custom kernel, you may want to re-use almost as is EchoKernel, instantiating your own Interpreter instead of EchoInterpreter. You may optionally handle command-line arguments differently, or add extra command-line options to pass to your Interpreter implementation.
For the kernel installation to work out-of-the-box, the kernel should to be launched via coursier in one way or another. (It is possible nonetheless to rely on e.g. uber JARs, by tweaking the arg or command fields of InstallOptions). To test your kernel locally, publish it locally via a publishLocal from sbt. Note its organization, module name, and version. Then create a launcher for your kernel with
Ensure a JDK (Java Development Kit) is installed on your machine. Java versions 8 or 11 are recommended, with 8 as minimum version. If you don't already have a JDK installed, you can install one by following the instructions on the AdoptOpenJDK website, or by grabbing the coursier command-line and using its cs java command. Your OS package manager (brew, apt, …) may also offer to install a JDK for you.
Once a JDK is installed, you should be able to run the java command, like
$ java -version
+java version "1.8.0_121"
+Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
+Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
+$ javac -version
+javac 1.8.0_121
+
Check-out the sources with git:
$ git clone https://github.com/almond-sh/almond.git
+$ cd almond
+
Almond is built with mill. A mill launcher ships with almond, so that you don't need to install mill yourself. We list below useful commands, to help get you started using mill to build almond.
+ Run a Jupyter notebook server without installing a kernel
$ ./mill -i jupyterFast
+
This should
+
+
build an almond launcher, then
+
start JupyterLab in the current directory.
+
From the JupyterLab instance, select the kernel "Scala (sources)".
Optionally, pass a Scala version and / or JupyterLab options, like
You can re-build a launcher upon source changes with
$ ./mill -w launcherFast
+
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill jupyter, beware to pass the same version to ./mill -w launcher.
It is recommended to generate Metals configuration files manually, rather than letting Metals load the project itself. In order to do that, run
$ ./mill mill.contrib.Bloop/install
+
If you're using Metals from VSCode, you can then run the "Metals: Connect to build server" command to take into account the newly generated files.
If the command above takes too long to run, comment out Scala versions in deps.sc. If no 2.12 versions are left, also comment out the few 2.12-specific projects in build.sc (look for 212 to find them). Same if no 2.13 versions are left (look for 213 to spot 2.13-specific projects).
Example notebooks live under examples/. These are run on the CI using nbconvert, and the resulting outputs are compared to the committed ones. Any difference results in the examples job on the CI to fail.
To validate the examples locally, run
$ ./mill -i scala.examples.test
+
Optionally, you can pass a glob to filter notebook names:
Ensure a JDK (Java Development Kit) is installed on your machine. Java versions 8 or 11 are recommended, with 8 as minimum version. If you don't already have a JDK installed, you can install one by following the instructions on the AdoptOpenJDK website, or by grabbing the coursier command-line and using its cs java command. Your OS package manager (brew, apt, …) may also offer to install a JDK for you.
Once a JDK is installed, you should be able to run the java command, like
$ java -version
+java version "1.8.0_121"
+Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
+Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
+$ javac -version
+javac 1.8.0_121
+
Check-out the sources with git:
$ git clone https://github.com/almond-sh/almond.git
+$ cd almond
+
Almond is built with mill. A mill launcher ships with almond, so that you don't need to install mill yourself. We list below useful commands, to help get you started using mill to build almond.
+ Run a Jupyter notebook server without installing a kernel
$ ./mill -i jupyterFast
+
This should
+
+
build an almond launcher, then
+
start JupyterLab in the current directory.
+
From the JupyterLab instance, select the kernel "Scala (sources)".
Optionally, pass a Scala version and / or JupyterLab options, like
You can re-build a launcher upon source changes with
$ ./mill -w launcherFast
+
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill jupyter, beware to pass the same version to ./mill -w launcher.
It is recommended to generate Metals configuration files manually, rather than letting Metals load the project itself. In order to do that, run
$ ./mill mill.contrib.Bloop/install
+
If you're using Metals from VSCode, you can then run the "Metals: Connect to build server" command to take into account the newly generated files.
If the command above takes too long to run, comment out Scala versions in deps.sc. If no 2.12 versions are left, also comment out the few 2.12-specific projects in build.sc (look for 212 to find them). Same if no 2.13 versions are left (look for 213 to spot 2.13-specific projects).
Example notebooks live under examples/. These are run on the CI using nbconvert, and the resulting outputs are compared to the committed ones. Any difference results in the examples job on the CI to fail.
To validate the examples locally, run
$ ./mill -i scala.examples.test
+
Optionally, you can pass a glob to filter notebook names:
If you're getting an error message like Cannot resolve $file import: almond/scripts/website0/Website.sc make sure you have checked out the git submodule for the website script:
git submodule update --init --recursive
+
If the generation is successful, you can run a small webserver to serve the website locally, like
$ npx http-server docs/website/build
+
This command should print the address to access the local website, like http://127.0.0.1:8080.
If you're getting an error message like Cannot resolve $file import: almond/scripts/website0/Website.sc make sure you have checked out the git submodule for the website script:
git submodule update --init --recursive
+
If the generation is successful, you can run a small webserver to serve the website locally, like
$ npx http-server docs/website/build
+
This command should print the address to access the local website, like http://127.0.0.1:8080.
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.12 version,
$ cs launch almond --scala 2.13.12 -- --install
+
This installs almond with the default kernel id, scala, and default display name, "Scala".
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.12 version,
$ cs launch almond --scala 2.13.12 -- --install
+
This installs almond with the default kernel id, scala, and default display name, "Scala".
Whether to install this kernel system-wide or not.
System-wide kernels get installed under /usr/local/share/jupyter/kernels (Linux and OS X). User-wide ones (default) under ~/.local/share/jupyter/kernels (Linux) or ~/Library/Jupyter/kernels (Mac).
By default, it is assumed coursier or a coursier launcher is involved during the installation. This allows to get the command that spawns the kernel for install. (Actual executable is located via the coursier.mainJar Java property, and its arguments via coursier.main.arg-0, coursier.main.arg-1, ... Java properties.) This command is then used to later start actual kernels.
To override that behavior, pass --args or --command options. --arg allows to pass each argument individually, like
Simple line magics such as %AddDeps (always assumed to be transitive as of writing this, --transitive is just ignored), %AddJar, and cell magics such as %%html or %%javascript are supported. Note that %%javascript only works from Jupyter classic, as JupyterLab doesn't allow for random javascript code execution.
Whether to install this kernel system-wide or not.
System-wide kernels get installed under /usr/local/share/jupyter/kernels (Linux and OS X). User-wide ones (default) under ~/.local/share/jupyter/kernels (Linux) or ~/Library/Jupyter/kernels (Mac).
By default, it is assumed coursier or a coursier launcher is involved during the installation. This allows to get the command that spawns the kernel for install. (Actual executable is located via the coursier.mainJar Java property, and its arguments via coursier.main.arg-0, coursier.main.arg-1, ... Java properties.) This command is then used to later start actual kernels.
To override that behavior, pass --args or --command options. --arg allows to pass each argument individually, like
Simple line magics such as %AddDeps (always assumed to be transitive as of writing this, --transitive is just ignored), %AddJar, and cell magics such as %%html or %%javascript are supported. Note that %%javascript only works from Jupyter classic, as JupyterLab doesn't allow for random javascript code execution.
When launched by other users, the kernel installed by the default command downloads its dependencies in the user-specific coursier cache upon first launch.
You may prefer to have the launcher embed all these JARs, so that nothing needs to be downloaded or picked from a cache upon launch. Passing --standalone to the coursier bootstrap command generates such a launcher,
$ coursier bootstrap --standalone \
+ almond:0.14.0-RC14 --scala 2.13.12 \
+ -o almond
+$ ./almond --install
+$ rm -f almond # the generated launcher can be removed after install, it copied itself in the kernel installation directory
+
When launched by other users, the kernel installed by the default command downloads its dependencies in the user-specific coursier cache upon first launch.
You may prefer to have the launcher embed all these JARs, so that nothing needs to be downloaded or picked from a cache upon launch. Passing --standalone to the coursier bootstrap command generates such a launcher,
$ coursier bootstrap --standalone \
+ almond:0.14.0-RC14 --scala 2.13.12 \
+ -o almond
+$ ./almond --install
+$ rm -f almond # the generated launcher can be removed after install, it copied itself in the kernel installation directory
+
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here).
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --fork almond -- --install --force). That will override any previous almond (or kernel with name scala).
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here).
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --fork almond -- --install --force). That will override any previous almond (or kernel with name scala).
Whilst there are scala plotting libraries, here is a pragmatic, flexible workflow without them. Play each part of the stack to it's strengths - use the vega editor to get your "spec" right, then scala to obatin and pipe the data into the spec. Any JSON library would do...
Whilst there are scala plotting libraries, here is a pragmatic, flexible workflow without them. Play each part of the stack to it's strengths - use the vega editor to get your "spec" right, then scala to obatin and pipe the data into the spec. Any JSON library would do...
Almond comes with a Spark integration module called almond-spark, which allows you to connect to a Spark cluster and to run Spark calculations interactively from a Jupyter notebook.
It is based on ammonite-spark, adding Jupyter specific features such as progress bars and cancellation for running Spark computations.
ammonite-spark handles loading Spark in a clever way, and does not rely on a specific Spark distribution. Because of that, you can use it with any Spark 2.x version. The only limitation is that the Scala version of Spark and the running Almond kernel must match, so make sure your kernel uses the same Scala version as your Spark cluster. Spark 2.0.x - 2.3.x requires Scala 2.11. Spark 2.4.x supports both Scala 2.11 and 2.12.
Note that as of almond 0.7.0, almond only supports Scala 2.12 and therefore requires Spark 2.4.x for Scala 2.12.
For more information, see the README of ammonite-spark.
To use it, import the almond-spark dependency as well as Spark 2.x itself.
import $ivy.`org.apache.spark::spark-sql:2.4.0` // Or use any other 2.x version here
+import $ivy.`sh.almond::almond-spark:0.14.0-RC14` // Not required since almond 0.7.0 (will be automatically added when importing spark)
+
Usually you want to disable logging in order to avoid polluting your cell outputs:
When running this, you should see that the cell output contains a link to the Spark UI.
Note the use of NotebookSparkSession.builder(), instead of SparkSession.builder() that one would use when e.g. writing a Spark job.
The builder returned by NotebookSparkSession.builder() extends the one of SparkSession.builder(), so that one can call .appName("foo"), .config("key", "value"), etc. on it.
Now you can get a SparkContext from the SparkSession and run Spark calculations.
defsc= spark.sparkContext
+
+val rdd = sc.parallelize(1 to 100000000, 100)
+
+val n = rdd.map(_ + 1).sum()
+
When you execute a Spark action like sum you should see a progress bar, showing the progress of the running Spark job, as well as a link to cancel the job if you are using the Jupyter classic UI.
If extra dependencies are loaded, via import $ivy.`…` after the SparkSession has been created, one should call NotebookSparkSession.sync() for the newly added JARs to be passed to the Spark executors.
Ensure the version of Spark used to start the master and executors matches the one loaded in the notebook session (via e.g. import $ivy.`org.apache.spark::spark-sql:X.Y.Z`), and that the machine running the kernel can access / is accessible from all nodes of the standalone cluster.
Almond comes with a Spark integration module called almond-spark, which allows you to connect to a Spark cluster and to run Spark calculations interactively from a Jupyter notebook.
It is based on ammonite-spark, adding Jupyter specific features such as progress bars and cancellation for running Spark computations.
ammonite-spark handles loading Spark in a clever way, and does not rely on a specific Spark distribution. Because of that, you can use it with any Spark 2.x version. The only limitation is that the Scala version of Spark and the running Almond kernel must match, so make sure your kernel uses the same Scala version as your Spark cluster. Spark 2.0.x - 2.3.x requires Scala 2.11. Spark 2.4.x supports both Scala 2.11 and 2.12.
Note that as of almond 0.7.0, almond only supports Scala 2.12 and therefore requires Spark 2.4.x for Scala 2.12.
For more information, see the README of ammonite-spark.
To use it, import the almond-spark dependency as well as Spark 2.x itself.
import $ivy.`org.apache.spark::spark-sql:2.4.0` // Or use any other 2.x version here
+import $ivy.`sh.almond::almond-spark:0.14.0-RC14` // Not required since almond 0.7.0 (will be automatically added when importing spark)
+
Usually you want to disable logging in order to avoid polluting your cell outputs:
When running this, you should see that the cell output contains a link to the Spark UI.
Note the use of NotebookSparkSession.builder(), instead of SparkSession.builder() that one would use when e.g. writing a Spark job.
The builder returned by NotebookSparkSession.builder() extends the one of SparkSession.builder(), so that one can call .appName("foo"), .config("key", "value"), etc. on it.
Now you can get a SparkContext from the SparkSession and run Spark calculations.
defsc= spark.sparkContext
+
+val rdd = sc.parallelize(1 to 100000000, 100)
+
+val n = rdd.map(_ + 1).sum()
+
When you execute a Spark action like sum you should see a progress bar, showing the progress of the running Spark job, as well as a link to cancel the job if you are using the Jupyter classic UI.
If extra dependencies are loaded, via import $ivy.`…` after the SparkSession has been created, one should call NotebookSparkSession.sync() for the newly added JARs to be passed to the Spark executors.
Ensure the version of Spark used to start the master and executors matches the one loaded in the notebook session (via e.g. import $ivy.`org.apache.spark::spark-sql:X.Y.Z`), and that the machine running the kernel can access / is accessible from all nodes of the standalone cluster.
Note that case-app can be replaced by the command-line parsing library of your choice.
The demo kernel mainly consists in EchoInterpreter, an implementation of almond.interpreter.Interpreter, and in EchoKernel, a small application that either installs the kernel, or actually runs it when it is launched by Jupyter.
When implementing a custom kernel, you may want to re-use almost as is EchoKernel, instantiating your own Interpreter instead of EchoInterpreter. You may optionally handle command-line arguments differently, or add extra command-line options to pass to your Interpreter implementation.
For the kernel installation to work out-of-the-box, the kernel should to be launched via coursier in one way or another. (It is possible nonetheless to rely on e.g. uber JARs, by tweaking the arg or command fields of InstallOptions). To test your kernel locally, publish it locally via a publishLocal from sbt. Note its organization, module name, and version. Then create a launcher for your kernel with
Note that case-app can be replaced by the command-line parsing library of your choice.
The demo kernel mainly consists in EchoInterpreter, an implementation of almond.interpreter.Interpreter, and in EchoKernel, a small application that either installs the kernel, or actually runs it when it is launched by Jupyter.
When implementing a custom kernel, you may want to re-use almost as is EchoKernel, instantiating your own Interpreter instead of EchoInterpreter. You may optionally handle command-line arguments differently, or add extra command-line options to pass to your Interpreter implementation.
For the kernel installation to work out-of-the-box, the kernel should to be launched via coursier in one way or another. (It is possible nonetheless to rely on e.g. uber JARs, by tweaking the arg or command fields of InstallOptions). To test your kernel locally, publish it locally via a publishLocal from sbt. Note its organization, module name, and version. Then create a launcher for your kernel with
Almond is built with mill. A mill launcher ships with almond, so that you don't need to install mill yourself. We list below useful commands, to help get you started using mill to build almond.
Run a Jupyter notebook server without installing a kernel
$ ./mill -i jupyterFast
+ Run a Jupyter notebook server without installing a kernel
$ ./mill -i dev.jupyterFast
This should
build an almond launcher, then
start JupyterLab in the current directory.
-
From the JupyterLab instance, select the kernel "Scala (sources)".
Optionally, pass a Scala version and / or JupyterLab options, like
Pass --help or see this page for the available options.
Watch for source changes
You can re-build a launcher upon source changes with
$ ./mill -w launcherFast
-
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill jupyter, beware to pass the same version to ./mill -w launcher.
You can re-build a launcher upon source changes with
$ ./mill -w dev.launcherFast
+
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill dev.jupyter, beware to pass the same version to ./mill -w dev.launcher.
Useful commands
List available Scala versions
$ ./mill scalaVersions
+ List available Scala versions
$ ./mill dev.scalaVersions
2.13.4
2.13.3
…
Print the latest supported Scala 2.13 version
$ ./mill scala213
+ Print the latest supported Scala 2.13 version
$ ./mill dev.scala213
2.13.4
Print the latest supported Scala 2.12 version
$ ./mill scala212
+ Print the latest supported Scala 2.12 version
Almond is built with mill. A mill launcher ships with almond, so that you don't need to install mill yourself. We list below useful commands, to help get you started using mill to build almond.
Run a Jupyter notebook server without installing a kernel
$ ./mill -i jupyterFast
+ Run a Jupyter notebook server without installing a kernel
$ ./mill -i dev.jupyterFast
This should
build an almond launcher, then
start JupyterLab in the current directory.
-
From the JupyterLab instance, select the kernel "Scala (sources)".
Optionally, pass a Scala version and / or JupyterLab options, like
Pass --help or see this page for the available options.
Watch for source changes
You can re-build a launcher upon source changes with
$ ./mill -w launcherFast
-
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill jupyter, beware to pass the same version to ./mill -w launcher.
You can re-build a launcher upon source changes with
$ ./mill -w dev.launcherFast
+
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill dev.jupyter, beware to pass the same version to ./mill -w dev.launcher.
Useful commands
List available Scala versions
$ ./mill scalaVersions
+ List available Scala versions
$ ./mill dev.scalaVersions
2.13.4
2.13.3
…
Print the latest supported Scala 2.13 version
$ ./mill scala213
+ Print the latest supported Scala 2.13 version
$ ./mill dev.scala213
2.13.4
Print the latest supported Scala 2.12 version
$ ./mill scala212
+ Print the latest supported Scala 2.12 version
Before diving into how Almond starts, a few words about the Almond installation. The installation of Almond can look complex by some aspects. Beyond that, customizing the installation, making sense of what it does in detail, can be tricky. This originates from two concerns that Almond tries to reconcile:
+
+
isolation of the internal dependencies of Almond from users
+
allowing the customization / tweaking of the Almond class path upon installation
Almond depends on fs2 and cats-effect. In order not to force its own cats / cats-effect / fs2 versions upon users, these internal dependencies of Almond are effectively hidden from users.
In more detail, isolation in Almond works this way: the scala-kernel-api module and its dependencies are user-facing. The rest of the Almond class path (the scala-kernel module and its dependencies, but for everything already pulled by scala-kernel-api) are internal dependencies, hidden from users.
Isolation between these two set of dependencies is achieved by starting Almond using launchers generated by coursier. (Note that the coursier documentation doesn't really detail how dependency isolation works. We detail that right below here.)
If we put dependencies isolation aside, these launchers are a small Java application, alongside a resource file listing URLs of the JARs that should be loaded. Upon startup, the launcher reads the URL list, check if these are in the coursier cache and downloads them if they're not. Then it creates a ClassLoader, loads the local copies of the JARs in it, and loads and calls the main class of the application from it.
With dependencies isolation enabled, these launchers actually embed two lists of JARs: a "top" one (corresponding to user dependencies for Almond), and a "bottom" one (for Almond, internal dependencies, with cats-effect, fs2, etc.). Upon startup, they create a first class loader with the top dependencies, and a second one, having the first class loader as a parent, with the bottom dependencies. In such cases, the way class loaders work on the JVM makes the classes loaded in the bottom class loader "see" the classes in the top one, but not the other way around. Later on, when the app runs, it can ask for the class loader of a class it knows is part of the top dependencies, which gives it a reference to the top class loader, that knows nothing about the bottom dependencies.
Almond relies on that mechanism to get a class loader that only knows about user-facing dependencies. That class loader is used as a parent of the class loader that's going to load the classes generated during the session (those corresponding to the input user code in the notebook). That way, users only "see" user-facing dependencies, not internal ones, and they can load whichever other version of the same internal dependencies as Almond.
When passed the --install option, Almond tries to determine the path of its own launcher, and copies it alongside the kernel.json file it generates for Jupyter to be able to launch Almond. That way, the launcher used during the installation can be safely deleted right after the installation.
This section describes how to install Almond for a specific Scala version. Even though the instructions below may rely on newer coursier features, installing Almond this way has been the recommended way to proceed since Almond exists. In the next section, we'll describe a novel way to install Almond, relying on an intermediate launcher, that allows notebook users to customize the Scala version, the JVM they use, Java options (including memory options), on a per notebook basis.
The easiest way to generate an Almond launcher consists in… not generating one. The cs launch command, when passed --use-bootstrap, launches an application via a temporary launcher it generates on-the-fly.
+ Creating an Almond launcher and installing it - newer launcher
The newer launcher allows users to configure Almond in the first cells of notebooks (more precisely, before any actual code - that is, not comments or blank lines - needs to be compiled), with directives like
//> using scala "2.12"
+//> using scala "2.12.19"
+//> using jvm "17"
+//> using javaOpt "-Xmx10g"
+//> using javaOpt "-Dfoo=bar"
+
Important thing to note here: the Scala version must be passed as argument to Almond itself, rather than to cs. The launcher uses its own Scala version (Scala 3), then, later on, spawns the same Almond kernel as above on its own, that takes over as a kernel. The argument passed via --scala corresponds to the default Scala version that will be used, if users don't specify a version of their own via a directive like //> using scala "2.13.14". Specify that option is optional: without it, Almond will default to the Scala 3 version that Almond uses at the time of its release (which should be the latest stable Scala 3 version at the time of the release).
To list the options that the launcher accepts, run
Custom protocol support for java.net.URL needs the JARs supporting it to be passed to the -cp option of java. Using the new launcher above, such JARs can be passed via --extra-startup-class-path, like
Support for the %sql magic, relying on Spark, needs to be enabled from a "predef" script. It also requires parts of the Toree magics support to be put in the user-facing part of the Almond class path (so that calls to it from the user side are picked up by Almond internals later on).
To enable that from a former launcher, pass --shared sh.almond::toree-hooks when generating the launcher:
If you want to use custom Maven repositories rather than Maven Central, you need to set COURSIER_REPOSITORIES at few places. It needs to be set when installing Almond, and when Jupyter starts Almond.
This can be achieved the following way, with the former launcher:
--env sets environment variables in the kernel spec that gets written when installing Almond. Jupyter sets those prior to launching Almond when users open notebooks.
Before diving into how Almond starts, a few words about the Almond installation. The installation of Almond can look complex by some aspects. Beyond that, customizing the installation, making sense of what it does in detail, can be tricky. This originates from two concerns that Almond tries to reconcile:
+
+
isolation of the internal dependencies of Almond from users
+
allowing the customization / tweaking of the Almond class path upon installation
Almond depends on fs2 and cats-effect. In order not to force its own cats / cats-effect / fs2 versions upon users, these internal dependencies of Almond are effectively hidden from users.
In more detail, isolation in Almond works this way: the scala-kernel-api module and its dependencies are user-facing. The rest of the Almond class path (the scala-kernel module and its dependencies, but for everything already pulled by scala-kernel-api) are internal dependencies, hidden from users.
Isolation between these two set of dependencies is achieved by starting Almond using launchers generated by coursier. (Note that the coursier documentation doesn't really detail how dependency isolation works. We detail that right below here.)
If we put dependencies isolation aside, these launchers are a small Java application, alongside a resource file listing URLs of the JARs that should be loaded. Upon startup, the launcher reads the URL list, check if these are in the coursier cache and downloads them if they're not. Then it creates a ClassLoader, loads the local copies of the JARs in it, and loads and calls the main class of the application from it.
With dependencies isolation enabled, these launchers actually embed two lists of JARs: a "top" one (corresponding to user dependencies for Almond), and a "bottom" one (for Almond, internal dependencies, with cats-effect, fs2, etc.). Upon startup, they create a first class loader with the top dependencies, and a second one, having the first class loader as a parent, with the bottom dependencies. In such cases, the way class loaders work on the JVM makes the classes loaded in the bottom class loader "see" the classes in the top one, but not the other way around. Later on, when the app runs, it can ask for the class loader of a class it knows is part of the top dependencies, which gives it a reference to the top class loader, that knows nothing about the bottom dependencies.
Almond relies on that mechanism to get a class loader that only knows about user-facing dependencies. That class loader is used as a parent of the class loader that's going to load the classes generated during the session (those corresponding to the input user code in the notebook). That way, users only "see" user-facing dependencies, not internal ones, and they can load whichever other version of the same internal dependencies as Almond.
When passed the --install option, Almond tries to determine the path of its own launcher, and copies it alongside the kernel.json file it generates for Jupyter to be able to launch Almond. That way, the launcher used during the installation can be safely deleted right after the installation.
This section describes how to install Almond for a specific Scala version. Even though the instructions below may rely on newer coursier features, installing Almond this way has been the recommended way to proceed since Almond exists. In the next section, we'll describe a novel way to install Almond, relying on an intermediate launcher, that allows notebook users to customize the Scala version, the JVM they use, Java options (including memory options), on a per notebook basis.
The easiest way to generate an Almond launcher consists in… not generating one. The cs launch command, when passed --use-bootstrap, launches an application via a temporary launcher it generates on-the-fly.
+ Creating an Almond launcher and installing it - newer launcher
The newer launcher allows users to configure Almond in the first cells of notebooks (more precisely, before any actual code - that is, not comments or blank lines - needs to be compiled), with directives like
//> using scala "2.12"
+//> using scala "2.12.19"
+//> using jvm "17"
+//> using javaOpt "-Xmx10g"
+//> using javaOpt "-Dfoo=bar"
+
Important thing to note here: the Scala version must be passed as argument to Almond itself, rather than to cs. The launcher uses its own Scala version (Scala 3), then, later on, spawns the same Almond kernel as above on its own, that takes over as a kernel. The argument passed via --scala corresponds to the default Scala version that will be used, if users don't specify a version of their own via a directive like //> using scala "2.13.14". Specify that option is optional: without it, Almond will default to the Scala 3 version that Almond uses at the time of its release (which should be the latest stable Scala 3 version at the time of the release).
To list the options that the launcher accepts, run
Custom protocol support for java.net.URL needs the JARs supporting it to be passed to the -cp option of java. Using the new launcher above, such JARs can be passed via --extra-startup-class-path, like
Support for the %sql magic, relying on Spark, needs to be enabled from a "predef" script. It also requires parts of the Toree magics support to be put in the user-facing part of the Almond class path (so that calls to it from the user side are picked up by Almond internals later on).
To enable that from a former launcher, pass --shared sh.almond::toree-hooks when generating the launcher:
If you want to use custom Maven repositories rather than Maven Central, you need to set COURSIER_REPOSITORIES at few places. It needs to be set when installing Almond, and when Jupyter starts Almond.
This can be achieved the following way, with the former launcher:
--env sets environment variables in the kernel spec that gets written when installing Almond. Jupyter sets those prior to launching Almond when users open notebooks.
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.12 version,
$ cs launch almond --scala 2.13.12 -- --install
-
This installs almond with the default kernel id, scala, and default display name, "Scala".
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.14 version,
$ cs launch almond --scala 2.13.14 -- --install
+
This installs almond with the default kernel id, scala, and default display name, "Scala".
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.12 version,
$ cs launch almond --scala 2.13.12 -- --install
-
This installs almond with the default kernel id, scala, and default display name, "Scala".
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.14 version,
$ cs launch almond --scala 2.13.14 -- --install
+
This installs almond with the default kernel id, scala, and default display name, "Scala".
When launched by other users, the kernel installed by the default command downloads its dependencies in the user-specific coursier cache upon first launch.
You may prefer to have the launcher embed all these JARs, so that nothing needs to be downloaded or picked from a cache upon launch. Passing --standalone to the coursier bootstrap command generates such a launcher,
$ coursier bootstrap --standalone \
- almond:0.14.0-RC14 --scala 2.13.12 \
+ almond:0.14.0-RC15 --scala 2.13.14 \
-o almond
$ ./almond --install
$ rm -f almond # the generated launcher can be removed after install, it copied itself in the kernel installation directory
@@ -162,7 +163,7 @@
When launched by other users, the kernel installed by the default command downloads its dependencies in the user-specific coursier cache upon first launch.
You may prefer to have the launcher embed all these JARs, so that nothing needs to be downloaded or picked from a cache upon launch. Passing --standalone to the coursier bootstrap command generates such a launcher,
$ coursier bootstrap --standalone \
- almond:0.14.0-RC14 --scala 2.13.12 \
+ almond:0.14.0-RC15 --scala 2.13.14 \
-o almond
$ ./almond --install
$ rm -f almond # the generated launcher can be removed after install, it copied itself in the kernel installation directory
@@ -162,7 +163,7 @@
Note that case-app can be replaced by the command-line parsing library of your choice.
The demo kernel mainly consists in EchoInterpreter, an implementation of almond.interpreter.Interpreter, and in EchoKernel, a small application that either installs the kernel, or actually runs it when it is launched by Jupyter.
When implementing a custom kernel, you may want to re-use almost as is EchoKernel, instantiating your own Interpreter instead of EchoInterpreter. You may optionally handle command-line arguments differently, or add extra command-line options to pass to your Interpreter implementation.
For the kernel installation to work out-of-the-box, the kernel should to be launched via coursier in one way or another. (It is possible nonetheless to rely on e.g. uber JARs, by tweaking the arg or command fields of InstallOptions). To test your kernel locally, publish it locally via a publishLocal from sbt. Note its organization, module name, and version. Then create a launcher for your kernel with
Note that case-app can be replaced by the command-line parsing library of your choice.
The demo kernel mainly consists in EchoInterpreter, an implementation of almond.interpreter.Interpreter, and in EchoKernel, a small application that either installs the kernel, or actually runs it when it is launched by Jupyter.
When implementing a custom kernel, you may want to re-use almost as is EchoKernel, instantiating your own Interpreter instead of EchoInterpreter. You may optionally handle command-line arguments differently, or add extra command-line options to pass to your Interpreter implementation.
For the kernel installation to work out-of-the-box, the kernel should to be launched via coursier in one way or another. (It is possible nonetheless to rely on e.g. uber JARs, by tweaking the arg or command fields of InstallOptions). To test your kernel locally, publish it locally via a publishLocal from sbt. Note its organization, module name, and version. Then create a launcher for your kernel with
Almond is built with mill. A mill launcher ships with almond, so that you don't need to install mill yourself. We list below useful commands, to help get you started using mill to build almond.
Run a Jupyter notebook server without installing a kernel
$ ./mill -i jupyterFast
+ Run a Jupyter notebook server without installing a kernel
$ ./mill -i dev.jupyterFast
This should
build an almond launcher, then
start JupyterLab in the current directory.
-
From the JupyterLab instance, select the kernel "Scala (sources)".
Optionally, pass a Scala version and / or JupyterLab options, like
Pass --help or see this page for the available options.
Watch for source changes
You can re-build a launcher upon source changes with
$ ./mill -w launcherFast
-
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill jupyter, beware to pass the same version to ./mill -w launcher.
You can re-build a launcher upon source changes with
$ ./mill -w dev.launcherFast
+
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill dev.jupyter, beware to pass the same version to ./mill -w dev.launcher.
Useful commands
List available Scala versions
$ ./mill scalaVersions
+ List available Scala versions
$ ./mill dev.scalaVersions
2.13.4
2.13.3
…
Print the latest supported Scala 2.13 version
$ ./mill scala213
+ Print the latest supported Scala 2.13 version
$ ./mill dev.scala213
2.13.4
Print the latest supported Scala 2.12 version
$ ./mill scala212
+ Print the latest supported Scala 2.12 version
Almond is built with mill. A mill launcher ships with almond, so that you don't need to install mill yourself. We list below useful commands, to help get you started using mill to build almond.
Run a Jupyter notebook server without installing a kernel
$ ./mill -i jupyterFast
+ Run a Jupyter notebook server without installing a kernel
$ ./mill -i dev.jupyterFast
This should
build an almond launcher, then
start JupyterLab in the current directory.
-
From the JupyterLab instance, select the kernel "Scala (sources)".
Optionally, pass a Scala version and / or JupyterLab options, like
Pass --help or see this page for the available options.
Watch for source changes
You can re-build a launcher upon source changes with
$ ./mill -w launcherFast
-
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill jupyter, beware to pass the same version to ./mill -w launcher.
You can re-build a launcher upon source changes with
$ ./mill -w dev.launcherFast
+
If you ran a JupyterLab server from the almond sources, you can restart the kernel from a notebook via JupyterLab to pick a newly built launcher. If you passed a Scala version to ./mill dev.jupyter, beware to pass the same version to ./mill -w dev.launcher.
Useful commands
List available Scala versions
$ ./mill scalaVersions
+ List available Scala versions
$ ./mill dev.scalaVersions
2.13.4
2.13.3
…
Print the latest supported Scala 2.13 version
$ ./mill scala213
+ Print the latest supported Scala 2.13 version
$ ./mill dev.scala213
2.13.4
Print the latest supported Scala 2.12 version
$ ./mill scala212
+ Print the latest supported Scala 2.12 version
Before diving into how Almond starts, a few words about the Almond installation. The installation of Almond can look complex by some aspects. Beyond that, customizing the installation, making sense of what it does in detail, can be tricky. This originates from two concerns that Almond tries to reconcile:
+
+
isolation of the internal dependencies of Almond from users
+
allowing the customization / tweaking of the Almond class path upon installation
Almond depends on fs2 and cats-effect. In order not to force its own cats / cats-effect / fs2 versions upon users, these internal dependencies of Almond are effectively hidden from users.
In more detail, isolation in Almond works this way: the scala-kernel-api module and its dependencies are user-facing. The rest of the Almond class path (the scala-kernel module and its dependencies, but for everything already pulled by scala-kernel-api) are internal dependencies, hidden from users.
Isolation between these two set of dependencies is achieved by starting Almond using launchers generated by coursier. (Note that the coursier documentation doesn't really detail how dependency isolation works. We detail that right below here.)
If we put dependencies isolation aside, these launchers are a small Java application, alongside a resource file listing URLs of the JARs that should be loaded. Upon startup, the launcher reads the URL list, check if these are in the coursier cache and downloads them if they're not. Then it creates a ClassLoader, loads the local copies of the JARs in it, and loads and calls the main class of the application from it.
With dependencies isolation enabled, these launchers actually embed two lists of JARs: a "top" one (corresponding to user dependencies for Almond), and a "bottom" one (for Almond, internal dependencies, with cats-effect, fs2, etc.). Upon startup, they create a first class loader with the top dependencies, and a second one, having the first class loader as a parent, with the bottom dependencies. In such cases, the way class loaders work on the JVM makes the classes loaded in the bottom class loader "see" the classes in the top one, but not the other way around. Later on, when the app runs, it can ask for the class loader of a class it knows is part of the top dependencies, which gives it a reference to the top class loader, that knows nothing about the bottom dependencies.
Almond relies on that mechanism to get a class loader that only knows about user-facing dependencies. That class loader is used as a parent of the class loader that's going to load the classes generated during the session (those corresponding to the input user code in the notebook). That way, users only "see" user-facing dependencies, not internal ones, and they can load whichever other version of the same internal dependencies as Almond.
When passed the --install option, Almond tries to determine the path of its own launcher, and copies it alongside the kernel.json file it generates for Jupyter to be able to launch Almond. That way, the launcher used during the installation can be safely deleted right after the installation.
This section describes how to install Almond for a specific Scala version. Even though the instructions below may rely on newer coursier features, installing Almond this way has been the recommended way to proceed since Almond exists. In the next section, we'll describe a novel way to install Almond, relying on an intermediate launcher, that allows notebook users to customize the Scala version, the JVM they use, Java options (including memory options), on a per notebook basis.
The easiest way to generate an Almond launcher consists in… not generating one. The cs launch command, when passed --use-bootstrap, launches an application via a temporary launcher it generates on-the-fly.
+ Creating an Almond launcher and installing it - newer launcher
The newer launcher allows users to configure Almond in the first cells of notebooks (more precisely, before any actual code - that is, not comments or blank lines - needs to be compiled), with directives like
//> using scala "2.12"
+//> using scala "2.12.19"
+//> using jvm "17"
+//> using javaOpt "-Xmx10g"
+//> using javaOpt "-Dfoo=bar"
+
Important thing to note here: the Scala version must be passed as argument to Almond itself, rather than to cs. The launcher uses its own Scala version (Scala 3), then, later on, spawns the same Almond kernel as above on its own, that takes over as a kernel. The argument passed via --scala corresponds to the default Scala version that will be used, if users don't specify a version of their own via a directive like //> using scala "2.13.14". Specify that option is optional: without it, Almond will default to the Scala 3 version that Almond uses at the time of its release (which should be the latest stable Scala 3 version at the time of the release).
To list the options that the launcher accepts, run
Custom protocol support for java.net.URL needs the JARs supporting it to be passed to the -cp option of java. Using the new launcher above, such JARs can be passed via --extra-startup-class-path, like
Support for the %sql magic, relying on Spark, needs to be enabled from a "predef" script. It also requires parts of the Toree magics support to be put in the user-facing part of the Almond class path (so that calls to it from the user side are picked up by Almond internals later on).
To enable that from a former launcher, pass --shared sh.almond::toree-hooks when generating the launcher:
If you want to use custom Maven repositories rather than Maven Central, you need to set COURSIER_REPOSITORIES at few places. It needs to be set when installing Almond, and when Jupyter starts Almond.
This can be achieved the following way, with the former launcher:
--env sets environment variables in the kernel spec that gets written when installing Almond. Jupyter sets those prior to launching Almond when users open notebooks.
Before diving into how Almond starts, a few words about the Almond installation. The installation of Almond can look complex by some aspects. Beyond that, customizing the installation, making sense of what it does in detail, can be tricky. This originates from two concerns that Almond tries to reconcile:
+
+
isolation of the internal dependencies of Almond from users
+
allowing the customization / tweaking of the Almond class path upon installation
Almond depends on fs2 and cats-effect. In order not to force its own cats / cats-effect / fs2 versions upon users, these internal dependencies of Almond are effectively hidden from users.
In more detail, isolation in Almond works this way: the scala-kernel-api module and its dependencies are user-facing. The rest of the Almond class path (the scala-kernel module and its dependencies, but for everything already pulled by scala-kernel-api) are internal dependencies, hidden from users.
Isolation between these two set of dependencies is achieved by starting Almond using launchers generated by coursier. (Note that the coursier documentation doesn't really detail how dependency isolation works. We detail that right below here.)
If we put dependencies isolation aside, these launchers are a small Java application, alongside a resource file listing URLs of the JARs that should be loaded. Upon startup, the launcher reads the URL list, check if these are in the coursier cache and downloads them if they're not. Then it creates a ClassLoader, loads the local copies of the JARs in it, and loads and calls the main class of the application from it.
With dependencies isolation enabled, these launchers actually embed two lists of JARs: a "top" one (corresponding to user dependencies for Almond), and a "bottom" one (for Almond, internal dependencies, with cats-effect, fs2, etc.). Upon startup, they create a first class loader with the top dependencies, and a second one, having the first class loader as a parent, with the bottom dependencies. In such cases, the way class loaders work on the JVM makes the classes loaded in the bottom class loader "see" the classes in the top one, but not the other way around. Later on, when the app runs, it can ask for the class loader of a class it knows is part of the top dependencies, which gives it a reference to the top class loader, that knows nothing about the bottom dependencies.
Almond relies on that mechanism to get a class loader that only knows about user-facing dependencies. That class loader is used as a parent of the class loader that's going to load the classes generated during the session (those corresponding to the input user code in the notebook). That way, users only "see" user-facing dependencies, not internal ones, and they can load whichever other version of the same internal dependencies as Almond.
When passed the --install option, Almond tries to determine the path of its own launcher, and copies it alongside the kernel.json file it generates for Jupyter to be able to launch Almond. That way, the launcher used during the installation can be safely deleted right after the installation.
This section describes how to install Almond for a specific Scala version. Even though the instructions below may rely on newer coursier features, installing Almond this way has been the recommended way to proceed since Almond exists. In the next section, we'll describe a novel way to install Almond, relying on an intermediate launcher, that allows notebook users to customize the Scala version, the JVM they use, Java options (including memory options), on a per notebook basis.
The easiest way to generate an Almond launcher consists in… not generating one. The cs launch command, when passed --use-bootstrap, launches an application via a temporary launcher it generates on-the-fly.
+ Creating an Almond launcher and installing it - newer launcher
The newer launcher allows users to configure Almond in the first cells of notebooks (more precisely, before any actual code - that is, not comments or blank lines - needs to be compiled), with directives like
//> using scala "2.12"
+//> using scala "2.12.19"
+//> using jvm "17"
+//> using javaOpt "-Xmx10g"
+//> using javaOpt "-Dfoo=bar"
+
Important thing to note here: the Scala version must be passed as argument to Almond itself, rather than to cs. The launcher uses its own Scala version (Scala 3), then, later on, spawns the same Almond kernel as above on its own, that takes over as a kernel. The argument passed via --scala corresponds to the default Scala version that will be used, if users don't specify a version of their own via a directive like //> using scala "2.13.14". Specify that option is optional: without it, Almond will default to the Scala 3 version that Almond uses at the time of its release (which should be the latest stable Scala 3 version at the time of the release).
To list the options that the launcher accepts, run
Custom protocol support for java.net.URL needs the JARs supporting it to be passed to the -cp option of java. Using the new launcher above, such JARs can be passed via --extra-startup-class-path, like
Support for the %sql magic, relying on Spark, needs to be enabled from a "predef" script. It also requires parts of the Toree magics support to be put in the user-facing part of the Almond class path (so that calls to it from the user side are picked up by Almond internals later on).
To enable that from a former launcher, pass --shared sh.almond::toree-hooks when generating the launcher:
If you want to use custom Maven repositories rather than Maven Central, you need to set COURSIER_REPOSITORIES at few places. It needs to be set when installing Almond, and when Jupyter starts Almond.
This can be achieved the following way, with the former launcher:
--env sets environment variables in the kernel spec that gets written when installing Almond. Jupyter sets those prior to launching Almond when users open notebooks.
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.12 version,
$ cs launch almond --scala 2.13.12 -- --install
-
This installs almond with the default kernel id, scala, and default display name, "Scala".
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.14 version,
$ cs launch almond --scala 2.13.14 -- --install
+
This installs almond with the default kernel id, scala, and default display name, "Scala".
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.12 version,
$ cs launch almond --scala 2.13.12 -- --install
-
This installs almond with the default kernel id, scala, and default display name, "Scala".
Several versions of almond can be installed side-by-side. This is useful e.g. to have kernels for several Scala versions, or to test newer / former almond versions.
To install several version of the kernel side-by-side, just ensure the different installed versions have different ids (required) and display names (recommended).
For example, let's install almond for the scala 2.13.14 version,
$ cs launch almond --scala 2.13.14 -- --install
+
This installs almond with the default kernel id, scala, and default display name, "Scala".
When launched by other users, the kernel installed by the default command downloads its dependencies in the user-specific coursier cache upon first launch.
You may prefer to have the launcher embed all these JARs, so that nothing needs to be downloaded or picked from a cache upon launch. Passing --standalone to the coursier bootstrap command generates such a launcher,
$ coursier bootstrap --standalone \
- almond:0.14.0-RC14 --scala 2.13.12 \
+ almond:0.14.0-RC15 --scala 2.13.14 \
-o almond
$ ./almond --install
$ rm -f almond # the generated launcher can be removed after install, it copied itself in the kernel installation directory
@@ -162,7 +163,7 @@
When launched by other users, the kernel installed by the default command downloads its dependencies in the user-specific coursier cache upon first launch.
You may prefer to have the launcher embed all these JARs, so that nothing needs to be downloaded or picked from a cache upon launch. Passing --standalone to the coursier bootstrap command generates such a launcher,
$ coursier bootstrap --standalone \
- almond:0.14.0-RC14 --scala 2.13.12 \
+ almond:0.14.0-RC15 --scala 2.13.14 \
-o almond
$ ./almond --install
$ rm -f almond # the generated launcher can be removed after install, it copied itself in the kernel installation directory
@@ -162,7 +163,7 @@
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here).
+
Note the -- before --install, separating the arguments passed to Almond from the ones handled by coursier.
You can specify explicit Almond and / or Scala versions, like
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here.
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --fork almond -- --install --force). That will override any previous almond (or kernel with name scala).
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --use-bootstrap almond -- --install --force). That will override any previous almond (or kernel with name scala).
Uninstall the almond kernel
To uninstall the almond kernel, use jupyter kernelspec remove scala.
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here).
+
Note the -- before --install, separating the arguments passed to Almond from the ones handled by coursier.
You can specify explicit Almond and / or Scala versions, like
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here.
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --fork almond -- --install --force). That will override any previous almond (or kernel with name scala).
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --use-bootstrap almond -- --install --force). That will override any previous almond (or kernel with name scala).
Uninstall the almond kernel
To uninstall the almond kernel, use jupyter kernelspec remove scala.
Almond comes with a Spark integration module called almond-spark, which allows you to connect to a Spark cluster and to run Spark calculations interactively from a Jupyter notebook.
It is based on ammonite-spark, adding Jupyter specific features such as progress bars and cancellation for running Spark computations.
ammonite-spark handles loading Spark in a clever way, and does not rely on a specific Spark distribution. Because of that, you can use it with any Spark 2.x version. The only limitation is that the Scala version of Spark and the running Almond kernel must match, so make sure your kernel uses the same Scala version as your Spark cluster. Spark 2.0.x - 2.3.x requires Scala 2.11. Spark 2.4.x supports both Scala 2.11 and 2.12.
Note that as of almond 0.7.0, almond only supports Scala 2.12 and therefore requires Spark 2.4.x for Scala 2.12.
For more information, see the README of ammonite-spark.
To use it, import the almond-spark dependency as well as Spark 2.x itself.
import $ivy.`org.apache.spark::spark-sql:2.4.0` // Or use any other 2.x version here
-import $ivy.`sh.almond::almond-spark:0.14.0-RC14` // Not required since almond 0.7.0 (will be automatically added when importing spark)
+import $ivy.`sh.almond::almond-spark:0.14.0-RC15` // Not required since almond 0.7.0 (will be automatically added when importing spark)
Usually you want to disable logging in order to avoid polluting your cell outputs:
Almond comes with a Spark integration module called almond-spark, which allows you to connect to a Spark cluster and to run Spark calculations interactively from a Jupyter notebook.
It is based on ammonite-spark, adding Jupyter specific features such as progress bars and cancellation for running Spark computations.
ammonite-spark handles loading Spark in a clever way, and does not rely on a specific Spark distribution. Because of that, you can use it with any Spark 2.x version. The only limitation is that the Scala version of Spark and the running Almond kernel must match, so make sure your kernel uses the same Scala version as your Spark cluster. Spark 2.0.x - 2.3.x requires Scala 2.11. Spark 2.4.x supports both Scala 2.11 and 2.12.
Note that as of almond 0.7.0, almond only supports Scala 2.12 and therefore requires Spark 2.4.x for Scala 2.12.
For more information, see the README of ammonite-spark.
To use it, import the almond-spark dependency as well as Spark 2.x itself.
import $ivy.`org.apache.spark::spark-sql:2.4.0` // Or use any other 2.x version here
-import $ivy.`sh.almond::almond-spark:0.14.0-RC14` // Not required since almond 0.7.0 (will be automatically added when importing spark)
+import $ivy.`sh.almond::almond-spark:0.14.0-RC15` // Not required since almond 0.7.0 (will be automatically added when importing spark)
Usually you want to disable logging in order to avoid polluting your cell outputs:
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here).
+
Note the -- before --install, separating the arguments passed to Almond from the ones handled by coursier.
You can specify explicit Almond and / or Scala versions, like
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here.
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --fork almond -- --install --force). That will override any previous almond (or kernel with name scala).
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --use-bootstrap almond -- --install --force). That will override any previous almond (or kernel with name scala).
Uninstall the almond kernel
To uninstall the almond kernel, use jupyter kernelspec remove scala.
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here).
+
Note the -- before --install, separating the arguments passed to Almond from the ones handled by coursier.
You can specify explicit Almond and / or Scala versions, like
Short Scala versions, like just 2.12 or 2.13, are accepted too. The available versions of Almond can be found here. Not all Almond and Scala versions combinations are available. See the possible combinations here.
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --fork almond -- --install --force). That will override any previous almond (or kernel with name scala).
To update the almond kernel, just re-install it, but passing the --force option to almond (like ./coursier launch --use-bootstrap almond -- --install --force). That will override any previous almond (or kernel with name scala).
Uninstall the almond kernel
To uninstall the almond kernel, use jupyter kernelspec remove scala.
Almond comes with a Spark integration module called almond-spark, which allows you to connect to a Spark cluster and to run Spark calculations interactively from a Jupyter notebook.
It is based on ammonite-spark, adding Jupyter specific features such as progress bars and cancellation for running Spark computations.
ammonite-spark handles loading Spark in a clever way, and does not rely on a specific Spark distribution. Because of that, you can use it with any Spark 2.x version. The only limitation is that the Scala version of Spark and the running Almond kernel must match, so make sure your kernel uses the same Scala version as your Spark cluster. Spark 2.0.x - 2.3.x requires Scala 2.11. Spark 2.4.x supports both Scala 2.11 and 2.12.
Note that as of almond 0.7.0, almond only supports Scala 2.12 and therefore requires Spark 2.4.x for Scala 2.12.
For more information, see the README of ammonite-spark.
To use it, import the almond-spark dependency as well as Spark 2.x itself.
import $ivy.`org.apache.spark::spark-sql:2.4.0` // Or use any other 2.x version here
-import $ivy.`sh.almond::almond-spark:0.14.0-RC14` // Not required since almond 0.7.0 (will be automatically added when importing spark)
+import $ivy.`sh.almond::almond-spark:0.14.0-RC15` // Not required since almond 0.7.0 (will be automatically added when importing spark)
Usually you want to disable logging in order to avoid polluting your cell outputs:
Almond comes with a Spark integration module called almond-spark, which allows you to connect to a Spark cluster and to run Spark calculations interactively from a Jupyter notebook.
It is based on ammonite-spark, adding Jupyter specific features such as progress bars and cancellation for running Spark computations.
ammonite-spark handles loading Spark in a clever way, and does not rely on a specific Spark distribution. Because of that, you can use it with any Spark 2.x version. The only limitation is that the Scala version of Spark and the running Almond kernel must match, so make sure your kernel uses the same Scala version as your Spark cluster. Spark 2.0.x - 2.3.x requires Scala 2.11. Spark 2.4.x supports both Scala 2.11 and 2.12.
Note that as of almond 0.7.0, almond only supports Scala 2.12 and therefore requires Spark 2.4.x for Scala 2.12.
For more information, see the README of ammonite-spark.
To use it, import the almond-spark dependency as well as Spark 2.x itself.
import $ivy.`org.apache.spark::spark-sql:2.4.0` // Or use any other 2.x version here
-import $ivy.`sh.almond::almond-spark:0.14.0-RC14` // Not required since almond 0.7.0 (will be automatically added when importing spark)
+import $ivy.`sh.almond::almond-spark:0.14.0-RC15` // Not required since almond 0.7.0 (will be automatically added when importing spark)
Usually you want to disable logging in order to avoid polluting your cell outputs:
