Installation - added NMBU

D2Dbook/2_Data_sources/APIs/downloads/weather.json

@@ -13,31 +13,31 @@
     ],
     "base": "stations",
     "main": {
-        "temp": 289.65,
-        "feels_like": 289.71,
-        "temp_min": 288.66,
-        "temp_max": 290,
-        "pressure": 1011,
-        "humidity": 90,
-        "sea_level": 1011,
-        "grnd_level": 994
+        "temp": 287.11,
+        "feels_like": 287.04,
+        "temp_min": 286.92,
+        "temp_max": 287.64,
+        "pressure": 1008,
+        "humidity": 95,
+        "sea_level": 1008,
+        "grnd_level": 992
     },
     "visibility": 10000,
     "wind": {
-        "speed": 4.17,
-        "deg": 144,
-        "gust": 10.11
+        "speed": 3.99,
+        "deg": 202,
+        "gust": 10.74
     },
     "clouds": {
-        "all": 98
+        "all": 96
     },
-    "dt": 1695899545,
+    "dt": 1695970926,
     "sys": {
         "type": 2,
         "id": 2006772,
         "country": "NO",
-        "sunrise": 1695878026,
-        "sunset": 1695920465
+        "sunrise": 1695964565,
+        "sunset": 1696006684
     },
     "timezone": 7200,
     "id": 3139081,

D2Dbook/6_Appendix/Installation.ipynb

@@ -6,7 +6,10 @@
    "source": [
     "# Installation\n",
     "## General note on Windows installations\n",
-    "When a system wide change like adding/changing environment variables or installing a program is performed, this does not appear in the current session of a Jupyter Notebook or other module that is started (directly or indirectly) from the command line. A restart of the whole Jupyter Notebook server may therefore be needed after the mentioned changes to see their effects."
+    "When a system wide change like adding/changing environment variables or installing a program is performed, this does not appear in the current session of a Jupyter Notebook or other module that is started (directly or indirectly) from the command line. A restart of the whole Jupyter Notebook server may therefore be needed after the mentioned changes to see their effects.\n",
+    "\n",
+    "### Company managed Windows computers\n",
+    "Some companies and organisations, e.g., the Norwegian University of Life Sciences (NMBU), do not give administrative rights to their employees by default. Instead they have a Software Center with installable programs. For NMBU the Software Center includes Anaconda, VS Code, Docker (with automatic WSL installation), Java, and Power BI Desktop."
    ]
   },
   {
@@ -29,10 +32,26 @@
     "    ```\n",
     "    conda create --name D2D_env\n",
     "    conda activate D2D_env\n",
+    "    conda install pip\n",
     "    pip install -r /path_to_D2Dbook/requirements.txt\n",
     "    ```"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Integrated Developer Environment - IDE\n",
+    "- There are many to choose between, e.g., the poular freeware applications [PyCharm](https://www.jetbrains.com/pycharm/) and Visual Studio Code ([VS Code](https://code.visualstudio.com/)).\n",
+    "- This book was developed using VS Code.\n",
+    "\n",
+    "### VS Code\n",
+    "- Download and install [VS Code](https://code.visualstudio.com/).\n",
+    "- Add extensions for Python and Jupyter, possibly also for JSON viewer, GitHub Copilot, etc.\n",
+    "- You may have to set the Conda and Python paths in the settings for the Python extension.\n",
+    "- If your favourite Conda environment is not detected, press Ctrl/Cmd+Shift+P and write _python:Select Interpreter_ to enable manual input of environment path."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -69,7 +88,6 @@
     "To install a newer version from the terminal you can write the following:  \n",
     "```\n",
     "pip install git+https://github.com/datastax/python-driver.git\n",
-    "pip install cql\n",
     "```"
    ]
   },

D2Dbook/_build/html/2_Data_sources/Databases/Spark.html

@@ -509,12 +509,12 @@ <h2>Spark and Cassandra<a class="headerlink" href="#spark-and-cassandra" title="
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>23/09/28 13:16:55 WARN Utils: Your hostname, Kristians-MacBook-Pro.local resolves to a loopback address: 127.0.0.1; using 10.42.126.12 instead (on interface en0)
-23/09/28 13:16:55 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>23/09/29 09:04:40 WARN Utils: Your hostname, Kristians-MacBook-Pro.local resolves to a loopback address: 127.0.0.1; using 10.42.126.20 instead (on interface en0)
+23/09/29 09:04:40 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
 Ivy Default Cache set to: /Users/kristian/.ivy2/cache
 The jars for the packages stored in: /Users/kristian/.ivy2/jars
 com.datastax.spark#spark-cassandra-connector_2.12 added as a dependency
-:: resolving dependencies :: org.apache.spark#spark-submit-parent-fdd44103-4973-4e43-8bb1-86c1052a4be5;1.0
+:: resolving dependencies :: org.apache.spark#spark-submit-parent-a03259f5-1d01-425e-a608-e03259fe2e3d;1.0
 	confs: [default]
 </pre></div>
 </div>
@@ -530,19 +530,19 @@ <h2>Spark and Cassandra<a class="headerlink" href="#spark-and-cassandra" title="
 	found com.typesafe#config;1.4.1 in central
 	found org.slf4j#slf4j-api;1.7.26 in central
 	found io.dropwizard.metrics#metrics-core;4.1.18 in central
-</pre></div>
-</div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>	found org.hdrhistogram#HdrHistogram;2.1.12 in central
+	found org.hdrhistogram#HdrHistogram;2.1.12 in central
 	found org.reactivestreams#reactive-streams;1.0.3 in central
 	found com.github.stephenc.jcip#jcip-annotations;1.0-1 in central
 	found com.github.spotbugs#spotbugs-annotations;3.1.12 in central
-	found com.google.code.findbugs#jsr305;3.0.2 in central
+</pre></div>
+</div>
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>	found com.google.code.findbugs#jsr305;3.0.2 in central
 	found com.datastax.oss#java-driver-mapper-runtime;4.13.0 in central
 	found com.datastax.oss#java-driver-query-builder;4.13.0 in central
 	found org.apache.commons#commons-lang3;3.10 in central
 	found com.thoughtworks.paranamer#paranamer;2.8 in central
 	found org.scala-lang#scala-reflect;2.12.11 in central
-:: resolution report :: resolve 454ms :: artifacts dl 14ms
+:: resolution report :: resolve 394ms :: artifacts dl 15ms
 	:: modules in use:
 	com.datastax.oss#java-driver-core-shaded;4.13.0 from central in [default]
 	com.datastax.oss#java-driver-mapper-runtime;4.13.0 from central in [default]
@@ -569,12 +569,12 @@ <h2>Spark and Cassandra<a class="headerlink" href="#spark-and-cassandra" title="
 	---------------------------------------------------------------------
 	|      default     |   19  |   0   |   0   |   0   ||   19  |   0   |
 	---------------------------------------------------------------------
-:: retrieving :: org.apache.spark#spark-submit-parent-fdd44103-4973-4e43-8bb1-86c1052a4be5
+:: retrieving :: org.apache.spark#spark-submit-parent-a03259f5-1d01-425e-a608-e03259fe2e3d
 	confs: [default]
-	0 artifacts copied, 19 already retrieved (0kB/9ms)
+	0 artifacts copied, 19 already retrieved (0kB/8ms)
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>23/09/28 13:16:56 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>23/09/29 09:04:41 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 Setting default log level to &quot;WARN&quot;.
 To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
 </pre></div>
@@ -596,10 +596,10 @@ <h2>Accessing tables<a class="headerlink" href="#accessing-tables" title="Permal
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>+---+----------+-------+
 |ind|   company|  model|
 +---+----------+-------+
-|  3|  Polestar|      3|
 |  1|     Tesla|Model S|
 |  2|     Tesla|Model 3|
 |  4|Volkswagen|   ID.4|
+|  3|  Polestar|      3|
 +---+----------+-------+
 </pre></div>
 </div>
@@ -669,14 +669,11 @@ <h3><a class="reference external" href="https://spark.apache.org/docs/latest/api
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>+--------------------+-------+-------+-------+
 |                  id|company|  model|  price|
 +--------------------+-------+-------+-------+
-|83bb52d0-5df0-11e...|  Tesla|Model S|21000.0|
-|83bb2bc0-5df0-11e...|  Tesla|Model S|20000.0|
+|71ba8aa0-5e96-11e...|  Tesla|Model S|21000.0|
+|71ba3c80-5e96-11e...|  Tesla|Model S|20000.0|
 +--------------------+-------+-------+-------+
 </pre></div>
 </div>
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
-</pre></div>
-</div>
 </div>
 </div>
 <div class="cell docutils container">
@@ -690,8 +687,8 @@ <h3><a class="reference external" href="https://spark.apache.org/docs/latest/api
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>+--------------------+-------+-------+-------+
 |                  id|company|  model|  price|
 +--------------------+-------+-------+-------+
-|83bb2bc0-5df0-11e...|  Tesla|Model S|20000.0|
-|83bb52d0-5df0-11e...|  Tesla|Model S|21000.0|
+|71ba3c80-5e96-11e...|  Tesla|Model S|20000.0|
+|71ba8aa0-5e96-11e...|  Tesla|Model S|21000.0|
 +--------------------+-------+-------+-------+
 </pre></div>
 </div>
@@ -732,21 +729,21 @@ <h3><a class="reference external" href="https://spark.apache.org/docs/latest/api
   <tbody>
     <tr>
       <th>0</th>
-      <td>83bb52d0-5df0-11ee-8b2f-4f23491759a5</td>
+      <td>71ba3c80-5e96-11ee-8b2f-4f23491759a5</td>
       <td>Tesla</td>
       <td>Model S</td>
-      <td>21000.0</td>
+      <td>20000.0</td>
     </tr>
     <tr>
       <th>1</th>
-      <td>83bb2bc0-5df0-11ee-8b2f-4f23491759a5</td>
+      <td>71ba8aa0-5e96-11ee-8b2f-4f23491759a5</td>
       <td>Tesla</td>
       <td>Model S</td>
-      <td>20000.0</td>
+      <td>21000.0</td>
     </tr>
     <tr>
       <th>2</th>
-      <td>83bba0f0-5df0-11ee-8b2f-4f23491759a5</td>
+      <td>71bab1b0-5e96-11ee-8b2f-4f23491759a5</td>
       <td>Oldsmobile</td>
       <td>Model 6C</td>
       <td>135000.0</td>
@@ -791,17 +788,17 @@ <h3><a class="reference external" href="https://spark.apache.org/docs/latest/api
   <tbody>
     <tr>
       <th>0</th>
-      <td>83bb52d0-5df0-11ee-8b2f-4f23491759a5</td>
+      <td>71ba3c80-5e96-11ee-8b2f-4f23491759a5</td>
       <td>Tesla</td>
       <td>Model S</td>
-      <td>21000.0</td>
+      <td>20000.0</td>
     </tr>
     <tr>
       <th>1</th>
-      <td>83bb2bc0-5df0-11ee-8b2f-4f23491759a5</td>
+      <td>71ba8aa0-5e96-11ee-8b2f-4f23491759a5</td>
       <td>Tesla</td>
       <td>Model S</td>
-      <td>20000.0</td>
+      <td>21000.0</td>
     </tr>
   </tbody>
 </table>
@@ -842,7 +839,7 @@ <h3><a class="reference external" href="https://spark.apache.org/docs/latest/api
   <tbody>
     <tr>
       <th>0</th>
-      <td>83bb52d0-5df0-11ee-8b2f-4f23491759a5</td>
+      <td>71ba8aa0-5e96-11ee-8b2f-4f23491759a5</td>
       <td>Tesla</td>
       <td>Model S</td>
       <td>21000.0</td>
@@ -1019,9 +1016,9 @@ <h3>Write data to Cassandra<a class="headerlink" href="#write-data-to-cassandra"
   <tbody>
     <tr>
       <th>0</th>
-      <td>3</td>
-      <td>Polestar</td>
-      <td>3</td>
+      <td>4</td>
+      <td>Volkswagen</td>
+      <td>ID.4</td>
     </tr>
     <tr>
       <th>1</th>
@@ -1037,22 +1034,22 @@ <h3>Write data to Cassandra<a class="headerlink" href="#write-data-to-cassandra"
     </tr>
     <tr>
       <th>3</th>
-      <td>4</td>
-      <td>Volkswagen</td>
-      <td>ID.4</td>
-    </tr>
-    <tr>
-      <th>4</th>
       <td>1</td>
       <td>Tesla</td>
       <td>Model S</td>
     </tr>
     <tr>
-      <th>5</th>
+      <th>4</th>
       <td>2</td>
       <td>Tesla</td>
       <td>Model 3</td>
     </tr>
+    <tr>
+      <th>5</th>
+      <td>3</td>
+      <td>Polestar</td>
+      <td>3</td>
+    </tr>
   </tbody>
 </table>
 </div></div></div>