Added latest ALMA cycles configurations (#634)

* Scripts fetches and converts ALMA config files into OSKAR files 📟

* Latest configs for ALMA cycles 9, 10, 11 🚑

* Resolved type conflict (was complained about by mypy) 😕

* Updated documentation as suggested in comments to PR #634 ♍

doc/src/development.md

@@ -132,6 +132,29 @@ Code to be adjusted:
 
 Setup info to run the script can be found in the README in karabo.data._add_oskar_ska_layouts.
 
+## Add ALMA layouts as OSKAR telescopes
+
+The dishes of the ALMA telescope (Atacama Large Microwave Array) can operate in different configurations. These 'cycles' are set up at different points in time (see link [1]). For the next year (2025) three new cycles are added to the configuration: cycle 9, 10 and 11. There is a script that helps to import new cycle configurations to Karabo. It fetches the latest config files from the ALMA server (see link [2]) and converts them to an OSKAR telescope model (.tm directory). This can then be added to Karabo by copying it to `karabo/data/`. 
+
+The script and a README file can be found in `karabo.data._add_oskar_alma_layouts`.
+
+The files are fetched directly from the ALMA server. The url is not under our control. Therefore, the url may change which breaks the code. In this case have a look at link [2] and update the variable `CONFIG_FILE_URL` in the code.
+
+###  Setup
+1. Create and activate a Python virtual env / conda env or similar
+2. `pip install requests` should be enough
+3. Adjust code line `ALMA_CYCLE = 10` according to the desired version of the ALMA cycle.
+
+### Convert the configurations
+1. Run the script. The OSKAR .tm-foders are written to the current working directory
+2. Copy the directories to  `karabo/data/`
+3. Add the cycle name and version to `karabo/simulation/telescope.py` and `karabo/simulation/telescope_versions.py`, respectively.
+
+### Important links
+1. The ALMA configuration schedule: https://almascience.eso.org/news/the-alma-configuration-schedule-for-cycles-9-10-and-11
+2. The configuration files: https://almascience.eso.org/tools/casa-simulator
+
+
 ## Update Documentation
 
 The docs are built from the python source code and other doc source files located in /doc/src.

karabo/data/_add_oskar_alma_layouts/README.md

@@ -0,0 +1,12 @@
+## What this script is for
+The dishes of the ALMA telescope (Atacama Large Microwave Array) can operate in different configurations. These 'cycles' are set up at different points in time (see link [1]). For the next year (2025) three new cycles are added to the configuration: cycle 9, 10 and 11. This script can be used to download the latest or update the current configuration files.
+
+The files are fetched directly from the ALMA server. The url is not under our control. Therefore, the url may change which breaks the code. In this case have a look at link [2] and update the variable `CONFIG_FILE_URL` in the code.
+
+##  Setup
+1. Create and activate a Python virtual env / conda env / similar
+2. `pip install requests` should be enough
+
+## Important links:
+1. The ALMA configuration schedule: https://almascience.eso.org/news/the-alma-configuration-schedule-for-cycles-9-10-and-11
+2. The configuration files: https://almascience.eso.org/tools/casa-simulator

karabo/data/_add_oskar_alma_layouts/alma_config_to_oskar_tm.py

@@ -0,0 +1,148 @@
+""" Gets the ALMA telescope configuration files for an observation cycle.
+    The files are downloaded as an archive from the ALMA server. See
+    constant CONFIG_FILE_URL for the URL. It converts the configuration
+    files to an OSKAR telescope model directory. The folder are created
+    in the current working directory.
+    The script is based on the script 'array_config_to_oskar_tm.py'
+    by M. Pluess, FHNW. You can find the code here:
+    karabo/data/_add_oskar_ska_layouts.
+"""
+
+import glob
+import math
+import os
+import shutil
+import tarfile
+import tempfile
+from tarfile import TarFile
+from zipfile import ZipFile
+
+import requests
+
+ALMA_CYCLE = 10
+CONFIG_FILE_URL = f"https://almascience.eso.org/tools/documents-and-tools/cycle{ALMA_CYCLE}/alma-configuration-files"  # noqa: E501
+CONFIG_FILE_BASENAME = f"alma-cycle{ALMA_CYCLE}-configurations"
+
+# ALMA center coordinates
+ALMA_CENTER_LON = -67.7538
+ALMA_CENTER_LAT = -23.0234
+
+# The config files are returned in an archive. Which one is in the header.
+archive_type = {
+    "application/zip": "zip",
+    "application/x-tar": "tar",
+    "application/gzip": "gz",
+}
+
+
+def convert_to_oskar_file(cfg_filename: str) -> str:
+    """Convert a configuration file into an OSKAR tm-folder
+    Args:
+        cfg_filename: Full path to config-file (*.cfg)
+
+    Raises:
+        ValueError: In case the coordinate system in the file
+            is not a local tangent plane (LOC)
+
+    Returns:
+        Full path to OSKAR output dir
+    """
+    coordsys = None
+    stations = []
+    with open(cfg_filename, "r") as f:
+        for line in f:
+            line = line.rstrip()
+            if line.startswith("# COFA="):
+                long, lat = [float(s) for s in line.replace("# COFA=", "").split(",")]
+            elif line.startswith("# coordsys="):
+                # There may be a comment after coordinate system name
+                coordsys = line.replace("# coordsys=", "").split(" ")[0]
+            elif not line.startswith("#") and not line == "":
+                x, y, z = line.split(" ")[:3]
+                stations.append((x, y, z))
+
+    if coordsys != "LOC":
+        raise ValueError(
+            f"Found coordinate system {coordsys} but expected"
+            " LOC (Local tangent plane)."
+            " This current coordinate system is not supported."
+        )
+
+    assert len(stations) > 0
+    print(f"Parsed {len(stations)} stations")
+
+    oskar_output_dir = f"{os.path.splitext(cfg_filename)[0]}.tm"
+    os.makedirs(oskar_output_dir)
+
+    with open(os.path.join(oskar_output_dir, "position.txt"), "w") as f:
+        f.write(f"{ALMA_CENTER_LON} {ALMA_CENTER_LAT}\n")
+
+    with open(os.path.join(oskar_output_dir, "layout.txt"), "w") as f:
+        for x, y, z in stations:
+            f.write(f"{x} {y} {z}\n")
+
+    def get_nr_of_digits(i: int) -> int:
+        return math.floor(math.log10(i) if i > 0 else 0.0) + 1
+
+    target_nr_of_digits = get_nr_of_digits(len(stations))
+    for station_nr in range(len(stations)):
+        nr_of_digits = get_nr_of_digits(station_nr)
+        station_nr_leading_zeros = "0" * (target_nr_of_digits - nr_of_digits) + str(
+            station_nr
+        )
+
+        station_dir = os.path.join(
+            oskar_output_dir, f"station{station_nr_leading_zeros}"
+        )
+        os.mkdir(station_dir)
+        with open(os.path.join(station_dir, "layout.txt"), "w") as f:
+            f.write("0.0 0.0\n")
+
+    return oskar_output_dir
+
+
+# Setup temp dir as cache and get the file
+temp_dir = tempfile.mkdtemp()
+response = requests.get(CONFIG_FILE_URL)
+
+if response.status_code != 200:
+    raise ConnectionError(
+        "Could not download file."
+        f" Server resoponded with status code {response.status_code}"
+    )
+
+# The config files come in different archives.
+# Check the header to know which one.
+file_extension = archive_type[response.headers["Content-Type"]]
+if not file_extension:
+    raise ValueError("Could not deduce file type from request header")
+
+temp_filename = os.path.join(temp_dir, f"{CONFIG_FILE_BASENAME}.{file_extension}")
+
+with open(temp_filename, "wb") as tmpfile:
+    tmpfile.write(response.content)
+
+if file_extension == "zip":
+    with ZipFile(temp_filename) as configzip:
+        configzip.extractall(path=temp_dir)
+if file_extension == "tar":
+    with TarFile(temp_filename) as configtar:
+        configtar.extractall(path=temp_dir)
+if file_extension == "gz":
+    # This takes some extra work
+    with tarfile.open(temp_filename, "r:gz") as tgz:
+        tgz.extractall(path=temp_dir)
+        # scandir return entries in arbitrary order. We must pich the folder
+        with os.scandir(temp_dir) as it:
+            for entry in it:
+                if entry.is_dir():
+                    temp_dir = os.path.join(temp_dir, entry)
+                    break
+
+
+cfg_files = glob.glob(os.path.join(temp_dir, "*.cfg"))
+for file in cfg_files:
+    tmp_folder = convert_to_oskar_file(file)
+    shutil.copytree(tmp_folder, os.path.join(os.getcwd(), os.path.basename(tmp_folder)))
+
+shutil.rmtree(temp_dir)

karabo/data/aca.cycle10.named.tm/layout.txt

@@ -0,0 +1,10 @@
+-37.01384552	-551.9319303	-2.325092815  
+-49.28641965	-547.2313425	-2.314768924  
+-58.3714508	-591.2840731	-2.322794526  
+-38.88669906	-585.9530019	-2.317152015  
+-47.99531371	-564.8585951	-2.318302577  
+-55.96985522	-568.8204563	-2.321721131  
+-63.03702802	-574.7165969	-2.320317857  
+-48.84480314	-574.4357151	-2.325168129  
+-35.89239576	-569.6206755	-2.318648465  
+-65.31846157	-560.7014943	-2.320087842  

karabo/data/aca.cycle10.named.tm/position.txt

@@ -0,0 +1 @@
+-67.7538 -23.0234

karabo/data/aca.cycle10.named.tm/station00/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station01/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station02/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station03/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station04/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station05/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station06/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station07/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station08/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.named.tm/station09/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/layout.txt

@@ -0,0 +1,10 @@
+-37.01384552	-551.9319303	-2.325092815  
+-49.28641965	-547.2313425	-2.314768924  
+-58.3714508	-591.2840731	-2.322794526  
+-38.88669906	-585.9530019	-2.317152015  
+-47.99531371	-564.8585951	-2.318302577  
+-55.96985522	-568.8204563	-2.321721131  
+-63.03702802	-574.7165969	-2.320317857  
+-48.84480314	-574.4357151	-2.325168129  
+-35.89239576	-569.6206755	-2.318648465  
+-65.31846157	-560.7014943	-2.320087842  

karabo/data/aca.cycle10.tm/position.txt

@@ -0,0 +1 @@
+-67.7538 -23.0234

karabo/data/aca.cycle10.tm/station00/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station01/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station02/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station03/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station04/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station05/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station06/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station07/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station08/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle10.tm/station09/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/layout.txt

@@ -0,0 +1,10 @@
+-37.01384552	-551.9319303	-2.325092815  
+-49.28641965	-547.2313425	-2.314768924  
+-58.3714508	-591.2840731	-2.322794526  
+-38.88669906	-585.9530019	-2.317152015  
+-47.99531371	-564.8585951	-2.318302577  
+-55.96985522	-568.8204563	-2.321721131  
+-63.03702802	-574.7165969	-2.320317857  
+-48.84480314	-574.4357151	-2.325168129  
+-35.89239576	-569.6206755	-2.318648465  
+-65.31846157	-560.7014943	-2.320087842  

karabo/data/aca.cycle9.named.tm/position.txt

@@ -0,0 +1 @@
+-67.7538 -23.0234

karabo/data/aca.cycle9.named.tm/station00/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station01/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station02/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station03/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station04/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station05/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station06/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station07/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station08/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.named.tm/station09/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/layout.txt

@@ -0,0 +1,10 @@
+-37.01384552	-551.9319303	-2.325092815  
+-49.28641965	-547.2313425	-2.314768924  
+-58.3714508	-591.2840731	-2.322794526  
+-38.88669906	-585.9530019	-2.317152015  
+-47.99531371	-564.8585951	-2.318302577  
+-55.96985522	-568.8204563	-2.321721131  
+-63.03702802	-574.7165969	-2.320317857  
+-48.84480314	-574.4357151	-2.325168129  
+-35.89239576	-569.6206755	-2.318648465  
+-65.31846157	-560.7014943	-2.320087842  

karabo/data/aca.cycle9.tm/position.txt

@@ -0,0 +1 @@
+-67.7538 -23.0234

karabo/data/aca.cycle9.tm/station00/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station01/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station02/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station03/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station04/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station05/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station06/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station07/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station08/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/aca.cycle9.tm/station09/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/layout.txt

@@ -0,0 +1,43 @@
+-33.89412596 -712.7516484 -2.330089496
+-17.45390887 -709.6086972 -2.32867142
+-22.5589292 -691.9838607 -2.323742384
+-5.421013146 -723.7911045 -2.334250518
+25.24593899 -744.4318445 -2.336684256
+20.94634824 -721.464646 -2.333017829
+15.93453511 -700.6757482 -2.32967552
+9.482335836 -687.0764746 -2.729176727
+-9.835100334 -663.8265957 -2.722704263
+-20.89371519 -653.559982 -2.43766594
+37.82617818 -735.8760818 -2.484749162
+0.965953711 -702.6549839 -2.325874408
+10.73987943 -659.5673638 -2.726261907
+-20.62042134 -633.9073418 -2.527672839
+-6.712372688 -684.9519378 -2.3269437
+-41.23929872 -725.9777278 -2.33663097
+-50.30411069 -711.209498 -2.332024766
+-43.64407228 -652.7424613 -2.322696805
+-59.4042934 -667.5717395 -2.130365568
+-74.47461587 -684.6574595 -2.129342649
+-84.5230042 -704.8937799 -1.937681767
+-86.87286754 -732.2620426 -1.928810218
+-93.14292454 -745.9594809 -1.941492838
+-57.04992893 -645.4388456 -2.129055948
+-68.48107366 -698.470785 -2.334779889
+-95.8997254 -694.8901111 -1.927739915
+-105.4387377 -755.003357 -1.640752815
+-76.90232093 -718.1727584 -1.929073338
+-26.04186157 -726.1888088 -2.332575051
+-15.39951252 -746.3592279 -2.331879161
+-80.49715673 -765.1118313 -1.942599619
+-64.18907113 -771.1494896 -1.940145207
+-42.33954858 -777.8607093 -2.339781511
+-23.09339256 -778.6079363 -2.340774491
+2.382622676 -763.2855572 -2.838872989
+17.1053716 -766.5338029 -2.842286715
+-80.06703001 -780.3740238 -1.336451415
+-63.50521219 -786.8199568 -1.341060659
+-36.66707354 -762.1662821 -2.339843364
+-19.65399721 -794.5952314 -2.338743316
+-1.523355241 -788.9452581 -2.8370088
+30.35174917 -773.9188068 -3.336231664
+-15.04639784 -764.3348031 -2.335016467

karabo/data/alma.cycle10.1.tm/position.txt

@@ -0,0 +1 @@
+-67.7538 -23.0234

karabo/data/alma.cycle10.1.tm/station00/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station01/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station02/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station03/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station04/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station05/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station06/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station07/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station08/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station09/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station10/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station11/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station12/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station13/layout.txt

@@ -0,0 +1 @@
+0.0 0.0

karabo/data/alma.cycle10.1.tm/station14/layout.txt

@@ -0,0 +1 @@
+0.0 0.0