Skip to content

Commit

Permalink
Merge pull request #4 from esacdab/debian
Browse files Browse the repository at this point in the history
All about debian
  • Loading branch information
emmanuelmathot authored Mar 19, 2024
2 parents 879b153 + 70bd8be commit eeca39c
Show file tree
Hide file tree
Showing 27 changed files with 81 additions and 727 deletions.
502 changes: 0 additions & 502 deletions Use Cases/Scenario 1 - NDVI Mapping/config.sample.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion Use Cases/Scenario 1 - NDVI Mapping/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
If Python is not available, install it with the package manager and do point 3 again.

```console
$ sudo yum install python3 python3-pip
$ sudo apt-get install -y python3 python3-pip
```

4. Check that GDAL is available.
Expand Down
1 change: 0 additions & 1 deletion Use Cases/Scenario 2 - Rapid Mapping/conda-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
6 changes: 3 additions & 3 deletions Use Cases/Scenario 2 - Rapid Mapping/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

1. Provision a virtual machine on the target site. Preferably with the following specification
- 2 CPUs, 8GB RAM, 30GB disk
- CentOS 7
- Debian 11+
- Jupyter Lab (with Python 3 support), it can also be installed if not present.
- With data offer access if required

Expand Down Expand Up @@ -227,8 +227,8 @@

```console
$ ls -l ~
-rw-rw-r--. 1 centos centos 16212154 Jun 24 17:37 burned_area_20210305_20210424.rgb.tif
-rw-rw-r--. 1 centos centos 130225905 Jun 24 17:38 burned_area_20210305_20210424.tif
-rw-rw-r--. 1 debian debian 16212154 Jun 24 17:37 burned_area_20210305_20210424.rgb.tif
-rw-rw-r--. 1 debian debian 130225905 Jun 24 17:38 burned_area_20210305_20210424.tif
```

7. Download them to your computer and open it with any tool that can visualise TIFF files. Verify that the band/layer `NBR` shows an RGB image of the burn severity of the area of interest. [100%]
Expand Down
2 changes: 1 addition & 1 deletion Use Cases/Scenario 3 - Mosaicking/conda-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
sudo apt-get install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
2 changes: 1 addition & 1 deletion Use Cases/Scenario 3 - Mosaicking/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
1. Provision a virtual machine on the target site. Preferably with the following specification

- 2 CPUs, 8 GB RAM, 100 GB disk
- CentOS 7
- Debian 11+
- Python 3
- With data offer access if available

Expand Down
2 changes: 1 addition & 1 deletion Use Cases/Scenario 4 - Trends Mapping/conda-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
sudo apt-get install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
14 changes: 7 additions & 7 deletions Use Cases/Scenario 4 - Trends Mapping/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
1. Provision a virtual machine on the target site. Preferably with the following specification

- 2 CPUs, 8 GB RAM, 100 GB disk
- CentOS 7
- debian 7
- Python 3
- With data offer access if available

Expand Down Expand Up @@ -191,12 +191,12 @@
```console
$ ls -l output_data/
-rw-rw-r--. 1 centos centos 1702 May 31 16:40 graph_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.xml
-rw-rw-r--. 1 centos centos 1702 May 31 16:37 graph_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.xml
-rw-rw-r--. 1 centos centos 294126 May 31 16:42 lst_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 centos centos 925298 May 31 16:40 lst_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
-rw-rw-r--. 1 centos centos 12339468 May 31 16:42 rgba_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 centos centos 10451496 May 31 16:40 rgba_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 1702 May 31 16:40 graph_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.xml
-rw-rw-r--. 1 debian debian 1702 May 31 16:37 graph_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.xml
-rw-rw-r--. 1 debian debian 294126 May 31 16:42 lst_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 925298 May 31 16:40 lst_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 12339468 May 31 16:42 rgba_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 10451496 May 31 16:40 rgba_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
```

7. Download the files `rgba_*.tif` onto your computer and open it with a tool such as QGIS. Verify that it shows the desired information (land surface temperature) of the area of interest. [100%]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
sudo apt-get install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

1. Provision a virtual machine on the target site. Preferably with the following specification:
- 4 CPU, 32GB RAM, 100GB disk
- CentOS 7
- Debian 11+ or similar (Ubuntu)
- With data offer access if required

2. Open a terminal on the provisioned machine and install some prequisites, in case they are not yet present on the machine.

```
sudo yum install -y vim tree wget unzip libgfortran-4.8.5
sudo apt-get install -y bc wget unzip libgfortran5-amd64-cross
```
3. Install, if necessary, **conda** on the virtual machine and create the conda environment. Conda is needed as the vehicle to install the SNAP toolbox.
Expand Down
16 changes: 2 additions & 14 deletions src/cdab-client/cdab-client
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,8 @@
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/cdab-client

if [[ -n "$@" && $@ == *"TS1"* ]]; then
newargs="$@"
newargs=${newargs/-tn/-ts}
newargs=${newargs/--target_name/-ts}
newargs=${newargs/--target_credentials/-tc}
newargs=${newargs/-tu/-te}
newargs=${newargs/-tsn/-n}
newargs=${newargs/--testsite_name/-n}
newargs=${newargs/--load_factor/-lf}

newargs=`echo ${newargs} | sed -e 's/\( -\w*\)\ /\1=/g'`

echo "Command: /usr/bin/cdab-remote-client $newargs" >&2

exec /usr/bin/cdab-remote-client $newargs
echo "Use cdab-remote-client for this test" >&2
exit 1
fi

exec /usr/lib/cdab-client/cdab-testtools "$@"
20 changes: 10 additions & 10 deletions src/cdab-client/config.sample.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -685,7 +685,7 @@ service_providers:
two_factor_authentication_key: "KEY"
vm_name: "cdab-test"
key_name: "cdab-cdse-vmkey"
image_name: "CentOS 7"
image_name: "Debian 12"
flavor_name: "hm.medium"
cost_monthly: 70.04
cost_hourly: 0.114
Expand Down Expand Up @@ -1352,7 +1352,7 @@ service_providers:
identity_api_version: 3
vm_name: "cdab-test"
key_name: "cdab-cdse-vmkey"
image_name: "Standard_CentOS_7_latest"
image_name: "Standard_Debian_12_latest"
# From T-Systems Elastic Cloud Server creation wizard
# https://open-telekom-cloud.com/en/prices/price-calculator
flavor_name: "s3.large.8"
Expand Down Expand Up @@ -1728,7 +1728,7 @@ service_providers:
key_name: "cdab-key-onda"
# Name of image to be used for new virtual machine
# (choose from Compute > Images on the OpenStack dashboard).
image_name: "Centos 7"
image_name: "Debian 12"
# Name of flavour for new virtual machine
# (check under Compute > Instances > Launch Instance > Flavours/Flavors on the OpenStack dashboard).
# From https://www.onda-dias.eu/cms/services/catalogues/virtual-servers
Expand Down Expand Up @@ -1767,7 +1767,7 @@ service_providers:
# (must correspond to public key in key_name).
private_key_file: "/home/jenkins/cdab-key-onda.pem"
# User on virtual machine for SSH connections.
remote_user: "centos"
remote_user: "debian"
#Create an external volume for docker image and test execution; this is useful for flavours that have very limited main disk.
# The size of the additional disk is 20 GB.
#use_volume: True
Expand Down Expand Up @@ -2169,7 +2169,7 @@ service_providers:
identity_api_version: 3
vm_name: "cdab-test"
key_name: "cdab-key-creodias"
image_name: "CentOS 7"
image_name: "Debian 12"
# From https://creodias.eu/price-list
# eo2.large = 2 vCPU, 8GB RAM, 32GB HD, 51.57 EUR per month, 0.084 EUR per hour
# eo2.xlarge = 4 vCPU, 16GB RAM, 64GB HD, 103.15 EUR per month, 0.169 EUR per hour
Expand Down Expand Up @@ -2366,7 +2366,7 @@ service_providers:
compute:
connector: google
region_name: "europe-west1-b"
image_name: "projects/centos-cloud/global/images/centos-7-v20220303"
image_name: "product/google/debian12"
#flavor_name: "projects/modular-buffer-235216/zones/europe-west1-b/machineTypes/n1-highmem-2"
# n1-highmem-2 = 2 vCPU, 13GB RAM
# n2-highmem-2 = 2 vCPU, 16GB RAM
Expand Down Expand Up @@ -2492,7 +2492,7 @@ service_providers:
flavor_name: "r5a.large"
security_group: "sg-0cc75812b649925ce"
private_key_file: "/home/jenkins/cdab-key-aws.pem"
remote_user: "centos"
remote_user: "debian"
# https://calculator.aws -> EC2
cost_monthly: 94.27
cost_hourly: 0.1291
Expand Down Expand Up @@ -3208,7 +3208,7 @@ service_providers:
vm_name: "cdab-test"
key_name: "cdab-key-wekeo"
private_key_file: "/home/jenkins/cdab-key-wekeo.pem"
image_name: "CentOS 7"
image_name: "Debian 12"
# hm.medium = 2 vCPU, 16GB RAM, 64GB HD
flavor_name: "hm.medium"
cost_monthly: 70.04
Expand Down Expand Up @@ -3285,8 +3285,8 @@ service_providers:
vm_name: "cdab-test"
image:
publisher: "OpenLogic"
offer: "CentOS"
sku: "7_8-gen2"
offer: "Debian"
sku: "12"
version: "latest"
# Memory optimized E2s v3 = 2 vCPU, 16GB RAM, 32GB HD
flavor_name: "Standard_E2s_v3"
Expand Down
22 changes: 1 addition & 21 deletions src/cdab-remote-client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,26 +16,6 @@ The **cdab-remote-client** is a command line utility that performs a local CDAB

Note: The Openstack terminology is inconsistent and might be confusing; **servers** (as in the command-line client) and **instances** (as on the dashboard) refer to the same thing, **virtual machines**, as they are called below.

### Prerequisites

* CentOS 7
* Python 3
* OpenStack client (a Python module that provides a command-line tool for OpenStack access)
* The Amazon AWS SDK for Python
* The Google API Client Library for Python
* The Microsoft Azure libraries for Python
* Root or proper _sudo_ permissions

### Installation

* Add the terradue-stable-el7 repository configuration on YUM,
* Install the tool with:

```
$ sudo yum install cdab-remote-client -y
```


## Usage

The main parameters are shown on the help page.
Expand Down Expand Up @@ -166,7 +146,7 @@ Most of the values for the various keys can be obtained from the OpenStack dashb
* **project_id**: Name of the project on the Google Cloud Platform.
* **region_name**: Zone, e.g. *europe-west2-c*.
* **key_name**: This setting is not applicable, as Google creates all accounts for which there are keys defined and authorises all keys. Key pairs are defined under https://console.cloud.google.com/compute/metadata/sshKeys.
* **image_name**: Fully qualified name of boot disk image to be used for new virtual machine, e.g. *projects/centos-cloud/global/images/centos-7-v20200309*.
* **image_name**: Fully qualified name of boot disk image to be used for new virtual machine, e.g. *projects/debian-cloud/global/images/debian-12*.
* **flavor_name**: The machine type name name, e.g. *e2-standard-2*. Can also be specified, as in previous version, as fully qualified name of machine type for new virtual machine, e.g *projects/<project-name>/zones/europe-west2-c/machineTypes/e2-standard-2*.

##### Specific settings for Amazon AWS EC2
Expand Down
2 changes: 1 addition & 1 deletion src/cdab-remote-client/bin/cdab-remote-client
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
export PYTHONIOENCODING='utf8'
command="python3 /usr/lib/cdab-remote-client/cdab-remote-client.py3";
for a in "$@"; do command="${command} \"${a/"/\\\\"/}\""; done

python3 /usr/lib/cdab-remote-client/cdab-remote-client.py3 "$@"
98 changes: 0 additions & 98 deletions src/cdab-remote-client/etc/config.yaml.sample

This file was deleted.

Loading

0 comments on commit eeca39c

Please sign in to comment.