Merge pull request #4 from ministryofjustice/unit-tests

Unit tests

.gitignore

@@ -9,6 +9,6 @@ env/
 *.sha256
 terraform.tfstate
 
-test_data/**.csv
+data_src/**.csv
 ./idea/*
 **/__pycache__/

README.md

@@ -1,64 +1,23 @@
-# Ministry of Justice Template Repository
+# Juniper mist integration script
 
-[![repo standards badge](https://img.shields.io/endpoint?labelColor=231f20&color=005ea5&style=for-the-badge&label=MoJ%20Compliant&url=https%3A%2F%2Foperations-engineering-reports.cloud-platform.service.justice.gov.uk%2Fapi%2Fv1%2Fcompliant_public_repositories%2Fendpoint%2Ftemplate-repository&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACgAAAAoCAYAAACM/rhtAAAABmJLR0QA/wD/AP+gvaeTAAAHJElEQVRYhe2YeYyW1RWHnzuMCzCIglBQlhSV2gICKlHiUhVBEAsxGqmVxCUUIV1i61YxadEoal1SWttUaKJNWrQUsRRc6tLGNlCXWGyoUkCJ4uCCSCOiwlTm6R/nfPjyMeDY8lfjSSZz3/fee87vnnPu75z3g8/kM2mfqMPVH6mf35t6G/ZgcJ/836Gdug4FjgO67UFn70+FDmjcw9xZaiegWX29lLLmE3QV4Glg8x7WbFfHlFIebS/ANj2oDgX+CXwA9AMubmPNvuqX1SnqKGAT0BFoVE9UL1RH7nSCUjYAL6rntBdg2Q3AgcAo4HDgXeBAoC+wrZQyWS3AWcDSUsomtSswEtgXaAGWlVI2q32BI0spj9XpPww4EVic88vaC7iq5Hz1BvVf6v3qe+rb6ji1p3pWrmtQG9VD1Jn5br+Knmm70T9MfUh9JaPQZu7uLsR9gEsJb3QF9gOagO7AuUTom1LpCcAkoCcwQj0VmJregzaipA4GphNe7w/MBearB7QLYCmlGdiWSm4CfplTHwBDgPHAFmB+Ah8N9AE6EGkxHLhaHU2kRhXc+cByYCqROs05NQq4oR7Lnm5xE9AL+GYC2gZ0Jmjk8VLKO+pE4HvAyYRnOwOH5N7NhMd/WKf3beApYBWwAdgHuCLn+tatbRtgJv1awhtd838LEeq30/A7wN+AwcBt+bwpD9AdOAkYVkpZXtVdSnlc7QI8BlwOXFmZ3oXkdxfidwmPrQXeA+4GuuT08QSdALxC3OYNhBe/TtzON4EziZBXD36o+q082BxgQuqvyYL6wtBY2TyEyJ2DgAXAzcC1+Xxw3RlGqiuJ6vE6QS9VGZ/7H02DDwAvELTyMDAxbfQBvggMAAYR9LR9J2cluH7AmnzuBowFFhLJ/wi7yiJgGXBLPq8A7idy9kPgvAQPcC9wERHSVcDtCfYj4E7gr8BRqWMjcXmeB+4tpbyG2kG9Sl2tPqF2Uick8B+7szyfvDhR3Z7vvq/2yqpynnqNeoY6v7LvevUU9QN1fZ3OTeppWZmeyzRoVu+rhbaHOledmoQ7LRd3SzBVeUo9Wf1DPs9X90/jX8m/e9Rn1Mnqi7nuXXW5+rK6oU7n64mjszovxyvVh9WeDcTVnl5KmQNcCMwvpbQA1xE8VZXhwDXAz4FWIkfnAlcBAwl6+SjD2wTcmPtagZnAEuA3dTp7qyNKKe8DW9UeBCeuBsbsWKVOUPvn+MRKCLeq16lXqLPVFvXb6r25dlaGdUx6cITaJ8fnpo5WI4Wuzcjcqn5Y8eI/1F+n3XvUA1N3v4ZamIEtpZRX1Y6Z/DUK2g84GrgHuDqTehpBCYend94jbnJ34DDgNGArQT9bict3Y3p1ZCnlSoLQb0sbgwjCXpY2blc7llLW1UAMI3o5CD4bmuOlwHaC6xakgZ4Z+ibgSxnOgcAI4uavI27jEII7909dL5VSrimlPKgeQ6TJCZVQjwaOLaW8BfyWbPEa1SaiTH1VfSENd85NDxHt1plA71LKRvX4BDaAKFlTgLeALtliDUqPrSV6SQCBlypgFlbmIIrCDcAl6nPAawmYhlLKFuB6IrkXAadUNj6TXlhDcCNEB/Jn4FcE0f4UWEl0NyWNvZxGTs89z6ZnatIIrCdqcCtRJmcCPwCeSN3N1Iu6T4VaFhm9n+riypouBnepLsk9p6p35fzwvDSX5eVQvaDOzjnqzTl+1KC53+XzLINHd65O6lD1DnWbepPBhQ3q2jQyW+2oDkkAtdt5udpb7W+Q/OFGA7ol1zxu1tc8zNHqXercfDfQIOZm9fR815Cpt5PnVqsr1F51wI9QnzU63xZ1o/rdPPmt6enV6sXqHPVqdXOCe1rtrg5W7zNI+m712Ir+cer4POiqfHeJSVe1Raemwnm7xD3mD1E/Z3wIjcsTdlZnqO8bFeNB9c30zgVG2euYa69QJ+9G90lG+99bfdIoo5PU4w362xHePxl1slMab6tV72KUxDvzlAMT8G0ZohXq39VX1bNzzxij9K1Qb9lhdGe931B/kR6/zCwY9YvuytCsMlj+gbr5SemhqkyuzE8xau4MP865JvWNuj0b1YuqDkgvH2GkURfakly01Cg7Cw0+qyXxkjojq9Lw+vT2AUY+DlF/otYq1Ixc35re2V7R8aTRg2KUv7+ou3x/14PsUBn3NG51S0XpG0Z9PcOPKWSS0SKNUo9Rv2Mmt/G5WpPF6pHGra7Jv410OVsdaz217AbkAPX3ubkm240belCuudT4Rp5p/DyC2lf9mfq1iq5eFe8/lu+K0YrVp0uret4nAkwlB6vzjI/1PxrlrTp/oNHbzTJI92T1qAT+BfW49MhMg6JUp7ehY5a6Tl2jjmVvitF9fxo5Yq8CaAfAkzLMnySt6uz/1k6bPx59CpCNxGfoSKA30IPoH7cQXdArwCOllFX/i53P5P9a/gNkKpsCMFRuFAAAAABJRU5ErkJggg==)](https://operations-engineering-reports.cloud-platform.service.justice.gov.uk/public-report/template-repository)
+This repo has been developed by the DevOps Lan&Wi-Fi to automate site creation on juniper mist.
 
-This template repository equips you with the default initial files required for a Ministry of Justice GitHub repository.
+## Run script
 
-## Included Files
+## Development setup
 
-The repository comes with the following preset files:
+### Prerequisites
 
-- LICENSE
-- .gitignore
-- CODEOWNERS
-- dependabot.yml
-- GitHub Actions example file
-- Ministry of Justice Compliance Badge (Public repositories only)
+- Docker
+- IDE that integrates with docker (We use IntelliJ in this example)
 
-## Setup Instructions
+### Setup
 
-Once you've created your repository using this template, ensure the following steps:
+- Run `make build`
+- Integrate built docker container with IDE. [here](https://www.jetbrains.com/help/idea/configuring-remote-python-sdks.html#2546d02c) is the example for intelliJ
+- mark src directory & test directory within the IDE. [here](https://www.jetbrains.com/help/idea/content-roots.html)
 
-### Update README
+# Notes
 
-Edit this README.md file to document your project accurately. Take the time to create a clear, engaging, and informative README.md file. Include information like what your project does, how to install and run it, how to contribute, and any other pertinent details.
-
-### Update repository description
-
-After you've created your repository, GitHub provides a brief description field that appears on the top of your repository's main page. This is a summary that gives visitors quick insight into the project. Using this field to provide a succinct overview of your repository is highly recommended.
-
-This description and your README.md will be one of the first things people see when they visit your repository. It's a good place to make a strong, concise first impression. Remember, this is often visible in search results on GitHub and search engines, so it's also an opportunity to help people discover your project.
-
-### Grant Team Permissions
-
-Assign permissions to the appropriate Ministry of Justice teams. Ensure at least one team is granted Admin permissions. Whenever possible, assign permissions to teams rather than individual users.
-
-### Read about the GitHub repository standards
-
-Familiarise yourself with the Ministry of Justice GitHub Repository Standards. These standards ensure consistency, maintainability, and best practices across all our repositories.
-
-You can find the standards [here](https://operations-engineering.service.justice.gov.uk/documentation/services/repository-standards.html).
-
-Please read and understand these standards thoroughly and enable them when you feel comfortable.
-
-### Modify the GitHub Standards Badge
-
-Once you've ensured that all the [GitHub Repository Standards](https://operations-engineering.service.justice.gov.uk/documentation/services/repository-standards.html) have been applied to your repository, it's time to update the Ministry of Justice (MoJ) Compliance Badge located in the README file.
-
-The badge demonstrates that your repository is compliant with MoJ's standards. Please follow these [instructions](https://operations-engineering.service.justice.gov.uk/documentation/runbooks/services/add-repo-badge.html) to modify the badge URL to reflect the status of your repository correctly.
-
-**Please note** the badge will not function correctly if your repository is internal or private. In this case, you may remove the badge from your README.
-
-### Manage Outside Collaborators
-
-To add an Outside Collaborator to the repository, follow the guidelines detailed [here](https://github.com/ministryofjustice/github-collaborators).
-
-### Update CODEOWNERS
-
-(Optional) Modify the CODEOWNERS file to specify the teams or users authorized to approve pull requests.
-
-### Configure Dependabot
-
-Adapt the dependabot.yml file to match your project's [dependency manager](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem) and to enable [automated pull requests for package updates](https://docs.github.com/en/code-security/supply-chain-security).
-
-If your repository is private with no GitHub Advanced Security license, remove the .github/workflows/dependency-review.yml file.
+- To see options run `make help`
+- If you update [requirements.txt](src/requirements.txt), you will need to rebuild the docker container.

docker/Dockerfile

@@ -0,0 +1,24 @@
+FROM python:3.11.5
+
+# Set the working directory to /app
+WORKDIR /app
+
+# Copy the current directory contents into the container at /app
+COPY ../src/requirements.txt /tmp/requirements.txt
+
+# Copy entrypoint
+COPY docker/entrypoint.sh /entrypoint.sh
+COPY src/ /app/src
+
+# Make entrypoint executable
+RUN chmod +x /entrypoint.sh
+
+# Install any needed packages specified in requirements.txt
+RUN pip install --no-cache-dir -r ../tmp/requirements.txt
+
+# Set the src directory
+ENV PYTHONPATH=/app/src
+
+# Run tests when the container launches
+ENTRYPOINT ["/entrypoint.sh"]
+

docker/entrypoint.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+if [ "$RUN_UNIT_TESTS" = True ] ; then
+  python -m unittest discover -s /app -t /app
+else
+  python /app/src/main.py
+fi

makefile

@@ -0,0 +1,44 @@
+#!make
+.DEFAULT_GOAL := help
+SHELL := '/bin/bash'
+
+.PHONY: build
+build: ## Build the docker container
+	docker build -t juniper-mist -f docker/Dockerfile .
+
+.PHONO: create-dir
+make create-dir: ## Creates a directory for end user to put CSV file into
+	mkdir data_src;
+	echo "Please put csv file into data_src then run 'make-prod'";
+
+.PHONY: run-prod
+run-prod: ## Run the python script only mounting the host for csv-file. Format: MIST_API_TOKEN=foo ORG_ID=bar make run-prod
+	docker run -v $(shell pwd)/data_src:/data_src \
+				-e MIST_API_TOKEN=$$MIST_API_TOKEN \
+				-e ORG_ID=$$ORG_ID \
+				juniper-mist
+
+.PHONY: run-dev
+run-dev: ## Run the python script while mounting the host. This enables using the latest local src code without needing to wait for a container build. Format: MIST_API_TOKEN=foo ORG_ID=bar make run-dev
+	docker run -v $(shell pwd)/src:/app/src \
+				-v $(shell pwd)/data_src:/data_src \
+				-e MIST_API_TOKEN=$$MIST_API_TOKEN \
+				-e ORG_ID=$$ORG_ID \
+				juniper-mist
+
+.PHONY: tests
+tests: ## Run unit tests for the python app
+	docker run -v $(shell pwd)/src:/app/src \
+		-v $(shell pwd)/test:/app/test \
+		-e RUN_UNIT_TESTS=True juniper-mist
+
+.PHONY: shell
+shell: ## Make interactive docker container
+	docker run -it --entrypoint /bin/bash \
+		-v $(shell pwd)/src:/app/src \
+		-v $(shell pwd)/test:/app/test \
+		-v $(shell pwd)/data_src:/data_src \
+		-e RUN_UNIT_TESTS=True juniper-mist
+
+help:
+	@grep -h -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

src/geocode.py

@@ -7,9 +7,15 @@ def geocode(address) -> str:
     location = geolocator.geocode(address)
     try:
         latitude, longitude = location.latitude, location.longitude
-    except ValueError:
-        raise ValueError(
-            'geocode unable to find latitude & longitude for {address}'.format(address=address))
+    except AttributeError as e:
+        if "'NoneType' object has no attribute 'latitude'" in str(e):
+            raise AttributeError(
+                'geocode unable to find latitude & longitude for {address}'.format(address=address))
+        if "'NoneType' object has no attribute 'longitude'" in str(e):
+            raise AttributeError(
+                'geocode unable to find latitude & longitude for {address}'.format(address=address))
+        else:
+            raise  # Re-raise the original AttributeError if the message doesn't match
     return latitude, longitude
 
 
@@ -30,7 +36,7 @@ def find_timezone(gps) -> tuple:
     try:
         timezone_name = tf_init.timezone_at(lat=latitude, lng=longitude)
     except ValueError:
-        raise ValueError('The coordinates were out of bounds {latitude}:{longitude}'.format(
+        raise ValueError('The coordinates were out of bounds {lat}:{lng}'.format(
             lat=latitude, lng=longitude))
     if timezone_name is None:
         raise ValueError('GPS coordinates did not match a time_zone')

src/main.py

@@ -5,29 +5,22 @@
 
 
 # Convert CSV file to JSON object.
-def csv_to_json(file_path):
+def convert_csv_to_json(file_path):
     csv_rows = []
     with open(file_path) as csvfile:
         reader = csv.DictReader(csvfile, skipinitialspace=True, quotechar='"')
         title = reader.fieldnames
 
         for row in reader:
             csv_rows.extend([{title[i]: row[title[i]]
-                            for i in range(len(title))}])
+                              for i in range(len(title))}])
 
+    if csv_rows == None or csv_rows == []:
+        raise ValueError('Failed to convert CSV file to JSON. Exiting script.')
     return csv_rows
 
 
-if __name__ == '__main__':
-
-    csv_file_path = os.getcwd() + '/../test_data/sites_with_clients.csv'
-
-    # Convert CSV to valid JSON
-    data = csv_to_json(csv_file_path)
-    if data == None or data == []:
-        raise ValueError('Failed to convert CSV file to JSON. Exiting script.')
-
-    # Create each site from the CSV file
+def add_geocoding_to_json(data):
     for d in data:
         # Variables
         site_id = None
@@ -43,9 +36,21 @@ def csv_to_json(file_path):
         d['gps'] = gps
         d['country_code'] = country_code
         d['time_zone'] = time_zone
+    return data
+
+
+if __name__ == '__main__':
+
+    csv_file_path = os.getcwd() + '/../data_src/sites_with_clients.csv'
+
+    # Convert CSV to valid JSON
+    json_data_without_geocoding = convert_csv_to_json(csv_file_path)
+
+    json_data_with_geocoding = add_geocoding_to_json(
+        json_data_without_geocoding)
 
     juniper_script(
         mist_api_token=os.environ['MIST_API_TOKEN'],
         org_id=os.environ['ORG_ID'],
-        data=data
+        data=json_data_with_geocoding
     )

src/requirements.txt

@@ -1,3 +1,3 @@
-requests==2.21.0
+requests==2.31.0
 geopy==2.4.1
 timezonefinder==6.2.0

test/test_geocode.py

@@ -0,0 +1,112 @@
+import unittest
+from unittest.mock import patch, MagicMock
+from src.geocode import geocode, find_timezone, find_country_code
+
+
+class TestGeocode(unittest.TestCase):
+
+    @patch('src.geocode.Nominatim.geocode')
+    def test_given_list_of_valid_addresses_when_geocode_called_then_return_relevant_list_of_gps_locations(self, mock_nominatim):
+        # Define a list of addresses and their expected results
+        addresses = [
+            ("40 Mayflower Dr, Plymouth PL2 3DG", (50.3868633, -4.1539256)),
+            ("102 Petty France, London SW1H 9AJ",
+             (51.499929300000005, -0.13477761285315926)),
+            ("London", (51.4893335, -0.14405508452768728)),
+            ("Met Office, FitzRoy Road, Exeter, Devon, EX1 3PB",
+             (50.727350349999995, -3.4744726127760086))
+        ]
+        # Mock the geocode method to return the corresponding latitude and longitude
+        for address, (lat, lon) in addresses:
+            mock_nominatim.return_value.latitude = lat
+            mock_nominatim.return_value.longitude = lon
+
+            # Call the geocode function for each address
+            result = geocode(address)
+
+            # Assert that the result matches the expected latitude and longitude
+            self.assertEqual(result, (lat, lon))
+
+    @patch('src.geocode.Nominatim.geocode')
+    def test_geocode_invalid_address(self, mock_geocode):
+        # Arrange
+        address = "Invalid Address"
+        mock_geocode.return_value = None  # Simulate geocode returning None
+
+        # Act & Assert
+        with self.assertRaises(AttributeError) as context:
+            geocode(address)
+
+        expected_error_message = 'geocode unable to find latitude & longitude for {address}'.format(
+            address=address)
+        self.assertEqual(str(context.exception), expected_error_message)
+
+    @patch('src.geocode.TimezoneFinder')
+    def test_find_timezone_valid_coordinates(self, mock_timezone_finder):
+        tf_instance = MagicMock()
+        tf_instance.timezone_at.return_value = 'America/New_York'
+        mock_timezone_finder.return_value = tf_instance
+
+        gps_coordinates = (40.7128, -74.0060)
+        result = find_timezone(gps_coordinates)
+
+        self.assertEqual(result, 'America/New_York')
+
+    @patch('src.geocode.TimezoneFinder')
+    def test_find_timezone_out_of_bounds(self, mock_timezone_finder):
+        tf_instance = MagicMock()
+        tf_instance.timezone_at.side_effect = ValueError(
+            'The coordinates were out of bounds 40.7128:-74.0060')
+        mock_timezone_finder.return_value = tf_instance
+
+        gps_coordinates = (40.7128, -74.0060)
+
+        with self.assertRaises(ValueError) as context:
+            find_timezone(gps_coordinates)
+
+        self.assertEqual(str(context.exception),
+                         'The coordinates were out of bounds 40.7128:-74.006')
+
+    @patch('src.geocode.TimezoneFinder')
+    def test_find_timezone_no_matching_timezone(self, mock_timezone_finder):
+        tf_instance = MagicMock()
+        tf_instance.timezone_at.return_value = None
+        mock_timezone_finder.return_value = tf_instance
+
+        gps_coordinates = (40.7128, -74.0060)
+
+        with self.assertRaises(ValueError) as context:
+            find_timezone(gps_coordinates)
+
+        self.assertEqual(str(context.exception),
+                         'GPS coordinates did not match a time_zone')
+
+
+class TestFindCountryCode(unittest.TestCase):
+    @patch('src.geocode.Nominatim.geocode')
+    def test_find_country_code_valid_coordinates(self, mock_nominatim):
+        geolocator_instance = MagicMock()
+        location_instance = MagicMock()
+        location_instance.raw = {'address': {'country_code': 'us'}}
+        geolocator_instance.reverse.return_value = location_instance
+        mock_nominatim.return_value = geolocator_instance
+
+        gps_coordinates = (40.7128, -74.0060)
+        result = find_country_code(gps_coordinates)
+
+        self.assertEqual(result, 'US')
+
+    @patch('src.geocode.Nominatim.geocode')
+    def test_find_country_code_invalid_coordinates(self, mock_nominatim):
+        geolocator_instance = MagicMock()
+        geolocator_instance.reverse.side_effect = Exception(
+            'Invalid coordinates')
+        mock_nominatim.return_value = geolocator_instance
+
+        gps_coordinates = (1000.0, 2000.0)  # Invalid coordinates
+
+        with self.assertRaises(Exception) as context:
+            find_country_code(gps_coordinates)
+
+        self.assertEqual(str(context.exception),
+                         'Must be a coordinate pair or Point')