Add BurstCube TOO API endpoints

Add Validation to AstropyTime Add FOV check to BurstCubeTOO and other fixes Change healpix_order to healpix_scheme Compare ephem times in unix time as comparing freeform astropy time object can cause this to fail due to minute differences that create an out of bounds issue with the assertion in get_slice Fixes for modified_on/by Remove history table stuff for now Fix security for burstcube TOO Updates for mypy and cleanups Update import sorts Import sort Remove debug print statement Simplify database write/update Simplify database write/update Fix check for existing TOO Change BurstCubeTOO PUT to have the same arguments as POST, so we can use it to upload updated HEALPix Remove excess docstring from schema Reconfigure BurstCubeTOO to redo constraints check if PUT updates coordinates Change how uploads are done in the API Update TOO PUT schema Trap uploads of bad HEALPix files Fix bug Make schema and api return astropy native values Simplify requests.py Move to astropy native in schema. Clean up docstrings Clean up code reuse in BurstCubeTOO Json email selections (nasa-gcn#1977) Bump usehooks-ts from 2.15.0 to 2.15.1 Bumps [usehooks-ts](https://github.com/juliencrn/usehooks-ts) from 2.15.0 to 2.15.1. - [Release notes](https://github.com/juliencrn/usehooks-ts/releases) - [Commits](https://github.com/juliencrn/usehooks-ts/compare/[email protected]@2.15.1) --- updated-dependencies: - dependency-name: usehooks-ts dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Bump tiny-invariant from 1.3.1 to 1.3.3 Bumps [tiny-invariant](https://github.com/alexreardon/tiny-invariant) from 1.3.1 to 1.3.3. - [Release notes](https://github.com/alexreardon/tiny-invariant/releases) - [Commits](alexreardon/tiny-invariant@v1.3.1...v1.3.3) --- updated-dependencies: - dependency-name: tiny-invariant dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Support for reading gzipped fits files. Refactor reused code. Remove superfluous entries in app.arc Make BurstCubeTOO a dataclass to remove the init stuff Differentiate username and created by Back to username Remove a space in a comment. Compare jd instead of unix as it should always be consistent Put table into a property that can be set for pytest Don't hardcode TOO table name Clean up docstring Move table write into BurstCubeTOO Move BurstCubeTOOModel into schema.py Fix imports Remove types in docstrings Adds a confirmation alert (nasa-gcn#1957) * Adds a confirmation alert * Return intent and rename newItem to newCircular Better sentence text, adds heading Bump es5-ext from 0.10.61 to 0.10.63 Bumps [es5-ext](https://github.com/medikoo/es5-ext) from 0.10.61 to 0.10.63. - [Release notes](https://github.com/medikoo/es5-ext/releases) - [Changelog](https://github.com/medikoo/es5-ext/blob/main/CHANGELOG.md) - [Commits](medikoo/es5-ext@v0.10.61...v0.10.63) --- updated-dependencies: - dependency-name: es5-ext dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Announcement template (nasa-gcn#1958) Code of Conduct (nasa-gcn#1635) Docstring cleanups Fix use of old style Config in Pydantic model Add pytest to check CRUD operations on BurstCubeTOO Add check for SPACETRACK credentials Fix reject_reason Add old TOO rejection pytest Add test for double submission Add test for double submission Mock BurstCubeTLE so that we don't have to spam celestak or spacetrack Add pytest-mock requirement for testing Revert tle.py Mypy fixes Docstring fix Fix test time to a value that matches the test TLE Remove epoch for TLE Revert to Time.now() otherwise toos will get rejected a few days from now Move assert for time being within ephem to ephindex Try something to kick Github workflow Fix return type Co-authored-by: Leo Singer <[email protected]> Remove a comment Co-authored-by: Leo Singer <[email protected]> Add black in to check Change length to duration Add sub-types to u.Quantity return types Fix return quantity Remove black from workflow Propogate the length -> duration change Move depends into depends.py. Remove unused depend from BurstCube api.py. Put BurstCubeTOORequests into toorequest.py Switch to pytest-dynamodb for pytests Stuff to run pytest-dynamodb in the workflow Remove JsonStr as it's not needed. Update how DyDB puts work. Fix bug in definition of error_radius. Remove unncessary model_validator Fix pytests. Add tests for BurstCubeTOORequests. Remove debug code Put back BurstCubeTrigger model_validator Fix format Remove unnecessary changes Add BurstCube TOO API endpoints Add Validation to AstropyTime Add FOV check to BurstCubeTOO and other fixes Change healpix_order to healpix_scheme Compare ephem times in unix time as comparing freeform astropy time object can cause this to fail due to minute differences that create an out of bounds issue with the assertion in get_slice Fixes for modified_on/by Remove history table stuff for now Fix security for burstcube TOO Updates for mypy and cleanups Update import sorts Import sort Remove debug print statement Simplify database write/update Simplify database write/update Fix check for existing TOO Change BurstCubeTOO PUT to have the same arguments as POST, so we can use it to upload updated HEALPix Remove excess docstring from schema Reconfigure BurstCubeTOO to redo constraints check if PUT updates coordinates Change how uploads are done in the API Update TOO PUT schema Trap uploads of bad HEALPix files Fix bug Make schema and api return astropy native values Simplify requests.py Move to astropy native in schema. Clean up docstrings Clean up code reuse in BurstCubeTOO Json email selections (nasa-gcn#1977) Bump usehooks-ts from 2.15.0 to 2.15.1 Bumps [usehooks-ts](https://github.com/juliencrn/usehooks-ts) from 2.15.0 to 2.15.1. - [Release notes](https://github.com/juliencrn/usehooks-ts/releases) - [Commits](https://github.com/juliencrn/usehooks-ts/compare/[email protected]@2.15.1) --- updated-dependencies: - dependency-name: usehooks-ts dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Bump tiny-invariant from 1.3.1 to 1.3.3 Bumps [tiny-invariant](https://github.com/alexreardon/tiny-invariant) from 1.3.1 to 1.3.3. - [Release notes](https://github.com/alexreardon/tiny-invariant/releases) - [Commits](alexreardon/tiny-invariant@v1.3.1...v1.3.3) --- updated-dependencies: - dependency-name: tiny-invariant dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Support for reading gzipped fits files. Refactor reused code. Remove superfluous entries in app.arc Make BurstCubeTOO a dataclass to remove the init stuff Differentiate username and created by Back to username Remove a space in a comment. Compare jd instead of unix as it should always be consistent Put table into a property that can be set for pytest Don't hardcode TOO table name Clean up docstring Move table write into BurstCubeTOO Move BurstCubeTOOModel into schema.py Fix imports Remove types in docstrings Adds a confirmation alert (nasa-gcn#1957) * Adds a confirmation alert * Return intent and rename newItem to newCircular Better sentence text, adds heading Bump es5-ext from 0.10.61 to 0.10.63 Bumps [es5-ext](https://github.com/medikoo/es5-ext) from 0.10.61 to 0.10.63. - [Release notes](https://github.com/medikoo/es5-ext/releases) - [Changelog](https://github.com/medikoo/es5-ext/blob/main/CHANGELOG.md) - [Commits](medikoo/es5-ext@v0.10.61...v0.10.63) --- updated-dependencies: - dependency-name: es5-ext dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Announcement template (nasa-gcn#1958) Code of Conduct (nasa-gcn#1635) Docstring cleanups Fix use of old style Config in Pydantic model Add pytest to check CRUD operations on BurstCubeTOO Add check for SPACETRACK credentials Fix reject_reason Add old TOO rejection pytest Add test for double submission Add test for double submission Mock BurstCubeTLE so that we don't have to spam celestak or spacetrack Add pytest-mock requirement for testing Revert tle.py Mypy fixes Docstring fix Fix test time to a value that matches the test TLE Remove epoch for TLE Revert to Time.now() otherwise toos will get rejected a few days from now Move assert for time being within ephem to ephindex Try something to kick Github workflow Fix return type Co-authored-by: Leo Singer <[email protected]> Remove a comment Co-authored-by: Leo Singer <[email protected]> Add black in to check Change length to duration Add sub-types to u.Quantity return types Fix return quantity Remove black from workflow Propogate the length -> duration change Move depends into depends.py. Remove unused depend from BurstCube api.py. Put BurstCubeTOORequests into toorequest.py Switch to pytest-dynamodb for pytests Stuff to run pytest-dynamodb in the workflow Remove JsonStr as it's not needed. Update how DyDB puts work. Fix bug in definition of error_radius. Remove unncessary model_validator Put back BurstCubeTrigger model_validator Fix format Remove unnecessary changes
jak574 · Mar 15, 2024 · 1666800 · 1666800
1 parent d49b1f6
commit 1666800
Show file tree

Hide file tree

Showing 4 changed files with 298 additions and 2 deletions.
diff --git a/.github/actions/install-dynamodb/action.yml b/.github/actions/install-dynamodb/action.yml
@@ -0,0 +1,14 @@
+name: Install DynamoDB
+description: >
+  Install a local version of DynamoDB.
+  See https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html
+outputs:
+  dynamodb-dir:
+    description: Path to the directory containing the JAR file
+    value: ${{ steps.step.outputs.dynamodb-dir }}
+runs:
+  using: composite
+  steps:
+    - id: step
+      run: curl https://d1ni2b6xgvw0s0.cloudfront.net/v2.x/dynamodb_local_latest.tar.gz | tar -C ${{ github.action_path }} -xz && echo dynamodb-dir=${{ github.action_path }} >> $GITHUB_OUTPUT
+      shell: bash
diff --git a/.github/workflows/pull_request.yml b/.github/workflows/pull_request.yml
@@ -32,8 +32,15 @@ jobs:
       - name: Pip Install
         run: pip install -r requirements.txt
 
-      - name: Pytest
-        run: pytest python
+      - name: Install DynamoDB
+        id: install-dynamodb
+        uses: ./.github/actions/install-dynamodb
+
+      - name: Install Package
+        run: pip install pytest-cov -e .[test]
+
+      - name: Run Tests
+        run: pytest --dynamodb-dir=${{ steps.install-dynamodb.outputs.dynamodb-dir }} --cov --cov-report=xml
 
       - name: MyPy
         run: mypy python

diff --git a/python/tests/burstcubetoo/conftest.py b/python/tests/burstcubetoo/conftest.py
@@ -0,0 +1,135 @@
+from unittest.mock import Mock
+
+import astropy.units as u  # type: ignore[import]
+import pytest
+from across_api.base.schema import TLEEntry  # type: ignore[import]
+from across_api.burstcube.schema import BurstCubeTOOSchema  # type: ignore[import]
+from across_api.burstcube.toorequest import (  # type: ignore[import]
+    BurstCubeTOO,
+    BurstCubeTriggerInfo,
+)
+from astropy.time import Time  # type: ignore
+from astropy.time.core import TimeDelta  # type: ignore[import]
+
+
+@pytest.fixture
+def burstcube_tle():
+    return TLEEntry(
+        tle1="1 25544U 98067A   24059.70586912  .00019555  00000-0  35623-3 0  9995",
+        tle2="2 25544  51.6410 137.9505 0005676 302.8794 193.7648 15.49465684441577",
+        satname="ISS (ZARYA)",
+    )
+
+
+@pytest.fixture
+def mock_read_tle_db(mocker, burstcube_tle):
+    """Mock TLEEntry find_tles_between_epochs to return burstcube_tle fixture,
+    so we don't spam space-track.org or CelesTrak."""
+    mock = Mock()
+    mocker.patch.object(
+        TLEEntry, "find_tles_between_epochs", return_value=[burstcube_tle]
+    )
+    return mock
+
+
+@pytest.fixture
+def mock_toorequest_table(mocker, dynamodb):
+    """Replace the call to arc.tables.table with the pytest-dynamodb
+    dynamodb.Table call."""
+    mock = mocker.patch(
+        "across_api.burstcube.toorequest.dynamodb_table",
+        wraps=lambda x: dynamodb.Table(x),
+    )
+    return mock
+
+
+@pytest.fixture(scope="function")
+def create_too_table(dynamodb):
+    """Create the TOO Table"""
+    dynamodb.create_table(
+        AttributeDefinitions=[
+            {"AttributeName": "id", "AttributeType": "S"},
+        ],
+        TableName=BurstCubeTOOSchema.__tablename__,
+        KeySchema=[
+            {"AttributeName": "id", "KeyType": "HASH"},
+        ],
+        BillingMode="PAY_PER_REQUEST",
+    )
+
+
+@pytest.fixture(scope="function")
+def create_tle_table(dynamodb):
+    """Create the TLE Table so we can use BurstCubeEphem in tests"""
+    dynamodb.create_table(
+        TableName="acrossapi_tle",
+        AttributeDefinitions=[
+            {"AttributeName": "satname", "AttributeType": "S"},
+            {"AttributeName": "epoch", "AttributeType": "S"},
+        ],
+        KeySchema=[
+            {"AttributeName": "satname", "KeyType": "HASH"},
+            {"AttributeName": "epoch", "KeyType": "RANGE"},
+        ],
+        BillingMode="PAY_PER_REQUEST",
+    )
+
+
+@pytest.fixture
+def now():
+    yield Time.now()
+
+
+@pytest.fixture
+def username():
+    yield "testuser"
+
+
+@pytest.fixture
+def trigger_info():
+    yield BurstCubeTriggerInfo(
+        trigger_mission="BurstCube",
+        trigger_type="GRB",
+    )
+
+
+@pytest.fixture
+def burstcube_too(username, now, trigger_info):
+    too = BurstCubeTOO(
+        trigger_time=now,
+        trigger_info=trigger_info,
+        username=username,
+    )
+    yield too
+
+
+@pytest.fixture
+def burstcube_old_too(username, trigger_info):
+    too = BurstCubeTOO(
+        trigger_time=Time.now() - TimeDelta(48 * u.hr),
+        trigger_info=trigger_info,
+        username=username,
+    )
+    yield too
+
+
+@pytest.fixture
+def burstcube_three_toos(
+    create_too_table, create_tle_table, dynamodb, now, trigger_info
+):
+    """Create 3 BurstCubeTOO objects with different statuses."""
+    now = Time.now()
+    trigger_info = BurstCubeTriggerInfo(
+        trigger_mission="BurstCube",
+        trigger_type="GRB",
+    )
+    toos = []
+    for i in range(3):
+        too = BurstCubeTOO(
+            trigger_time=now - TimeDelta(i * u.hr),
+            trigger_info=trigger_info,
+            username="testuser",
+        )
+        too.post()
+        toos.append(too)
+    yield toos
diff --git a/python/tests/burstcubetoo/test_burstcube_too.py b/python/tests/burstcubetoo/test_burstcube_too.py
@@ -0,0 +1,140 @@
+import astropy.units as u  # type: ignore[import]
+from across_api.burstcube.schema import TOOReason  # type: ignore[import]
+from across_api.burstcube.toorequest import BurstCubeTOO  # type: ignore[import]
+from across_api.burstcube.toorequest import BurstCubeTOORequests  # type: ignore[import]
+from astropy.time.core import Time, TimeDelta  # type: ignore[import]
+
+
+# @mock_aws
+def test_burstcube_too_crud(
+    username,
+    burstcube_too,
+    create_tle_table,
+    create_too_table,
+    mock_read_tle_db,
+    mock_toorequest_table,
+):
+    from across_api.burstcube.toorequest import dynamodb_table
+
+    assert dynamodb_table == mock_toorequest_table
+    assert burstcube_too.post() is True
+    assert burstcube_too.id is not None
+
+    # Test fetching posted TOO
+    too = BurstCubeTOO(id=burstcube_too.id)
+    too.get()
+    assert too.id == burstcube_too.id
+
+    assert too.trigger_time == burstcube_too.trigger_time
+    assert too.trigger_info == burstcube_too.trigger_info
+    assert too.created_by == burstcube_too.username
+    assert too.status == "Requested" or too.status == "Rejected"
+    assert too.trigger_info.trigger_mission == "BurstCube"
+    assert too.trigger_info.trigger_type == "GRB"
+    # If the TOO was rejected, it should only be rejected due to SAA as no
+    # coordinates were given, and trigger time is less than 48 hours old
+    assert (
+        too.reject_reason == TOOReason.none
+        and too.status == "Requested"
+        or too.reject_reason == TOOReason.saa
+        and too.status == "Rejected"
+    )
+
+    # Test deleting posted TOO
+    too = BurstCubeTOO(id=burstcube_too.id, username=username)
+    too.delete()
+
+    too = BurstCubeTOO(id=burstcube_too.id)
+    too.get()
+    assert too.status == "Deleted"
+
+    # Test changing status to Approved
+    too.status = "Approved"
+    too.put()
+
+    too = BurstCubeTOO(id=burstcube_too.id)
+    too.get()
+    assert too.status == "Approved"
+
+
+def test_burstcube_old_too(
+    burstcube_old_too,
+    create_tle_table,
+    create_too_table,
+    mock_read_tle_db,
+    mock_toorequest_table,
+):
+    assert burstcube_old_too.post() is True
+    assert burstcube_old_too.trigger_time < Time.now() - TimeDelta(48 * u.hr)
+    assert burstcube_old_too.status == "Rejected"
+    assert "Trigger is too old." in burstcube_old_too.too_info
+    assert burstcube_old_too.reject_reason == TOOReason.too_old
+
+
+def test_burstcube_too_double_post(
+    burstcube_too,
+    create_tle_table,
+    create_too_table,
+    mock_read_tle_db,
+    mock_toorequest_table,
+):
+    assert burstcube_too.post() is True
+    try:
+        burstcube_too.post()
+    except Exception as e:
+        assert e.status_code == 409
+        assert e.detail == "BurstCubeTOO already exists."
+
+
+def test_burstcube_toorequests(
+    mock_toorequest_table,
+    create_tle_table,
+    create_too_table,
+    mock_read_tle_db,
+    now,
+    trigger_info,
+):
+    # Submit three TOOs to the API
+    now = Time("2024-03-01 16:40:00")
+    newtoos = []
+    for i in range(3):
+        newtoo = BurstCubeTOO(
+            trigger_time=now - TimeDelta(i * u.hr),
+            trigger_info=trigger_info,
+            username="testuser",
+        )
+        newtoo.post()
+        newtoos.append(newtoo)
+
+    # Check that BurstCube TOORequests returns the same number of TOOs as were
+    # submitted, and that the contents match
+
+    toos = BurstCubeTOORequests()
+    toos.get()
+    assert len(toos) == len(newtoos)
+    assert toos[0].model_dump_json() == newtoos[0].schema.model_dump_json()
+    assert toos[1].reject_reason == newtoos[1].reject_reason
+    assert toos[1].model_dump_json() == newtoos[1].schema.model_dump_json()
+
+    assert toos[2].model_dump_json() == newtoos[2].schema.model_dump_json()
+
+    # Check the limit parameter
+    toos = BurstCubeTOORequests(limit=1)
+    toos.get()
+
+    assert len(toos) == 1
+
+    # Check the duration parameter by fetching only TOOs that arrived in the
+    # last hour, this should be three. Then check the hour before, should be
+    # zero
+    toos = BurstCubeTOORequests(duration=1 * u.hr)
+    toos.get()
+    assert len(toos) == 3
+
+    # Use begin and end to fetch the TOOs from the hour before the previous one
+    # checked, should be no TOOs.
+    toos = BurstCubeTOORequests(
+        begin=Time.now() - TimeDelta(2 * u.hr), end=Time.now() - TimeDelta(2 * u.hr)
+    )
+    toos.get()
+    assert len(toos) == 0