Merge pull request #14 from us-irs/mueller/0.13.0rc2

v0.13.0rc2

CHANGELOG.md

@@ -8,6 +8,20 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [unreleased]
 
+## [v0.13.0rc2] 12.07.2022
+
+- Improved documentation, first docstrings
+- Added more re-exports, for example for the `ccsds` module
+- Added several dunder method implementations, especially `__repr__`, `__str__` and `__eq__`
+- Improved CFDP packet stack API, several improvements derived from the implementation
+  of a CFDP handler using it
+- Added generic abstraction for CFDP File Data and File Directive PDUs in form of the
+  `AbstractPduBase` and `AbstractFileDirectiveBase`
+- Generic `UnsignedByteField` implementation. This is a data structure which is regularly
+  used for something like variable sized identifier fields. It provides a lot of boilerplate
+  code like common dunder implementations
+- Split up and improve test structure a bit
+
 ## [v0.13.0rc1] 01.07.2022
 
 - Update `pyproject.toml` file for full support, but still keep `setup.cfg` for now

docs/api.rst

@@ -4,7 +4,7 @@ API Documentation
 This package is split into three subpackages:
 
 - :py:mod:`spacepackets.ccsds`: Contains CCSDS specific code. This includes the space packet
-  implementation inside the :py:mod:`spacepacket.ccsds.spacepackets` module and time related
+  implementation inside the :py:mod:`spacepackets.ccsds.spacepacket` module and time related
   implementations in the :py:mod:`spacepackets.ccsds.time` module
 
 - :py:mod:`spacepackets.cfdp`: Contains packet implementations related to the CCSDS File Delivery Protocol

docs/api/cfdp.rst

@@ -17,10 +17,10 @@ spacepackets.cfdp.conf module
    :undoc-members:
    :show-inheritance:
 
-spacepackets.cfdp.defintions module
+spacepackets.cfdp.defs module
 -------------------------------------
 
-.. automodule:: spacepackets.cfdp.definitions
+.. automodule:: spacepackets.cfdp.defs
    :members:
    :undoc-members:
    :show-inheritance:

docs/api/cfdp_pdu.rst

@@ -23,12 +23,23 @@ Following File Data PDUs are available in the subpackage
 - File Data: :py:mod:`spacepackets.cfdp.pdu.file_data`
 
 Every PDU type has a common PDU header which can be found inside the
-:py:mod:`spacepackets.cfdp.pdu.header` module
+:py:mod:`spacepackets.cfdp.pdu.header` module.
+
+The helper module :py:mod:`spacepackets.cfdp.pdu.helper` contains components like the
+:py:class:`spacepackets.cfdp.pdu.helper.PduWrapper` class which stores PDUs as a generic base type
+and allows typed conversion in to the concrete PDU type
+
+spacepackets.cfdp.pdu.helper module
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. automodule:: spacepackets.cfdp.pdu.helper
+   :members:
+   :undoc-members:
+   :show-inheritance:
 
 File Data Submodules
 ---------------------------
 
-
 spacepackets.cfdp.pdu.file_data module
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

docs/packets.rst

@@ -21,8 +21,9 @@ function from the `crcmod package`_  can be used to calculate this checksum.
 Telecommands
 ^^^^^^^^^^^^^^^^^^
 
-Extended information can be found in `ECSS-E-70-41A`_  on p.42 or in `ECSS-ST-E-70-41C`_ starting at
-page 442.
+This chapter contains some high level information about the
+:py:class:`spacepackets.ecss.tc.PusTelecommand` class. Extended information can be found
+in `ECSS-E-70-41A`_  on p.42 or in `ECSS-ST-E-70-41C`_ starting at page 442.
 
 The structure is shown as follows for a ping telecommand using the PUS service 17 with the
 subervice ID 1. This can also be denoted more briefly as TC[17,1]. The first part
@@ -80,8 +81,9 @@ PUS A is not supported anymore starting at version ``v0.12.0``.
 Telemetry
 ^^^^^^^^^^^^
 
-Extended information can be found in `ECSS-E-70-41A`_  on p.42 or in `ECSS-ST-E-70-41C`_ starting at
-page 442.
+This chapter contains some high level information about the
+:py:class:`spacepackets.ecss.tm.PusTelemetry` class. Extended information can be found
+in `ECSS-E-70-41A`_  on p.42 or in `ECSS-ST-E-70-41C`_ starting at page 442.
 
 The structure is shown as follows for a ping reply using the PUS service 17 with the
 subervice ID 2. This can also be denoted more briefly as TM[17,2]. The first part

spacepackets/__init__.py

@@ -1 +1,9 @@
-__version__ = "0.13.0rc1"
+from spacepackets.ccsds import (
+    SpacePacketHeader,
+    SpacePacket,
+    PacketTypes,
+    SequenceFlags,
+)
+
+
+__version__ = "0.13.0rc2"

spacepackets/ccsds/__init__.py

@@ -1,3 +1,10 @@
 """This package contains all CCSDS related components"""
-from .spacepacket import SpacePacketHeader, SpacePacket
+from .spacepacket import (
+    SpacePacketHeader,
+    SpacePacket,
+    PacketTypes,
+    SequenceFlags,
+    PacketId,
+    PacketSeqCtrl,
+)
 from .time import CdsShortTimestamp

spacepackets/ccsds/spacepacket.py

@@ -120,7 +120,21 @@ def __init__(
         seq_flags: SequenceFlags = SequenceFlags.UNSEGMENTED,
         ccsds_version: int = 0b000,
     ):
-        """Create a space packet header with the given field parameters
+        """Create a space packet header with the given field parameters.
+
+        >>> sph = SpacePacketHeader(packet_type=PacketTypes.TC, apid=0x42, seq_count=0, data_len=12)
+        >>> hex(sph.apid)
+        '0x42'
+        >>> sph.packet_type
+        <PacketTypes.TC: 1>
+        >>> sph.data_len
+        12
+        >>> sph.packet_len
+        19
+        >>> sph.packet_id
+        PacketId(ptype=<PacketTypes.TC: 1>, sec_header_flag=True, apid=66)
+        >>> sph.psc
+        PacketSeqCtrl(seq_flags=<SequenceFlags.UNSEGMENTED: 3>, seq_count=0)
 
         :param packet_type: 0 for Telemetery, 1 for Telecommands
         :param apid: Application Process ID, should not be larger
@@ -215,16 +229,18 @@ def apid(self, apid):
 
     @property
     def packet_len(self) -> int:
-        """Retrieve the full space packet size when packed
+        """Retrieve the full space packet size when packed.
+
         :return: Size of the TM packet based on the space packet header data length field.
-        The space packet data field is the full length of data field minus one without
-        the space packet header.
+            The space packet data field is the full length of data field minus one without
+            the space packet header.
         """
         return SPACE_PACKET_HEADER_SIZE + self.data_len + 1
 
     @classmethod
     def unpack(cls, space_packet_raw: bytes) -> SpacePacketHeader:
-        """Unpack a raw space packet into the space packet header instance
+        """Unpack a raw space packet into the space packet header instance.
+
         :raise ValueError: Raw packet length invalid
         """
         if len(space_packet_raw) < SPACE_PACKET_HEADER_SIZE:

spacepackets/cfdp/__init__.py

@@ -1,12 +1,15 @@
 from .defs import (
     PduType,
     ChecksumTypes,
+    Direction,
     CrcFlag,
-    FileSize,
+    LargeFileFlag,
     SegmentationControl,
     SegmentMetadataFlag,
     TransmissionModes,
     ConditionCode,
+    FaultHandlerCodes,
+    NULL_CHECKSUM_U32,
 )
 from .tlv import (
     CfdpTlv,
@@ -21,3 +24,5 @@
     FilestoreResponseStatusCode,
 )
 from .lv import CfdpLv
+from .conf import PduConfig
+from .pdu import DirectiveType

spacepackets/cfdp/conf.py

@@ -4,12 +4,13 @@
 
 from spacepackets.cfdp.defs import (
     TransmissionModes,
-    FileSize,
+    LargeFileFlag,
     CrcFlag,
     Direction,
     SegmentationControl,
 )
 from spacepackets.log import get_console_logger
+from spacepackets.util import UnsignedByteField, ByteFieldU8, ByteFieldEmpty
 
 
 @dataclass
@@ -21,58 +22,54 @@ class PduConfig:
     specifying parameter which rarely change repeatedly
     """
 
-    transaction_seq_num: bytes
+    source_entity_id: UnsignedByteField
+    dest_entity_id: UnsignedByteField
+    transaction_seq_num: UnsignedByteField
     trans_mode: TransmissionModes
-    file_size: FileSize = FileSize.GLOBAL_CONFIG
-    crc_flag: CrcFlag = CrcFlag.GLOBAL_CONFIG
+    file_flag: LargeFileFlag = LargeFileFlag.NORMAL
+    crc_flag: CrcFlag = CrcFlag.NO_CRC
     direction: Direction = Direction.TOWARDS_RECEIVER
     seg_ctrl: SegmentationControl = (
         SegmentationControl.NO_RECORD_BOUNDARIES_PRESERVATION
     )
-    source_entity_id: bytes = bytes()
-    dest_entity_id: bytes = bytes()
 
     @classmethod
     def empty(cls) -> PduConfig:
+        """Empty PDU configuration which is not valid for usage because the contained unsigned
+        byte fields are empty (sequence number and both entity IDs)
+        """
         return PduConfig(
-            transaction_seq_num=bytes([0]),
+            transaction_seq_num=ByteFieldEmpty(),
             trans_mode=TransmissionModes.ACKNOWLEDGED,
-            source_entity_id=bytes([0]),
-            dest_entity_id=bytes([0]),
-            file_size=FileSize.GLOBAL_CONFIG,
-            crc_flag=CrcFlag.GLOBAL_CONFIG,
+            source_entity_id=ByteFieldEmpty(),
+            dest_entity_id=ByteFieldEmpty(),
+            file_flag=LargeFileFlag.NORMAL,
+            crc_flag=CrcFlag.NO_CRC,
         )
 
-    def __post_init__(self):
-        """Ensure that the global configuration is converted to the actual value immediately"""
-        if self.crc_flag == CrcFlag.GLOBAL_CONFIG:
-            self.crc_flag = get_default_pdu_crc_mode()
-        if self.file_size == FileSize.GLOBAL_CONFIG:
-            self.file_size = get_default_file_size()
+    @classmethod
+    def default(cls):
+        """Valid PDU configuration"""
+        return PduConfig(
+            transaction_seq_num=ByteFieldU8(0),
+            trans_mode=TransmissionModes.ACKNOWLEDGED,
+            source_entity_id=ByteFieldU8(0),
+            dest_entity_id=ByteFieldU8(0),
+            file_flag=LargeFileFlag.NORMAL,
+            crc_flag=CrcFlag.NO_CRC,
+        )
 
 
 class CfdpDict(TypedDict):
     source_dest_entity_ids: Tuple[bytes, bytes]
-    with_crc: CrcFlag
-    file_size: FileSize
 
 
 # TODO: Protect dict access with a dedicated lock for thread-safety
 __CFDP_DICT: CfdpDict = {
     "source_dest_entity_ids": (bytes(), bytes()),
-    "with_crc": CrcFlag.NO_CRC,
-    "file_size": FileSize.NORMAL,
 }
 
 
-def set_default_pdu_crc_mode(with_crc: CrcFlag):
-    __CFDP_DICT["with_crc"] = with_crc
-
-
-def get_default_pdu_crc_mode() -> CrcFlag:
-    return __CFDP_DICT["with_crc"]
-
-
 def set_entity_ids(source_entity_id: bytes, dest_entity_id: bytes):
     __CFDP_DICT["source_dest_entity_ids"] = (source_entity_id, dest_entity_id)
 
@@ -82,14 +79,6 @@ def get_entity_ids() -> Tuple[bytes, bytes]:
     return __CFDP_DICT["source_dest_entity_ids"]
 
 
-def set_default_file_size(file_size: FileSize):
-    __CFDP_DICT["file_size"] = file_size
-
-
-def get_default_file_size() -> FileSize:
-    return __CFDP_DICT["file_size"]
-
-
 def check_packet_length(
     raw_packet_len: int, min_len: int, warn_on_fail: bool = True
 ) -> bool:

spacepackets/cfdp/defs.py

@@ -1,5 +1,5 @@
+from __future__ import annotations
 import enum
-import struct
 
 
 class PduType(enum.IntEnum):
@@ -8,6 +8,8 @@ class PduType(enum.IntEnum):
 
 
 class Direction(enum.IntEnum):
+    """This is used for PDU forwarding"""
+
     TOWARDS_RECEIVER = 0
     TOWARDS_SENDER = 1
 
@@ -20,7 +22,6 @@ class TransmissionModes(enum.IntEnum):
 class CrcFlag(enum.IntEnum):
     NO_CRC = 0
     WITH_CRC = 1
-    GLOBAL_CONFIG = 2
 
 
 class SegmentMetadataFlag(enum.IntEnum):
@@ -45,12 +46,11 @@ class FaultHandlerCodes(enum.IntEnum):
 
 
 class LenInBytes(enum.IntEnum):
+    ZERO_OR_NONE = 0
     ONE_BYTE = 1
     TWO_BYTES = 2
     FOUR_BYTES = 4
     EIGHT_BYTES = 8
-    GLOBAL = 90
-    NONE = 99
 
 
 class ConditionCode(enum.IntEnum):
@@ -70,33 +70,12 @@ class ConditionCode(enum.IntEnum):
     CANCEL_REQUEST_RECEIVED = 0b1111
 
 
-def get_transaction_seq_num_as_bytes(
-    transaction_seq_num: int, byte_length: LenInBytes
-) -> bytearray:
-    """Return the byte representation of the transaction sequece number
-    :param transaction_seq_num:
-    :param byte_length:
-    :raises ValueError: Invalid input
-    :return:
-    """
-    if byte_length == LenInBytes.ONE_BYTE and transaction_seq_num < 255:
-        return bytearray([transaction_seq_num])
-    if byte_length == LenInBytes.TWO_BYTES and transaction_seq_num < pow(2, 16) - 1:
-        return bytearray(struct.pack("!H", transaction_seq_num))
-    if byte_length == LenInBytes.FOUR_BYTES and transaction_seq_num < pow(2, 32) - 1:
-        return bytearray(struct.pack("!I", transaction_seq_num))
-    if byte_length == LenInBytes.EIGHT_BYTES and transaction_seq_num < pow(2, 64) - 1:
-        return bytearray(struct.pack("!Q", transaction_seq_num))
-    raise ValueError
-
-
 # File sizes, determine the field sizes of FSS fields
-class FileSize(enum.IntEnum):
+class LargeFileFlag(enum.IntEnum):
     # 32 bit maximum file size and FSS size
     NORMAL = 0
     # 64 bit maximum file size and FSS size
     LARGE = 1
-    GLOBAL_CONFIG = 2
 
 
 # Checksum types according to the SANA Checksum Types registry
@@ -106,5 +85,9 @@ class ChecksumTypes(enum.IntEnum):
     MODULAR = 0
     CRC_32_PROXIMITY_1 = 1
     CRC_32C = 2
+    # Polynomial: 0x4C11DB7. This is the preferred checksum for now.
     CRC_32 = 3
     NULL_CHECKSUM = 15
+
+
+NULL_CHECKSUM_U32 = bytes([0x00, 0x00, 0x00, 0x00])

spacepackets/cfdp/lv.py

@@ -43,3 +43,9 @@ def unpack(cls, raw_bytes: bytes) -> CfdpLv:
         if detected_len == 0:
             return cls(value=bytes())
         return cls(value=raw_bytes[1 : 1 + detected_len])
+
+    def __repr__(self):
+        return f"{self.__class__.__name__}(value={self.value!r})"
+
+    def __str__(self):
+        return f"CFDP LV with data 0x[{self.value.hex(sep=',')}] of length {len(self.value)}"