Skip to content

Commit

Permalink
feat: mpr #11 dev
Browse files Browse the repository at this point in the history
Run tests via github actions
  • Loading branch information
rilshok committed Jun 22, 2024
2 parents a1be220 + a5f06c2 commit 8df7c26
Show file tree
Hide file tree
Showing 7 changed files with 178 additions and 45 deletions.
45 changes: 7 additions & 38 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ name: Publish to PyPI

on:
push:
branches:
- main
tags:
- "v*.*.*"

jobs:
build:
Expand All @@ -25,21 +25,6 @@ jobs:
name: python-package-distributions
path: dist/

test:
name: Run tests with pytest
runs-on: ubuntu-latest
needs:
- build
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Run tests
run: pytest

publish-to-pypi:
name: Publish Python 🐍 distribution 📦 to PyPI
needs:
Expand Down Expand Up @@ -77,27 +62,11 @@ jobs:
with:
name: python-package-distributions
path: dist/
- name: Sign the dists with Sigstore
uses: sigstore/[email protected]

- name: Release
uses: softprops/action-gh-release@v2
if: startsWith(github.ref, 'refs/tags/')
with:
inputs: >-
files: |
./dist/*.tar.gz
./dist/*.whl
- name: Create GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
run: >-
gh release create
'${{ github.ref_name }}'
--repo '${{ github.repository }}'
--notes ""
- name: Upload artifact signatures to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `dist/` contains the built packages, and the
# sigstore-produced signatures and certificates.
run: >-
gh release upload
'${{ github.ref_name }}' dist/**
--repo '${{ github.repository }}'
40 changes: 40 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Tests

on:
push:
branches: [ "main", "dev" ]
pull_request:
branches: [ "*" ]

permissions:
contents: write
checks: write
pull-requests: write

jobs:
build:
name: Run tests
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
pip install --upgrade pip
pip install -e .[dev]
- name: Run pre-commit hooks
uses: pre-commit/[email protected]
- name: Build coverage file
run: |
pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=iokit tests/ | tee pytest-coverage.txt
- name: Pytest coverage comment
uses: MishaKav/pytest-coverage-comment@main
with:
pytest-coverage-path: ./pytest-coverage.txt
junitxml-path: ./pytest.xml
126 changes: 124 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,124 @@
# iok
input / output kit
# I/O Kit Python Library

IOKit is a Python library that offers a suite of utilities for managing a wide range of input/output operations. Central to its design is the concept of a `State`, where each state signifies a unit of data that can be loaded, saved, or transformed. Each state corresponds to a valid file state, represented as a bytes-like object.

IOKit abstracts and unifies serialization and deserialization operations from various libraries into a single, cohesive interface. This allows for direct manipulation of the file's state in memory, eliminating the need for disk interaction. Consequently, it facilitates the (de)serialization of data in multiple formats, such as `json`, `txt`, `tar`, `gzip`, among others. This abstraction not only simplifies data handling but also enhances efficiency by reducing disk I/O operations.

## Installation

You can install the IOkit library using pip:

```bash
pip install iokit
```

## Usage

Here are some examples of how to use the I/O Kit library:

### Text File Handling

```python
from iokit import Txt

text = "Hello, World!"
state = Txt(text, name="text")
print(state)
print(state.load())
```

```plain-text
text.txt (13B)
Hello, World!
```

### JSON

```python
from iokit import Json

data = {"key": "value"}
state = Json(data, name="single")
print(state)
print(state.load())
```

```plain-text
single.json (16B)
{'key': 'value'}
```

### GZip Compression

```python
from iokit import Txt, Gzip

data = "Hello, World! "* 1000
state = Gzip(Txt(data, name="data"))
print(state)
print(len(state.load().load()))
```

```plain-text
data.txt.gz (133B)
14000
```

### Tar Archive

```python
from iokit import Tar, Txt

state1 = Txt("First file", name="text1")
state2 = Txt("Second file", name="text2")
archive = Tar([state1, state2], name="archive")
states = archive.load()
print(states)
print(states[0].load())
print(states[1].load())
```

```plain-text
[text1.txt (10B), text2.txt (11B)]
First file
Second file
```

### Find State

```python
from iokit import Tar, find_state

state1 = Txt("First file", name="text1")
state2 = Txt("Second file", name="text2")
archive = Tar([state1, state2], name="archive")

state = find_state(archive.load(), "?e*2.txt")
print(state.load())
```

```plain-text
Second file
```

### Byte input handling

```python
from iokit import State

state = State(b"{\"first\": 1, \"second\": 2}", name="data.json")
print(state.load())
```

```plain-text
{'first': 1, 'second': 2}
```


## Contributing

Contributions to the IOkit library are welcome. Please feel free to submit a pull request or open an issue on the GitHub repository.

## License

The IOkit library is licensed under the MIT License. You can use it for commercial and non-commercial projects without any restrictions.
2 changes: 1 addition & 1 deletion src/iokit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"save_file",
"save_temp",
]
__version__ = "0.1.0"
__version__ = "0.1.1"

from .extensions import Gzip, Json, Jsonl, Tar, Txt
from .state import State, filter_states, find_state
Expand Down
7 changes: 5 additions & 2 deletions tests/test_gzip.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
from iokit import Json, Gzip, load_file, save_temp

import os

from iokit import Gzip, Json, load_file, save_temp


def random_utf8_string(length: int) -> str:
random_bytes = os.urandom(length)
string = random_bytes.decode("utf-8", errors="replace")
return string


def test_gzip_state() -> None:
data = {"a": 1, "b": 2}
state = Gzip(Json(data, name="data"))
Expand All @@ -18,6 +19,7 @@ def test_gzip_state() -> None:
assert state.load().load() == data
assert state.size > 0


def test_gzip_compression() -> None:
string = random_utf8_string(10_000)
state = Json(string, name="data")
Expand All @@ -32,6 +34,7 @@ def test_gzip_compression() -> None:
assert compressed3.load().load() == string
assert compressed9.load().load() == string


def test_gzip_save_load_file() -> None:
data = {"a": 1, "b": 2}
state = Gzip(Json(data, name="data"))
Expand Down
1 change: 0 additions & 1 deletion tests/test_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ def test_json_different() -> None:
assert loaded["int"] == 42



def test_json_is_string() -> None:
state = Json("hello", name="string")
assert state.load() == "hello"
Expand Down
2 changes: 1 addition & 1 deletion tests/test_tar.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from iokit import Tar, Txt, find_state, Gzip
from iokit import Gzip, Tar, Txt, find_state


def test_tar_state() -> None:
Expand Down

3 comments on commit 8df7c26

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coverage

Coverage Report
FileStmtsMissCoverMissing
src/iokit
   state.py1142281%31–32, 39, 50, 55–58, 82–84, 90, 93, 104–106, 114, 126–127, 135, 144, 157
src/iokit/extensions
   tar.py28293%27, 30
   txt.py9189%9
src/iokit/storage
   local.py29486%32–33, 35–36
TOTAL2492988% 

Tests Skipped Failures Errors Time
20 0 💤 0 ❌ 0 🔥 0.271s ⏱️

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coverage

Coverage Report
FileStmtsMissCoverMissing
src/iokit
   state.py1142281%31–32, 39, 50, 55–58, 82–84, 90, 93, 104–106, 114, 126–127, 135, 144, 157
src/iokit/extensions
   tar.py28293%27, 30
   txt.py9189%9
src/iokit/storage
   local.py29486%32–33, 35–36
TOTAL2492988% 

Tests Skipped Failures Errors Time
20 0 💤 0 ❌ 0 🔥 0.249s ⏱️

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coverage

Coverage Report
FileStmtsMissCoverMissing
src/iokit
   state.py1142281%31–32, 39, 50, 55–58, 82–84, 90, 93, 104–106, 114, 126–127, 135, 144, 157
src/iokit/extensions
   tar.py28293%27, 30
   txt.py9189%9
src/iokit/storage
   local.py29486%32–33, 35–36
TOTAL2492988% 

Tests Skipped Failures Errors Time
20 0 💤 0 ❌ 0 🔥 0.441s ⏱️

Please sign in to comment.