Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: hanging stream identification #189 #190

Closed
wants to merge 3 commits into from
Closed

Conversation

carlwilson
Copy link
Member

  • added simple test, with no assert, for file identification; and
  • added similar for stream identification which demonstrates hang.

- added simple test, with no assert, for file identification; and
- added similar for stream identification which demonstrates hang.
@carlwilson carlwilson added bug A product defect that needs fixing P1 High priority issues to be scheduled in the upcoming release labels May 14, 2020
@carlwilson carlwilson requested review from replaceafill and sevein May 14, 2020 23:37
@carlwilson carlwilson self-assigned this May 14, 2020
@carlwilson carlwilson linked an issue May 14, 2020 that may be closed by this pull request
@carlwilson carlwilson added this to the v1.6 milestone May 14, 2020
@carlwilson carlwilson marked this pull request as draft May 15, 2020 00:01
- fixed termination condition in `blocking_read`.
@carlwilson
Copy link
Member Author

New commit added that fixes the loop and stops the tests failing.

Comment on lines +21 to +37
def test_file_identification():
"""Reference for Fido-based format identification
1. Create a byte-stream with a known magic number and serialise to tempfile.
2. Call identify_file(...) to identify the file against Fido's known formats.
"""
# Create a temporary file on the host operating system.
tmp = tempfile.mkstemp()
tmp_file = tmp[1]

# Write to the file our known magic-number.
with open(tmp_file, "wb") as new_file:
new_file.write(MAGIC)

# Create a Fido instance and call identify_file. The identify_file function
# will create and manage a file for itself.
f = fido.Fido()
f.identify_file(tmp_file)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice tests, Carl. I think this one could be a good use case for the tmp_path fixture provided by pytest. It gives us a pathlib.Path object which has a nice API. pytest also takes care of removing old temporary directories, always keeping the last three.

Suggested change
def test_file_identification():
"""Reference for Fido-based format identification
1. Create a byte-stream with a known magic number and serialise to tempfile.
2. Call identify_file(...) to identify the file against Fido's known formats.
"""
# Create a temporary file on the host operating system.
tmp = tempfile.mkstemp()
tmp_file = tmp[1]
# Write to the file our known magic-number.
with open(tmp_file, "wb") as new_file:
new_file.write(MAGIC)
# Create a Fido instance and call identify_file. The identify_file function
# will create and manage a file for itself.
f = fido.Fido()
f.identify_file(tmp_file)
def test_file_identification(tmp_path):
"""Reference for Fido-based format identification
1. Create a byte-stream with a known magic number and serialise to tempfile.
2. Call identify_file(...) to identify the file against Fido's known formats.
"""
# Create a temporary file on the host operating system.
tmp_file = tmp_path / "file"
# Write to the file our known magic-number.
tmp_file.write_bytes(MAGIC)
# Create a Fido instance and call identify_file. The identify_file function
# will create and manage a file for itself.
f = fido.Fido()
f.identify_file(str(tmp_file))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the original ideas here came from @ross-spencer (that's intended to give credit not share blame for temp file management). I agree that these could be better for test purposes. The PR will remain draft for now as I'm going to test a couple of ideas around the API.

@ross-spencer
Copy link

@carlwilson I hadn't seen @sevein's inline suggestion when I put this on my todo list. I've only just registered this coming back to it tonight but I had already created a branch and PR for you to take a look at here, which does the following too:

  • Demos tmp_path like @sevein above: 18de5c0
  • Brings in Pytest's capsys to allow us to consume the CSV and make assertions about it: 3b9e73f

They might be good enough for you to merge, but maybe also develop out into some negative FIDO result testing too, i.e. negative identifications, if you have time/interest which it seems like you do (interest anyway, if not heaps of time! 😉 )

There was a build warning I noticed late on and corrected as well in that series of commits.

Anyhow, I hope this work is a useful platform and not an annoyance at all. I just happened to have time at the end of my day today to take a look.

@carlwilson
Copy link
Member Author

@adamfarquhar the work in this and #191 above might be useful for developing unit tests.

@carlwilson carlwilson changed the base branch from rc/1.6 to main September 2, 2024 15:24
@carlwilson
Copy link
Member Author

Closed as contained in #226

@carlwilson carlwilson closed this Nov 14, 2024
Copy link

codecov bot commented Nov 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 34.00%. Comparing base (cd85271) to head (96c3007).
Report is 39 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #190       +/-   ##
===========================================
+ Coverage   23.42%   34.00%   +10.58%     
===========================================
  Files          11       11               
  Lines        1490     1491        +1     
===========================================
+ Hits          349      507      +158     
+ Misses       1141      984      -157     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A product defect that needs fixing P1 High priority issues to be scheduled in the upcoming release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fido hanging on skeleton stream (fmt/1000)
3 participants