Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Number returned in scientific notation instead of standard decimal format #460

Open
adnan-awan opened this issue Jan 3, 2025 · 3 comments

Comments

@adnan-awan
Copy link
Contributor

When performing a division operation, the result is returned in scientific notation, even though it could be represented in standard decimal format for better readability. For example, the result of 1 / 11091 returns 9.01631953836444e-05, while the expected result is something like 0.0000901 in decimal format.

Steps to Reproduce:

Perform the division operation: 1 / 11091.
Observe the result is in scientific notation.
Expected Behavior: The result should be displayed in a readable decimal format (e.g., 0.0000901).

Actual Behavior: The result is returned in scientific notation: 9.01631953836444e-05.

Additional Information: It would be helpful if there was an option to toggle between scientific notation and standard decimal format for small numbers.

https://github.com/scrapinghub/spidermon/blob/master/spidermon/utils/field_coverage.py#L41

@VMRuiz
Copy link
Collaborator

VMRuiz commented Jan 7, 2025

Hello @adnan-awan, what you are mentioning is a behavior of Python itself.
However, are you looking for a way to customize the display of such stats in Scrapy logs?
If you have any specific examples or use cases where this would be particularly useful

@adnan-awan
Copy link
Contributor Author

Hi @VMRuiz ,

======================================================================
FAIL: Field Coverage Monitor/test_check_if_field_coverage_rules_are_met


Traceback (most recent call last):
File "/app/python/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/monitors.py", line 476, in > test_check_if_field_coverage_rules_are_met
self.assertTrue(len(failures) == 0, msg=msg)
AssertionError:
The following items did not meet field coverage rules:
dict/additionalProperties/name/certification level (expected 0.0001, got 9.01631953836444e-05)

@VMRuiz
Copy link
Collaborator

VMRuiz commented Jan 7, 2025

Hey @adnan-awan, I see where you're coming from.

It seems like what you're really looking for is a tolerance level – a margin of error within which the results are considered acceptable. So instead of strictly requiring 0.0001, maybe anything >= 0.000090 would pass.
This issue actually came up before. It seems like there's a desire for more flexibility with these thresholds.

Perhaps we could add a configuration option to specify this 'tolerance' value for field coverage tests? That would give users more control over how strict the checks are

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants