Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue_156: Performance Log Metrics and more #157

Merged
merged 1 commit into from
Dec 3, 2019
Merged

Conversation

galenatjpl
Copy link
Contributor

@galenatjpl galenatjpl commented Nov 27, 2019

This change adds performance timing logging of array content
and label validation. The logging will be enabled at verbosity
level 1 (INFO) or below (e.g. DEBUG).

Also, the hooks for experimentation of multi-threading has
been setup at the LabelInFolderRule level. In other words a
thread pool exists, that can run targets within a folder
concurrently. However, due to thread safety issues with the
current code and libraries, the thread level is currently set
to one (no multi-threading). In the future, this can be
experimented with. A few synchronized locks have been placed
throughout the code, where multi-threading proved problematic
during experimentation. More may be needed in the future.
When the thread level is set to greater than one, the
performance does improve, but there are failures sometimes,
due to thread saftety issues. If this can be solved, real
performance gains can be realized.

Some basic GNU parallel scripts have been added in the
src/test directory, as well, for future experimentation into
running validate concurrently at the JVM level.

Also, some library versions have been upgraded in the pom.xml.

Resolves #156

This change adds performance timing logging of array content
and label validation.  The logging will be enabled at verbosity
level 1 (INFO) or below (e.g. DEBUG).

Also, the hooks for experimentation of multi-threading has
been setup at the LabelInFolderRule level.  In other words a
thread pool exists, that can run targets within a folder
concurrently.  However, due to thread safety issues with the
current code and libraries, the thread level is currently set
to one (no multi-threading).  In the future, this can be
experimented with.  A few synchronized locks have been placed
throughout the code, where multi-threading proved problematic
during experimentation.  More may be needed in the future.
When the thread level is set to greater than one, the
performance does improve, but there are failures sometimes,
due to thread saftety issues.  If this can be solved, real
performance gains can be realized.

Some basic GNU parallel scripts have been added in the
src/test directory, as well, for future experimentation into
running validate concurrently at the JVM level.

Also, some library versions have been upgraded in the pom.xml.
@galenatjpl
Copy link
Contributor Author

Some interesting performance observations (not conclusive).

TIMINGS:

NEW CODE WIFI LOCATION #1
1m37.385s
1m50.389s

NEW CODE WIFI LOCATION #2 (faster internet than #1)
1m35.443s
1m34.912s
1m34.004s

OLD CODE WIFI LOCATION #1
2m14.229s
2m40.070s

OLD CODE WIFI LOCATION #2 (faster internet than #1)
1m37.496s
1m32.752s

CONCLUSION:
The newer code seems to be faster when the internet speeds are slower. This may be because certain parts of the code remove reliance on unnecessary URL resolution....

@jordanpadams jordanpadams merged commit f4a28fa into master Dec 3, 2019
@jordanpadams jordanpadams deleted the issue_156 branch December 3, 2019 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Introduce label and array timing metrics logging
3 participants