Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add query management functionality #1

Merged
merged 69 commits into from
Feb 11, 2025
Merged
Changes from 1 commit
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
ce1a783
Add `.gitignore`
Paulchen5 Jan 28, 2025
027a3d6
Add requirements and setup file
Paulchen5 Jan 28, 2025
f44f3c0
Add query manager functionality
Paulchen5 Jan 28, 2025
4c71a79
Add test cases
Paulchen5 Jan 28, 2025
60bff09
Add `utilities/helper.py` as dummy object of `requests.Response`
Paulchen5 Jan 28, 2025
dd95146
Add filter to reduce the stored data amount
Paulchen5 Jan 28, 2025
ad0a9a2
Update `utilities/query_management.py`
Paulchen5 Jan 28, 2025
cfe8a70
Add DataCleaner to reduce data amount a second time
Paulchen5 Jan 28, 2025
badebfa
Update utilities/__init__.py
Paulchen5 Jan 28, 2025
8b1f68b
Update thread handling in `utilities/semaphore.py`
Paulchen5 Jan 28, 2025
a09bbd3
Update `main.py`
Paulchen5 Jan 28, 2025
8a7dc00
Update Error reporting message in `utilities/data_filter.py`
Paulchen5 Jan 28, 2025
57ef645
Update test cases | Add `tests/Test_Filter.py` for `utilities/data_fi…
Paulchen5 Jan 28, 2025
1ae5e2d
Add default option for semaphore threads
Paulchen5 Jan 28, 2025
d80026c
Update `utilities/semaphore.py` | Remove unused comment
Paulchen5 Jan 28, 2025
5c0da75
Update index reference
Paulchen5 Jan 28, 2025
4f832b9
Use list comprehension
Paulchen5 Jan 28, 2025
767a2e2
Update readability
Paulchen5 Jan 28, 2025
370e971
Update readability
Paulchen5 Jan 28, 2025
e9b6b55
Update readability
Paulchen5 Jan 28, 2025
7db3b4a
Update comment
Paulchen5 Jan 28, 2025
a3e6973
Update utilities/query_management.py
Paulchen5 Jan 28, 2025
f7f2c05
Update readability
Paulchen5 Jan 28, 2025
7c09f97
Update utilities/query_management.py
Paulchen5 Jan 28, 2025
6d6d510
Update requirements.txt
Paulchen5 Jan 28, 2025
b247dff
Update main.py
Paulchen5 Jan 28, 2025
dbbe7a1
Update main.py
Paulchen5 Jan 28, 2025
c8fa828
Update utilities/calc.py
Paulchen5 Jan 28, 2025
b950f36
Update readability
Paulchen5 Jan 28, 2025
ecf2c87
Update utilities/query_management.py
Paulchen5 Jan 28, 2025
25324d1
Update main.py
Paulchen5 Jan 28, 2025
950b354
Update main.py
Paulchen5 Jan 28, 2025
36abc72
Update readability
Paulchen5 Jan 28, 2025
fcaf213
Update utilities/data_filter.py
Paulchen5 Jan 28, 2025
8fbbc08
Update readability
Paulchen5 Jan 28, 2025
86ac419
Update tests/Test_Query.py
Paulchen5 Jan 28, 2025
113679c
Update readability
Paulchen5 Jan 28, 2025
4c15db7
Use for directory management
Paulchen5 Jan 28, 2025
4470db2
Remove executing code
Paulchen5 Jan 29, 2025
577035d
Remove unused code
Paulchen5 Jan 29, 2025
86ace21
Update list indexing
Paulchen5 Jan 30, 2025
c942daf
Add test case for filter function
Paulchen5 Jan 31, 2025
5d24622
Update staging process
Paulchen5 Jan 31, 2025
3205867
Add reset functionality
Paulchen5 Jan 31, 2025
b9910da
Add Copilot generated docs
Paulchen5 Jan 31, 2025
6e40d61
Update main.py
Paulchen5 Feb 4, 2025
3226b8a
Update main.py
Paulchen5 Feb 4, 2025
e2a8caf
Update main.py
Paulchen5 Feb 4, 2025
5fbacc5
Update main.py
Paulchen5 Feb 4, 2025
e58ae4f
Update main.py
Paulchen5 Feb 4, 2025
6eeda29
Update exception handling in `utilities/query_management.py`
Paulchen5 Feb 10, 2025
57bf5c2
Update TODO comment in `main.py`
Paulchen5 Feb 10, 2025
a6aa158
unit added
viennaa Feb 10, 2025
1fe75db
Update past calculation handling
Paulchen5 Feb 10, 2025
c860902
Update tmp folder hanndling | Remove tmp folder after filtering
Paulchen5 Feb 10, 2025
022d6eb
Fix https://github.com/sapcc/alertmagnet/pull/1#discussion_r1946542208
Paulchen5 Feb 10, 2025
24a84d8
Fix https://github.com/sapcc/alertmagnet/pull/1#discussion_r1946549219
Paulchen5 Feb 10, 2025
090d7de
Add required instruction https://github.com/sapcc/alertmagnet/pull/1#…
Paulchen5 Feb 10, 2025
b7500df
Update parameter naming due to https://github.com/sapcc/alertmagnet/p…
Paulchen5 Feb 10, 2025
7238c07
Update semaphore thread handling due to https://github.com/sapcc/aler…
Paulchen5 Feb 10, 2025
d416c31
Update installation process
Paulchen5 Feb 10, 2025
a737466
Update `README.md`
Paulchen5 Feb 10, 2025
fc8680a
Update main.py
Paulchen5 Feb 10, 2025
58090c4
Update main.py
Paulchen5 Feb 10, 2025
856544b
Fix https://github.com/sapcc/alertmagnet/pull/1#discussion_r194956724…
Paulchen5 Feb 11, 2025
717135b
Update cleanup process
Paulchen5 Feb 11, 2025
ef528cf
Update utilities/data_cleaner.py
Paulchen5 Feb 11, 2025
1eea421
Fix https://github.com/sapcc/alertmagnet/pull/1#discussion_r1950617489
Paulchen5 Feb 11, 2025
cbfbba7
Update max storage handling
Paulchen5 Feb 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update readability
Due to: #1 (comment)
Paulchen5 committed Jan 28, 2025
commit e9b6b554ce01293e2473ea745e44925cd7907fee
11 changes: 6 additions & 5 deletions utilities/data_filter.py
Original file line number Diff line number Diff line change
@@ -63,11 +63,12 @@ def create_time_ranges(
value = data[-1]
if start is None:
out.append((value, 0))
return out

if not value == prev + step: # consider using >
out.append((start, prev - start))
out.append((value, 0))
else:
if not value == prev + step: # consider using >
out.append((start, prev - start))
out.append((value, 0))
else:
out.append((start, value - start))
out.append((start, value - start))

return out