Enhance Load Generator to Query Pre-Generated Data Blocks #782
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ed blocks. Signed-off-by: Kushal Shukla <[email protected]>
kushalShukla-web
force-pushed
the
query-data
branch
from
d3ec226
to
fe9cad2
Compare
kushalShukla-web
changed the title
|
Instant query is still missing; currently, it does not perform instant queries on downloaded blocks. |
kushalShukla-web
force-pushed
the
query-data
branch
3 times, most recently
from
6976d42
to
6100f15
Compare
kakkoyun
changed the title
kakkoyun
reviewed
Nov 13, 2024
tools/load-generator/main.go
Outdated
| defer resp.Body.Close() | ||
|
|
||
| // Print the body content as a string | ||
| fmt.Println(string(body), "Hi from body side!!!") |
There was a problem hiding this comment.
It needs to a bit of clean up.
kushalShukla-web
force-pushed
the
query-data
branch
from
6100f15
to
01a5662
Compare
…d loadgen start and end time! Signed-off-by: Kushal Shukla <[email protected]>
kushalShukla-web
force-pushed
the
query-data
branch
7 times, most recently
from
b224f59
to
a053628
Compare
Signed-off-by: Kushal Shukla <[email protected]>
kushalShukla-web
force-pushed
the
query-data
branch
from
a053628
to
2748652
Compare
Signed-off-by: Kushal Shukla <[email protected]>
2. Replaced images to prominfra docker registery. Signed-off-by: Kushal Shukla <[email protected]>
bboreham
reviewed
Nov 28, 2024
tools/load-generator/main.go
Outdated
| return &bucketConfig, nil | ||
| } | ||
|
|
||
| func configstate(v *BucketConfig, err error) *configState { |
There was a problem hiding this comment.
It is potentially confusing to have two identifiers which differ only in the case of some letters.
| defer wg.Done() | ||
| fmt.Printf("Running querier %s %s for %s\n", q.target, q.name, q.url) | ||
| time.Sleep(20 * time.Second) | ||
|
|
||
| for { | ||
| start := time.Now() | ||
|
|
||
| runBlockMode := "current" |
There was a problem hiding this comment.
I think the idea here is to alternate between current and absolute?
At least, put a comment saying that.
tools/load-generator/main.go
Outdated
| } | ||
| return &configState{ | ||
| bucketConfig: v, | ||
| Err: err, |
There was a problem hiding this comment.
I think the idea is we carry err around, and act like this config doesn't exist if err is non-nil.
This at least deserves a comment. Maybe an alternative would work, just having nil for this configState object, if there would be an error, and checking that?
tools/load-generator/main.go
Outdated
| func (q *Querier) run(wg *sync.WaitGroup) { | ||
| // Function to load `minTime` and `maxTime` from bucket-config.yml | ||
| func loadKeyConfig() (*BucketConfig, error) { | ||
| filePath := "/config/bucket-config.yml" |
There was a problem hiding this comment.
This should probably be a parameter, a CLI flag.
2. Removed Err and absoluteTime from the configState object. 3. Added flag package for debugging purpose and removed hard coded file path. Signed-off-by: Kushal Shukla <[email protected]>
kushalShukla-web
force-pushed
the
query-data
branch
from
bb03968
to
8b752ef
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
changed Load generator to query from downloaded blocks. Here i have added "current" and "absolute" time to query data between current data (head_blocks) and absolute data (Downloaded) blocks.
Things left - i have to configure to add instant query on downloaded blocks !