Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON Streaming #412

Merged
merged 25 commits into from
Feb 15, 2024
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
d8b7e9f
wip: ingest streaming
rvazarkar Feb 12, 2024
c0b4bb7
wip: ingest streaming
rvazarkar Feb 12, 2024
c9448a6
chore: code golf
zinic Feb 12, 2024
4759996
wip: ingest streaming
rvazarkar Feb 12, 2024
e96836b
wip: wip commit
rvazarkar Feb 12, 2024
a37832f
Merge remote-tracking branch 'origin/main' into BED-4114
rvazarkar Feb 12, 2024
6886b98
feat: json file streaming implementation
rvazarkar Feb 12, 2024
46fcd82
chore: static analysis fixes
rvazarkar Feb 13, 2024
057c1f9
fix: incorrect comparison breaking tests
rvazarkar Feb 13, 2024
dcf6386
fix: don't error out immediately on invalid meta tag
rvazarkar Feb 13, 2024
2edf4ba
chore: add missing copyright
rvazarkar Feb 13, 2024
1e6dcad
chore: add missing copyright
rvazarkar Feb 13, 2024
4056358
example: added example unit testing to SeekToDataTag
superlinkx Feb 13, 2024
211ff0b
chore: add some more tests, and validate data tag during meta tag checks
rvazarkar Feb 14, 2024
13a3a86
chore: export variables for use in bhe
rvazarkar Feb 14, 2024
6e3273c
Merge remote-tracking branch 'origin/main' into BED-4114
rvazarkar Feb 14, 2024
4177dd3
fix: initialize azure struct
rvazarkar Feb 14, 2024
8f81ed1
chore: remove unnecessary change
rvazarkar Feb 14, 2024
8d8232b
chore: use var blocks, optimize reallocations
rvazarkar Feb 14, 2024
fb0a559
chore: add comment
rvazarkar Feb 14, 2024
d720c3c
chore: add comment
rvazarkar Feb 14, 2024
10231a2
chore: more cleanup
rvazarkar Feb 14, 2024
998f3c6
chore: fix static analysis nit
rvazarkar Feb 14, 2024
4f64ee6
Merge branch 'main' into BED-4114
rvazarkar Feb 14, 2024
86bf46c
Merge remote-tracking branch 'origin/main' into BED-4114
rvazarkar Feb 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: remove unnecessary change
rvazarkar committed Feb 14, 2024
commit 8f81ed10986c999909f40e8281ea308c2e87ce6c
1 change: 0 additions & 1 deletion cmd/api/src/daemons/datapipe/decoders.go
Original file line number Diff line number Diff line change
@@ -126,7 +126,6 @@ func decodeAzureData(batch graph.Batch, reader io.ReadSeeker) error {
}

convertedData := ConvertedAzureData{}
rvazarkar marked this conversation as resolved.
Show resolved Hide resolved
convertedData.Clear()
var data AzureBase
count := 0
for decoder.More() {