Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Simplify Barrage Viewport Table Updates #6347

Merged
merged 38 commits into from
Nov 18, 2024
Merged
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
1852b0a
Restore initial commit from grpc-java, plus a few local changes
niloc132 Oct 21, 2024
68b925a
Guard writing payload as hex if FINEST is enabled
niloc132 Oct 21, 2024
e88c47e
Apply upstream "Fix AsyncServletOutputStreamWriterConcurrencyTest
niloc132 Oct 21, 2024
f9a19fc
Apply upstream "Avoid flushing headers when the server returns a single
niloc132 Oct 21, 2024
4733524
Apply upstream "servlet: introduce ServletServerBuilder.buildServlet()"
niloc132 Oct 21, 2024
06e63ec
Bump grpc vers, add inprocess dep for tests
niloc132 Oct 21, 2024
09ade64
Merge branch 'main' into grpc-history-replay
niloc132 Oct 28, 2024
c8af47c
Apply https://github.com/deephaven/deephaven-core/pull/6301
niloc132 Oct 28, 2024
57c8008
Bump to 1.65.1 to better match arrow 18
niloc132 Nov 1, 2024
cbf8ab2
Merge remote-tracking branch 'colin/grpc-history-replay' into vp_simp…
nbauernfeind Nov 6, 2024
85f604f
Version Upgrades; MavenLocal
nbauernfeind Nov 6, 2024
70a0207
Implement Simplified Viewport Table Updates in BMP/BT
nbauernfeind Nov 8, 2024
0089d62
Ryan's Synchronous Review
nbauernfeind Nov 9, 2024
485746d
Merge remote-tracking branch 'upstream/main' into vp_simplification
nbauernfeind Nov 9, 2024
ad8de73
Remove SNAPSHOT version and mavenLocal references
nbauernfeind Nov 11, 2024
02ce2ad
Fixes removed/added rows in most VP cases
nbauernfeind Nov 12, 2024
da23e2b
Bug fixes around viewport snapshot rowsRemoved and rowsAdded
nbauernfeind Nov 12, 2024
299f56e
Bugfix for correct growing VP logic
nbauernfeind Nov 12, 2024
9d6f389
remaining java side fixes
nbauernfeind Nov 13, 2024
fd5aced
Ryan's feedback on javaserver/client impls
nbauernfeind Nov 14, 2024
53b1eed
Inline Feedback from VC w/Ryan
nbauernfeind Nov 14, 2024
6e7fe94
Do not propagate modifies for any repainted rows
nbauernfeind Nov 14, 2024
d568eb7
Minor cleanup from personal review
nbauernfeind Nov 14, 2024
6653ca6
Ryan's feedback latest round.
nbauernfeind Nov 14, 2024
44cdf93
jsAPI mostly complete; looking for tree table issue
nbauernfeind Nov 15, 2024
d549d79
Fixes for jsapi and HierarchicalTable
nbauernfeind Nov 15, 2024
b4d5b69
Lazily compute rowset encoding
nbauernfeind Nov 15, 2024
6c12314
Fixup jsapi tests
nbauernfeind Nov 15, 2024
f9be6e5
Quick round feedback
nbauernfeind Nov 15, 2024
4252622
spotless
nbauernfeind Nov 15, 2024
2767def
Double checked locking fixes
nbauernfeind Nov 15, 2024
78c4cb7
Ryan's final review
nbauernfeind Nov 15, 2024
ea6f898
Clarify strategy on who owns RowSets passed into getSubView
nbauernfeind Nov 15, 2024
3eeb628
npe fix
nbauernfeind Nov 15, 2024
84a6100
Bugfix if HT is empty or viewport past end of table
nbauernfeind Nov 16, 2024
476ae65
Colin's feedback
nbauernfeind Nov 16, 2024
738cb11
Limit jsapi data change event to prev and curr table sizes
nbauernfeind Nov 16, 2024
44fadff
Colin's Final Feedback
nbauernfeind Nov 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Ryan's Synchronous Review
nbauernfeind committed Nov 9, 2024
commit 0089d629fa6278b0cb423c83b6f3bc9a362487c1
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
repositories {
mavenCentral()
mavenLocal()
maven {
url 'https://oss.sonatype.org/content/repositories/snapshots'
content {
includeGroup 'com.vertispan.flatbuffers'
}
}
maven {
url 'https://jitpack.io'
content {
Original file line number Diff line number Diff line change
@@ -34,29 +34,18 @@ public static BarrageSnapshotOptions of(final BarrageSnapshotRequest snapshotReq
return of(snapshotRequest.snapshotOptions());
}

/**
* By default, prefer to communicate null values using the arrow-compatible validity structure.
*
* @return whether to use deephaven nulls
*/
@Override
@Default
public boolean useDeephavenNulls() {
return false;
}

/**
* @return the preferred batch size if specified
*/
@Override
@Default
public int batchSize() {
return 0;
}

/**
* @return the maximum GRPC message size if specified
*/
@Override
@Default
public int maxMessageSize() {
@@ -70,32 +59,63 @@ public int previewListLengthLimit() {
}

public int appendTo(FlatBufferBuilder builder) {
return io.deephaven.barrage.flatbuf.BarrageSnapshotOptions.createBarrageSnapshotOptions(
builder, useDeephavenNulls(),
return io.deephaven.barrage.flatbuf.BarrageSnapshotOptions.createBarrageSnapshotOptions(builder,
useDeephavenNulls(),
batchSize(),
maxMessageSize(),
previewListLengthLimit());
}

public interface Builder {

/**
* See {@link StreamReaderOptions#useDeephavenNulls()} for details.
*
* @param useDeephavenNulls whether to use deephaven nulls
* @return this builder
*/
Builder useDeephavenNulls(boolean useDeephavenNulls);

/**
* Deprecated since 0.37.0 and is marked for removal. (our GWT artifacts do not yet support the attributes)
* The default conversion mode is to Stringify all objects that do not have a registered encoding. Column
* conversion modes are no longer supported.
*
* @deprecated Since 0.37.0 and is marked for removal. (Note, GWT does not support encoding this context via
* annotation values.)
*/
@FinalDefault
@Deprecated
default Builder columnConversionMode(ColumnConversionMode columnConversionMode) {
return this;
}

/**
* See {@link StreamReaderOptions#batchSize()} for details.
*
* @param batchSize the ideal number of records to send per record batch
* @return this builder
*/
Builder batchSize(int batchSize);

/**
* See {@link StreamReaderOptions#maxMessageSize()} for details.
*
* @param messageSize the maximum size of a GRPC message in bytes
* @return this builder
*/
Builder maxMessageSize(int messageSize);

/**
* See {@link StreamReaderOptions#previewListLengthLimit()} for details.
*
* @param previewListLengthLimit the magnitude of the number of elements to include in a preview list
* @return this builder
*/
Builder previewListLengthLimit(int previewListLengthLimit);

/**
* @return a new BarrageSnapshotOptions instance
*/
BarrageSnapshotOptions build();
}
}
Original file line number Diff line number Diff line change
@@ -72,7 +72,7 @@ BarrageStreamGenerator newGenerator(
* @param isFullSubscription whether this is a full subscription (possibly a growing viewport)
* @param viewport is the position-space viewport
* @param reverseViewport is the viewport reversed (relative to end of table instead of beginning)
* @param keyspaceViewportPrev is the key-space viewport in prior to applying the update
* @param keyspaceViewportPrev is the key-space viewport prior to applying the update
* @param keyspaceViewport is the key-space viewport
* @param subscribedColumns are the columns subscribed for this view
* @return a MessageView filtered by the subscription properties that can be sent to that subscriber
Loading