Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update DecodedResultsTsType to use bigint in Emscripten binding (fixes #33). #34

Merged
merged 1 commit into from
Nov 11, 2024

Conversation

junhaoliao
Copy link
Collaborator

@junhaoliao junhaoliao commented Nov 10, 2024

Description

  1. Update timestamp type declaration for DecodedResultsTsType.

Validation performed

  1. Ran task to generate the compiled assets.
  2. Validated in both build/clp-ffi-js/ClpFfiJs-node.d.ts and build/clp-ffi-js/ClpFfiJs-woker.d.ts that the return result of ClpStreamReader.decodeRange has been updated as Array<[string, bigint, number, number]>

Summary by CodeRabbit

  • New Features

    • Enhanced data handling with updated numerical data representation.
    • Improved error handling for better feedback on processing issues.
  • Bug Fixes

    • Refined error messages for specific IR error codes during processing.
    • Enhanced exception handling for JSON parsing failures.
  • Chores

    • Added logging for buffer length to assist with debugging.

Copy link

coderabbitai bot commented Nov 10, 2024

Walkthrough

The changes in this pull request primarily involve modifications to the StreamReader.cpp file, focusing on data type handling and error management within the ClpStreamReader class. The return type for DecodedResultsTsType has been updated from Array<[string, number, number, number]> to Array<[string, bigint, number, number]>, reflecting a shift to bigint for numerical representation. Error handling has been improved in the rewind_reader_and_validate_encoding_type and get_version functions, with enhanced exception reporting. Additionally, logging has been added to the create method for better debugging.

Changes

File Path Change Summary
src/clp_ffi_js/ir/StreamReader.cpp - Updated return type in DecodedResultsTsType from Array<[string, number, number, number]> to Array<[string, bigint, number, number]>.
- Enhanced error handling in rewind_reader_and_validate_encoding_type and get_version functions, including detailed exception messages.
- Improved logging in create method for buffer length and refined exception handling for ZstdDecompressor.

Possibly related issues

Possibly related PRs

Suggested reviewers

  • kirkrodrigues: Suggested as a reviewer for this PR.

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 450e8f8 and cbc336e.

📒 Files selected for processing (1)
  • src/clp_ffi_js/ir/StreamReader.cpp (1 hunks)
🔇 Additional comments (2)
src/clp_ffi_js/ir/StreamReader.cpp (2)

127-127: Excellent choice using bigint for timestamp representation!

The change from number to bigint is a significant improvement for timestamp handling. JavaScript's number type can only safely represent integers up to 2^53-1, which could lead to precision loss for larger timestamp values. Using bigint ensures accurate representation of timestamps without any precision concerns.


127-127: Verify all consumers of decodeRange method

While the type change is correct, we should ensure all JavaScript/TypeScript code consuming the decodeRange method's output has been updated to handle bigint values instead of number.

✅ Verification successful

Type change appears to be an internal implementation detail

Based on the investigation:

  1. The decodeRange method is only exposed through the C++ FFI interface
  2. The type change is properly registered with Emscripten's type system
  3. The package exports compiled JavaScript bundles from the dist directory
  4. No direct JavaScript/TypeScript consumers were found in the source code

Since this is a compiled FFI library where the type change is handled at the binding layer and properly registered, there's no need for manual verification of JavaScript consumers. The TypeScript type definitions will be generated during the build process to match the new type signature.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for usages of decodeRange to identify potential consumers that need updating
rg -l "decodeRange" --type ts --type js

# Search for any existing bigint type usage to verify consistency
rg -l "bigint" --type ts --type js

Length of output: 372


Script:

#!/bin/bash
# Let's first find all TypeScript and JavaScript files in the repository
fd "\.(js|ts|jsx|tsx)$"

# Then search for decodeRange usage in all files
rg "decodeRange" -A 3 -B 3

# Also check for any bigint usage
rg "bigint" -A 3 -B 3

Length of output: 1311


Script:

#!/bin/bash
# Let's check the implementation of decode_range to understand its usage
ast-grep --pattern 'decode_range($$$) {
  $$$
}'

# Let's also check for any test files that might be using this functionality
fd "test|spec" "\.(js|ts|jsx|tsx|cpp)$"

# And check the package.json or similar build files for any TypeScript configurations
fd "package.json|tsconfig.json"

Length of output: 249


Script:

#!/bin/bash
# Let's check the package.json for dependencies and TypeScript configurations
cat package.json

# Let's try finding test files with a corrected syntax
fd -e js -e ts -e jsx -e tsx -e cpp . | grep -i "test\|spec"

# Let's also check if there are any direct imports of StreamReader
rg -B 3 -A 3 "import.*StreamReader|require.*StreamReader"

Length of output: 1040


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@junhaoliao junhaoliao changed the title Update timestamp type declaration for DecodedResultsTsType. Update DecodedResultsTsType to use bigint in Emscripten binding. Nov 10, 2024
@@ -124,7 +124,7 @@ EMSCRIPTEN_BINDINGS(ClpStreamReader) {
.value("STRUCTURED", clp_ffi_js::ir::StreamType::Structured)
.value("UNSTRUCTURED", clp_ffi_js::ir::StreamType::Unstructured);
emscripten::register_type<clp_ffi_js::ir::DecodedResultsTsType>(
"Array<[string, number, number, number]>"
"Array<[string, bigint, number, number]>"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note the behaviour with logEventNum of type size_t is different: #33 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change (perhaps in another PR) the log event indices into uint32_t? I feel like having more than 4B log events in a file is unlikely.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change (perhaps in another PR) the log event indices into uint32_t?

Let's first investigate the behaviour after setting -sMEMORY64=1. If the underlying type of size_t changes to 64 bit unsigned, I think it makes sense to always use uint32_t.

@junhaoliao junhaoliao changed the title Update DecodedResultsTsType to use bigint in Emscripten binding. Update DecodedResultsTsType to use bigint in Emscripten binding (fixes #33). Nov 10, 2024
junhaoliao added a commit to junhaoliao/yscope-log-viewer that referenced this pull request Nov 11, 2024
@davemarco
Copy link
Contributor

davemarco commented Nov 11, 2024

Before we merge this we should finalize what to do with json bigint, as this could possible break log viewer

@junhaoliao
Copy link
Collaborator Author

Before we merge this we should finalize what to do with json bigint, as this could possible break log viewer

The returned timestamp is not used in any JSON.stringify() (yet), right?

@davemarco
Copy link
Contributor

davemarco commented Nov 11, 2024

The returned timestamp is not used in any JSON.stringify() (yet), right?

Yes you are correct as discussed. Nevermind for json parse as it just reduces precision

@junhaoliao junhaoliao merged commit 288783a into y-scope:main Nov 11, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants