Enforce consistent datetime serialization #3389

sug0 · 2024-06-07T12:43:36Z

Describe your changes

Our DateTimeUtc type allowed a relaxed representation of RFC3339 strings. We now enforce a string subset of this format, to guarantee deterministic serialization.

Indicate on which release or other PRs this topic is based on

v0.39.0

Checklist before merging to `draft`

I have added a changelog
Git history is in acceptable state

tzemanovic

looks good, just in crates/apps_lib/src/cli.rs pls update help for NAMADA_START_TIME

codecov · 2024-06-07T15:58:11Z

Codecov Report

Attention: Patch coverage is 91.89189% with 6 lines in your changes missing coverage. Please review.

Project coverage is 53.93%. Comparing base (879a326) to head (7f90393).
Report is 4 commits behind head on main.

Files	Patch %	Lines
crates/apps_lib/src/cli.rs	0.00%	4 Missing ⚠️
crates/core/src/time.rs	95.83%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3389   +/-   ##
=======================================
  Coverage   53.92%   53.93%           
=======================================
  Files         317      317           
  Lines      107575   107610   +35     
=======================================
+ Hits        58011    58037   +26     
- Misses      49564    49573    +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

murisi

Looks good to me. Thanks!

Test completeness.

murisi

Just to be sure that hardware wallet will continue to work, could we please demonstrate in a unit test that DateTimeUtc is a subset of DateTime somewhere?

crates/core/src/time.rs

* tiago/consistent-datetime-serialization: Changelog for #3389 Improve tx salting Fix from tm time impl for `DateTimeUtc` Misc fixes Increase gas limit in `FinalizeBlock` tests Rebuild tx fixtures Resign localnet genesis txs Rebuild wasms for tests gen_localnet.py: Fix genesis time string Add datetime encoding tests Enforce fixed RFC3339 format

Co-authored-by: Yuji Ito <[email protected]>

murisi · 2024-06-18T12:59:29Z

@murisi could you please test this pr with the ledger hw wallet?

This PR works with the hardware wallet. We just need to make a (hopefully) minor change on the hardware wallet to increase the precision it displays. See for instance the following where Actual is hardware wallet output and Expected is what's produced in the test vectors.

1: Expected: is equal to "2 | Timestamp : 4900-04-22 06:26:45.041754692 UTC"
1:   Actual: "2 | Timestamp : 4900-04-22 06:26:45.041754 UTC"

murisi

Looks good to me. Thanks!

Test vectors contain data not found in serializations.

murisi

Could we please change the line

namada/crates/sdk/src/signing.rs

Line 1917 in 7f90393

format!("Timestamp : {}", tx.header.timestamp.0),

to format!("Timestamp : {}", tx.header.timestamp), ? This will change the expected output on the Ledger app from 2 | Timestamp : 4900-04-22 06:26:45.041754692 UTC to something like 3 | Timestamp : 4440-02-19T23:31:34.674481+00:00. This is required because the Borsh serialization does not contain the last 3 decimal places.

Either we do the above, or we increase the number of decimal places in the Borsh serialization of a DateTimeUtc so that the hardware wallet is able to meet the precise expectations.

Tested changes before the forced pushes.

murisi

Great! This PR does not break the hardware wallet. Thanks!

* tiago/consistent-datetime-serialization: Changelog for #3389 Regen tx fixtures Resign localnet genesis txs Rebuild wasms for tests gen_localnet.py: Fix genesis time string Increase precision of timestamps to 9 nanos Fix genesis time in tests Keep nanoseconds during CometBFT time conversions Switch to fixed offset format in UTC Increase robustness of datetime test Improve tx salting Fix from tm time impl for `DateTimeUtc` Misc fixes Increase gas limit in `FinalizeBlock` tests Add datetime encoding tests Enforce fixed RFC3339 format

* origin/tiago/consistent-datetime-serialization: Changelog for #3389 Regen tx fixtures Resign localnet genesis txs Rebuild wasms for tests gen_localnet.py: Fix genesis time string Increase precision of timestamps to 9 nanos Fix genesis time in tests Keep nanoseconds during CometBFT time conversions Switch to fixed offset format in UTC Increase robustness of datetime test Improve tx salting Fix from tm time impl for `DateTimeUtc` Misc fixes Increase gas limit in `FinalizeBlock` tests Add datetime encoding tests Enforce fixed RFC3339 format

Enforce fixed RFC3339 format

ad4bae8

sug0 added the bug Something isn't working label Jun 7, 2024

sug0 added a commit that referenced this pull request Jun 7, 2024

Changelog for #3389

a587e9b

sug0 requested a review from tzemanovic June 7, 2024 12:44

sug0 added a commit that referenced this pull request Jun 7, 2024

Changelog for #3389

1ef2adf

sug0 force-pushed the tiago/consistent-datetime-serialization branch from a587e9b to 1ef2adf Compare June 7, 2024 12:48

sug0 added breaking:state and removed breaking:state labels Jun 7, 2024

tzemanovic previously approved these changes Jun 7, 2024

View reviewed changes

Add datetime encoding tests

18b76ac

sug0 added a commit that referenced this pull request Jun 7, 2024

Changelog for #3389

7f38267

sug0 dismissed tzemanovic’s stale review via 7f38267 June 7, 2024 15:07

sug0 force-pushed the tiago/consistent-datetime-serialization branch from 1ef2adf to 7f38267 Compare June 7, 2024 15:07

sug0 added a commit that referenced this pull request Jun 7, 2024

Changelog for #3389

3352219

sug0 force-pushed the tiago/consistent-datetime-serialization branch from 7f38267 to 3352219 Compare June 7, 2024 15:15

sug0 added a commit that referenced this pull request Jun 7, 2024

Changelog for #3389

834a468

sug0 force-pushed the tiago/consistent-datetime-serialization branch from 3352219 to 834a468 Compare June 7, 2024 15:29

sug0 requested a review from tzemanovic June 7, 2024 15:29

cwgoes mentioned this pull request Jun 7, 2024

Draft #3377

Closed

sug0 marked this pull request as draft June 7, 2024 20:35

sug0 added a commit that referenced this pull request Jun 7, 2024

Changelog for #3389

c18864e

sug0 force-pushed the tiago/consistent-datetime-serialization branch from 834a468 to c18864e Compare June 7, 2024 21:10

sug0 marked this pull request as ready for review June 7, 2024 21:11

sug0 requested a review from murisi June 7, 2024 21:12

murisi previously approved these changes Jun 8, 2024

View reviewed changes

murisi requested changes Jun 8, 2024

View reviewed changes

crates/core/src/time.rs Show resolved Hide resolved

murisi reviewed Jun 8, 2024

View reviewed changes

crates/core/src/time.rs Outdated Show resolved Hide resolved

sug0 and others added 14 commits June 18, 2024 10:54

Increase gas limit in FinalizeBlock tests

d624cc8

Misc fixes

d43e301

Fix from tm time impl for DateTimeUtc

be6ed63

Improve tx salting

b9cf51c

Increase robustness of datetime test

03ccc85

Switch to fixed offset format in UTC

798dfe6

Keep nanoseconds during CometBFT time conversions

6183a62

Co-authored-by: Yuji Ito <[email protected]>

Fix genesis time in tests

6050624

Increase precision of timestamps to 9 nanos

8589b40

gen_localnet.py: Fix genesis time string

45c15e1

Rebuild wasms for tests

543942b

Resign localnet genesis txs

d1a9e54

Regen tx fixtures

bcad8e8

Changelog for #3389

7f90393

sug0 force-pushed the tiago/consistent-datetime-serialization branch from 0ac361d to 7f90393 Compare June 18, 2024 09:55

murisi previously approved these changes Jun 18, 2024

View reviewed changes

murisi previously requested changes Jun 18, 2024

View reviewed changes

murisi approved these changes Jun 18, 2024

View reviewed changes

sug0 added the ready for draft label Jun 19, 2024

brentstone added a commit that referenced this pull request Jun 25, 2024

fixup! Merge branch 'tiago/consistent-datetime-serialization' (#3389)

0b2fb8f

brentstone added the merged to draft label Jun 28, 2024

brentstone added a commit that referenced this pull request Jul 4, 2024

fixup! Merge branch 'tiago/consistent-datetime-serialization' (#3389)

4501f14

brentstone merged commit b9ce576 into main Jul 5, 2024
19 of 21 checks passed

brentstone deleted the tiago/consistent-datetime-serialization branch July 5, 2024 21:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enforce consistent datetime serialization #3389

Enforce consistent datetime serialization #3389

sug0 commented Jun 7, 2024 •

edited

Loading

tzemanovic left a comment

codecov bot commented Jun 7, 2024 •

edited

Loading

murisi left a comment

murisi left a comment

murisi commented Jun 18, 2024

murisi left a comment

murisi left a comment

murisi left a comment

Enforce consistent datetime serialization #3389

Enforce consistent datetime serialization #3389

Conversation

sug0 commented Jun 7, 2024 • edited Loading

Describe your changes

Indicate on which release or other PRs this topic is based on

Checklist before merging to draft

tzemanovic left a comment

Choose a reason for hiding this comment

codecov bot commented Jun 7, 2024 • edited Loading

Codecov Report

murisi left a comment

Choose a reason for hiding this comment

murisi left a comment

Choose a reason for hiding this comment

murisi commented Jun 18, 2024

murisi left a comment

Choose a reason for hiding this comment

murisi left a comment

Choose a reason for hiding this comment

murisi left a comment

Choose a reason for hiding this comment

sug0 commented Jun 7, 2024 •

edited

Loading

Checklist before merging to `draft`

codecov bot commented Jun 7, 2024 •

edited

Loading