Skip to content
This repository has been archived by the owner on Jun 7, 2023. It is now read-only.

RFC: Transaction type #3

Closed
wants to merge 28 commits into from
Closed
Changes from 6 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
22f6a6b
Added RFC for implementing model crate
samuel-rufi Sep 6, 2019
5f21bf8
Aligned with RFC folder structure. Little changes in code.
samuel-rufi Sep 18, 2019
9d43426
Updating RFC with Bundle and Bundle Builder.
samuel-rufi Sep 24, 2019
2fd4d3d
Replaced "receipt" with "receiver" and "withdrawal" with sender.
samuel-rufi Sep 28, 2019
b3fe519
Added metadata struct, and a proof of work example
samuel-rufi Oct 1, 2019
2bdcd48
Snapshot index fix
samuel-rufi Oct 1, 2019
8195ed9
Update "Name" to "name" since template change
samuel-rufi Oct 1, 2019
da1937c
Update "Date" to "date" since template change
samuel-rufi Oct 1, 2019
b6abf6a
update iotaledger/bee-rfcs#000 to iotaledger/bee-rfcs#3
samuel-rufi Oct 1, 2019
4e37c07
Order fields, fix i64 to u64, updated "Unresolved questions"
samuel-rufi Oct 1, 2019
740353f
Fix description of last_index field. Fix unsigned/signed types
samuel-rufi Oct 1, 2019
0fdbed2
Moved motivation part to summary, displayed transaction fields in a t…
samuel-rufi Oct 1, 2019
7846456
Enlarge RFC, Adding questions to "Unresolved questions"
samuel-rufi Oct 1, 2019
fdd56e5
changed "built" do "build"
samuel-rufi Oct 2, 2019
b00dbd7
removed "hash suffix" in table. moved nonce after all other fields
samuel-rufi Oct 2, 2019
7ca1c8b
Merge remote-tracking branch 'origin/master'
samuel-rufi Oct 2, 2019
4a3f627
Update text/0000-bee-model-crate/0000-bee-model-crate.md
samuel-rufi Oct 7, 2019
43c2355
Replaced types of Transaction/TransactionBuilder. Fields-validator is…
samuel-rufi Oct 7, 2019
cc8628d
Changed types of TransactionMetadata and provided impl with setter & …
samuel-rufi Oct 7, 2019
6cb0e6b
Merge remote-tracking branch 'origin/master'
samuel-rufi Oct 7, 2019
5b2fb19
added why transactions should not be modifiable, what is meant with "…
samuel-rufi Oct 8, 2019
4870bae
Update unresolved questions.
samuel-rufi Oct 8, 2019
a9c6db7
Add explanation for the field types. Added unresolved questions. Adde…
samuel-rufi Oct 8, 2019
fd22880
Simplified summary, use transaction version following IRI
Oct 10, 2019
7163c78
Added motivational section
Oct 10, 2019
d4bda0d
Rewrote detailed design
Oct 10, 2019
50871c2
Fix table formatting
Oct 11, 2019
df0b8b8
Simplify Transaction design
Oct 14, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
334 changes: 334 additions & 0 deletions text/0000-bee-model-crate/0000-bee-model-crate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,334 @@
+ Feature Name: `bee-model-crate`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should rename it to something like iri-transaction-model or mainnet-transaction-model or v1-transaction-model.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd vote for versioning since mainnet-transaction-model is not time-proof and iri-transaction-model is not really the case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you want to version it?

samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
+ Start Date: 2019-09-06
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
+ RFC PR: [iotaledger/bee-rfcs#0000](https://github.com/iotaledger/bee-rfcs/pull/0000)
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
+ Bee issue: [iotaledger/bee#43](https://github.com/iotaledger/bee/issues/43)

# Summary

This feature is responsible for the creation and interpretation of transactions and bundles.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'ld like the Summary a little bit more detailed, what is actually introduced by this feature. I know that transactions and bundles are absolutely essential to IOTA, but for consistency with other RFCs could you write this section "as if" you would introduce those concepts with this proposal. I think we can expect the reader to have "some" DLT knowledge (so this summary should still remain rather short), but IMHO we should write those RFCs also for people that are fairly new to IOTA to get as many people involved as possible. The summary/abstract will be the first thing someone interested in it will read, and most of the times the last thing as well.

This comment was marked as resolved.


# Motivation

IOTA is a distributed ledger that was designed for payment settlement and data transfer between machines and devices in the Internet of Things (IoT) ecosystem.
The data packets that are sent through the network are called "transactions".
Settlements or data transfers can be done with the help of these transactions. Payment settlements require the help of multiple transactions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think those sentences should move into the summary as they very well provide the necessary context to understand this proposal. To not repeat yourself in this section, you could pick up from the summary and elaborate on what transactions and bundles are in more detail, and how they are useful or even indispensable for the Bee framework.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @Alex6323 here.

# Detailed design

## General

A transaction consists of several fields (e.g. address, value, timestamp, tag). Each field is of static length. One transaction consists of 2673 trytes:
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

- **trunk hash** the hash of the first transaction referenced/approved = 81 trytes
- **branch hash** the hash of the second transaction referenced/approved = 81 trytes
- **signature_and_message_fragment** contains the signature of the transfer or user-defined message data = 2187 trytes
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
- **value** the transferred amount in IOTA = 27 trytes
- **address** receiver (output) if value > 0, or sender (input) if value < 0 = 81 trytes
- **timestamp** the time when the transaction was issued = 9 trytes
- **attachment_timestamp** the timestamp for when Proof-of-Work is completed = 9 trytes
- **attachment_timestamp_lowerbound** is a slot for future use = 9 trytes
- **attachment_timestamp_upperbound** is a slot for future use = 9 trytes
- **tag** arbitrary user-defined value = 27 trytes
- **obsolete_tag** another arbitrary user-defined tag = 27 trytes
- **nonce** is the Proof-of-Work nonce of the transaction = 27 trytes
- **bundle hash** is the hash of the entire bundle = 81 trytes
- **current_index** the position of the transaction in its bundle = 9 trytes
- **last_index** the total number of transactions in the bundle = 9 trytes
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

A bundle is a collection of specific transactions. Bundles are required to bundle related information.
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
For example, if not all data fits into one transaction, it has to be split across multiple transactions. An example would be payment settlements - they require the help of multiple transactions.
A bundle then represents the collection of these transactions. It should be noted, that each transaction of a bundle will be sent separately through the network.

Each transaction is identified by its hash, the transaction hash. Transactions should be final and therefore not modifiable after construction.
The same applies to bundles. Bundles are final, and therefore not modifiable after construction.
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

Transactions can be built from Transaction Builders.
Bundles can be built from Bundle Builders. Both builder objects can be manipulated as desired.
It should be noted, Bundle Builders don't process Transactions (as transactions are final), instead, they process Transaction Builders.
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

## Exposed Interface

### Transaction

A transaction consists of following fields:

```rust
struct Transaction {

signature_fragments: String,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like the name to reflect that it doesn't necessarily contain a signature. signature_or_message ?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to even not contain this field if the instance doesn't have signature_or_message? It would be sth like this:

Suggested change
signature_fragments: String,
signature_or_message: Option<String>,

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Option still allocates memory for the field so it's not really helpful

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SuperFluffy what are your thoughts regarding Option. Semantically, it would be more correct, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it makes more sense "semantically", an empty sig_or_msg doesn't mean there is no sig_or_msg. There is always a sig_or_msg :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, so maybe Option isn't necessary. And if we use static types, we could initialize it with an empty (9's) SignatureOrMessageFragment and could be sure the internal state of the builder object is always valid.

address: String,
value: i64,
obsolete_tag: String,
timestamp: i64,
current_index: usize,
last_index: usize,
bundle_hash: String,
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
trunk: String,
branch: String,
nonce: String,
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
tag: String,
attachment_timestamp: i64,
attachment_timestamp_lower_bound: i64,
attachment_timestamp_upper_bound: i64,
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

}
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
```
Copy link
Contributor

@Alex6323 Alex6323 Sep 26, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest a different approach here, because I see two main issues:

  1. the transaction fields are (by design) of constant size known upfront. We should right of the bat make use of this knowledge, and store those fields in fixed sized arrays (if we go that route which I wouldn't suggest as it involves lots of allocations -> see proposal below)
  2. this proposal of a transaction seems to be used for eagerly deserializing the transaction data from the received gossip (T5B1 or T9B2) bytes, which probably involves unnecessary calling of conversion functions in the hot-path, e.g. converting timestamps (i64). I haven't implemented a lot of IOTA nodes, but wouldn't it make more sense to only deserialize transaction fields lazily, that is on-demand? This might be more painful with T5B1 encoding, but with T9B2 each field has an integer byte size as well, which makes partial deserializing a breeze. That's why maybe it would be better to model `Transaction like so:
struct Transaction {
    raw: Trits<T9B2>,
}
impl Transaction {
    ...
    fn address(&self) -> String {/*deserialize at specific location in self.raw*/}
    fn value(&self) -> i64 {/*same*/.}
    ...
}

It is also possible to choose a route in the middle of both extremes, where fields, that are always required, are deserialized eagerly. Just add fields for those to the Transaction struct (e.g. trunk, branch). If those T9B2 encoded trits are also used for transport (e.g. among Bee nodes) then in-memory copy operations could be reduced by a lot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree you fully, let's wait for the others what they think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we indeed know the size of each field a priori, then we should model it accordingly. In addition, I think we should probably introduce a type for each of these fields in case they have to uphold invariants themselves.

Also, an array of Ts with m elements, [T; m], is allocated on the stack in Rust. If we want to have vec semantics, there is also the smallvec crate written by the Servo project.

As regards the T9B2 encoding, we have discussed this yesterday (so after Alex left his review comment here) that we will for now stay within the already existing binary-coded trinary representations.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also agree this. The return value can be what @SuperFluffy said, a fixed size array, or I think maybe it's also possible to be a reference slice.


As mentioned, Transactions are final. Transaction fields therefore should be only accessible by getter functions.
Besides that, a transaction can only be created from a Transaction Builder, or from a constructor function which expects encoded bytes as received e.g. from a network socket.

```rust
impl Transaction {

pub fn from_bytes(bytes: &Vec<u8>) -> Result<Transaction, TransactionBuilderValidationError> {
unimplemented!()
}

pub fn signature_fragments(&self) -> &String {
&self.signature_fragments
}

pub fn address(&self) -> &String {
&self.address
}

pub fn value(&self) -> &i64 {
&self.value
}

pub fn obsolete_tag(&self) -> &String {
&self.obsolete_tag
}

pub fn timestamp(&self) -> &i64 {
&self.timestamp
}

pub fn current_index(&self) -> &usize {
&self.current_index
}

pub fn last_index(&self) -> &usize {
&self.last_index
}

pub fn bundle_hash(&self) -> &String {
&self.bundle_hash
}

pub fn trunk(&self) -> &String {
&self.trunk
}

pub fn branch(&self) -> &String {
&self.branch
}

pub fn nonce(&self) -> &String {
&self.nonce
}

pub fn tag(&self) -> &String {
&self.tag
}

pub fn attachment_timestamp(&self) -> &i64 {
&self.attachment_timestamp
}

pub fn attachment_timestamp_lower_bound(&self) -> &i64 {
&self.attachment_timestamp_lower_bound
}

pub fn attachment_timestamp_upper_bound(&self) -> &i64 {
&self.attachment_timestamp_upper_bound
}

}
```
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

The Transcation_Builder struct contains public accessible fields.
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved
The set values can be validated in the build() function which then returns the constructed transaction.
Moreover, also the Proof-Of-Work will be called in the build() function.

```rust
#[derive(Default)]
pub struct TransactionBuilder {
pub signature_fragments: String,
pub address: String,
pub value: i64,
pub obsolete_tag: String,
pub current_index: usize,
pub last_index: usize,
pub bundle_hash: String,
pub trunk: String,
pub branch: String,
pub tag: String,
pub attachment_timestamp: i64,
pub attachment_timestamp_lower_bound: i64,
pub attachment_timestamp_upper_bound: i64,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a transaction builder, the proper semantics are to keep the fields as Option<T>. Also, I would also leave the fields non-pub and only allow setting them through setter methods.

Copy link
Member Author

@samuel-rufi samuel-rufi Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a transaction builder, the proper semantics are to keep the fields as Option

Great idea

Also, I would also leave the fields non-pub and only allow setting them through setter methods.

Where would you put the validation of the set values? Directly in the setter, or as it currently is in build()? Currently the validation happens one time, else it could be done in each setter directly. Not sure what suites best here. If we decide in each setter, we need to keep in mind that every setter could throw an error and how handy it is for the programmer.

Copy link
Member Author

@samuel-rufi samuel-rufi Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding non-pub there is the idea to build the actual Transaction (creating the struct) in Pow::compute(). Currently we have the Transaction fields non-pub. It seems like this ends in something like this https://doc.rust-lang.org/error-index.html#E0451

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use visibility we introduced yesterday then. I think pub(crate) should be fine.


impl TransactionBuilder {

pub fn build(self) -> Result<(Transaction, TranscationMetadata), TransactionBuilderValidationError> {

match TransactionBuilderValidator::validate(&self) {
Ok(()) => { ; }
Err(e) => { return Err(e) }
}

Ok(Pow::compute(&self))

}

}
```

The Pow object takes the Transaction Builder as argument and returns the built transaction together with its metadata.
Every transaction contains a mutable metadata which consists of following attributes:
```rust
pub struct TransactionMetadata {
pub transaction_hash: String,
pub is_solid: bool,
pub snapshot_index: usize
}
thibault-martinez marked this conversation as resolved.
Show resolved Hide resolved
```
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

Even though Proof-Of-Work should be handled in its own RFC, this example shows how it could interact with the TransactionBuilder and how Transactions could be built.
This example assumes that the actual Transaction building takes place here.


```rust
pub struct Pow;

impl Pow {
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

// operates on String encoding, should be replaced with trits interface once ready
pub fn compute(transaction_builder: &TransactionBuilder) -> (Transaction, TranscationMetadata) {

const CHARSET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ9";

let iv: String = (0..27)
.map(|_| {
let mut rng = rand::thread_rng();
let idx = rng.gen_range(0, CHARSET.len());
char::from(unsafe { *CHARSET.get_unchecked(idx) })
})
.collect();

let trytes_backup = transaction_builder.signature_fragments.clone(); // + value + all other fields
let mut hash;
let mut nonce = iv.clone();
let mut timestamp = 0;
let mut transaction_metadata: TranscationMetadata = TranscationMetadata::default();

loop {

let mut trytes = trytes_backup.clone();
trytes.push_str(&nonce);

hash = HashFunction::hash(&trytes);
nonce = String::from(&hash[..27]);
// timestamp = ... (update timestamp)

if hash.ends_with(MIN_WEIGHT_MAGNITUDE_AS_STRING){
transaction_metadata.hash = hash.clone();
break
}

}

(
Transaction {
signature_fragments: transaction_builder.signature_fragments.clone(),
address: transaction_builder.address.clone(),
value: transaction_builder.value.clone(),
obsolete_tag: transaction_builder.obsolete_tag.clone(),
timestamp,
current_index: transaction_builder.current_index.clone(),
last_index: transaction_builder.last_index.clone(),
bundle_hash: transaction_builder.bundle_hash.clone(),
trunk: transaction_builder.trunk.clone(),
branch: transaction_builder.branch.clone(),
nonce,
tag: transaction_builder.tag.clone(),
attachment_timestamp: transaction_builder.attachment_timestamp.clone(),
attachment_timestamp_lower_bound: transaction_builder.attachment_timestamp_lower_bound.clone(),
attachment_timestamp_upper_bound: transaction_builder.attachment_timestamp_upper_bound.clone(),
},

transaction_metadata

)

}

}

```


### Bundle
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

All transactions in the same bundle have the same value in the bundle field. This field contains the bundle hash, which is derived from a hash of the values of each transaction's address, value, obsoleteTag, currentIndex, lastIndex and timestamp fields.
- **address**
- **value**
- **obsoleteTag**
- **currentIndex**
- **lastIndex**
- **timestamp**

#### Bundle Builder

Similar to Transaction Builder, BundleBuilder makes it possible to create a Bundle. As mentioned, a bundle is a special collection of transactions.
Bundles are read from head to tail but created from tail to head. This is why it makes sense to have a dedicated class for this purpose.
The transactions inside a bundle are connected through the trunk. The trunk field of the head transaction (index 0) references the transaction with index 1, and so on.

```rust

struct BundleBuilder<'a> {

pub tailToHead: Vec<&'a TransactionBuilder>

}

impl<'a> BundleBuilder<'a> {

pub fn append(&mut self, transaction_builder: &'a TransactionBuilder) {
self.tailToHead.push(transaction_builder);
}

pub fn build(&self) -> Result<Bundle, BundleBuildError> {

if self.tailToHead.size() == 0 {
return Err(BundleBuilder("Cannot build: bundle is empty (0 transactions)."))
}

setFlags();
buildTrunkLinkedChainAndReturnHead();

Ok(Bundle::new(tailToHead))
}

}

```


# Drawbacks

Without any bee-model crate, nodes can not exchange transactions. Therefore this crate seems necessary.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drawbacks should ideally also discuss the drawbacks of this design. :)

I am thinking of the String types, for example, which could also be done as their own types.


# Rationale and alternatives
samuel-rufi marked this conversation as resolved.
Show resolved Hide resolved

- The distinction between Transaction and Transaction Builder as well as Bundle and Bundle Builder makes the code cleaner
and helps achieve correctness among the data objects. Properties are clearly assigned to specific data objects and not mixed up.

- The proposed crate interface is intuitive. Completely different alternatives did not naturally come to mind.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's intuitive is for users to decide. I agree however that it is a very natural proposition to basically treat the incoming transaction as some encoded binary blob.


- The validation logic could be done in a separate validator class, which then will be called in the build() function.

# Unresolved questions

- How is Proof-Of-Work acceleration handled?