diff --git a/doc/psidk/src/SUMMARY.md b/doc/psidk/src/SUMMARY.md index 715484a80..8dbf624ad 100644 --- a/doc/psidk/src/SUMMARY.md +++ b/doc/psidk/src/SUMMARY.md @@ -3,6 +3,7 @@ - [Introduction](README.md) - [Specifications](specifications/README.md) + - [Blockchain](specifications/blockchain/README.md) - [TaPoS](specifications/blockchain/tapos.md) - [Smart authorization](specifications/blockchain/smart-authorization.md) @@ -29,6 +30,7 @@ - [App Packages](specifications/data-formats/package.md) - [Development guides](development/README.md) + - [Services](development/services/README.md) - [Action scripts](development/services/action-scripts.md) - [Standards](development/services/standards.md) @@ -57,6 +59,7 @@ - [GraphQL](development/services/rust-service/graphql.md) - [Reference]() - [Web Services](development/services/rust-service/reference/web-services.md) + - [Chicken Scratch](development/services/rust-service/reference/chicken-scratch.md) - [Plugins]() - [Front-ends](development/front-ends/README.md) - [User onboarding]() @@ -66,6 +69,7 @@ - [JS libraries](development/front-ends/reference/js-libraries.md) - [Running infrastructure](run-infrastructure/README.md) + - [Installation]() - [Native binaries]() - [Docker containers]() @@ -78,6 +82,7 @@ - [Logging](run-infrastructure/configuration/logging.md) - [Default apps](default-apps/README.md) + - [accounts](default-apps/accounts.md) - [x-admin](default-apps/x-admin.md) - [auth-sig](default-apps/auth-sig.md) diff --git a/doc/psidk/src/development/services/rust-service/reference/chicken-scratch.md b/doc/psidk/src/development/services/rust-service/reference/chicken-scratch.md new file mode 100644 index 000000000..e462a11f7 --- /dev/null +++ b/doc/psidk/src/development/services/rust-service/reference/chicken-scratch.md @@ -0,0 +1,192 @@ +# Rust Dev Chicken Scratch + +This is a place to capture + +1. dev challenges we come across while working that either need better docs or better dev tools +2. bugs that need addresses +3. dev weak points that require a dev to know to much about the system, i.e., there's no reason the dev experience should be so esoteric + +Note: some doc updates are done here simply because we're more intersted in capturing the issues (rather than forgetting them) than we are in writing perfect docs. Once they're captured, we can always go back and 1) write good docs and 2) fix dev issues we have recorded. If we don't capture them as they happen, they disappear to time, and the dev experience sucks (as we acclimate to hardships we've gotten used to and forget new devs will be in the dark about.) + +## Potential Tasks + +1. add tests for psibase_macros +2. fix hygiene of psibase_macros.service_impl (details below in Bugs section) + +## Doc Updates + +### Tables + +#### Singleton Tables + +[Current doc on Singletons](development/services/rust-service/tables.html#singletons) uses in-place code, rather than a generalized, explicit Singleton construct. Probably worth defining a `Singleton` thing that makes it explicit what's happening. Could be a type def, a struct, or some kind of Trait or Key type. + +The code snippets I think could be improved are as follows: + +``` +// A somewhat esoteric way to define the key +impl LastUsed { + // The primary key is an empty tuple. Rust functions + // which return nothing actually return (). + #[primary_key] + fn pk(&self) {} +} +``` + +``` +// Requesting the only table record: the esoteric key = `&()` +let mut lastUsed = + table.get_index_pk().get(&()).unwrap_or_default(); +``` + +#### Externalizing Table definitions and their structs / Code Splitting + +The basic Table docs for "[Storing Structs Defined Elsewhere](development/services/rust-service/tables.html#storing-structs-defined-elsewhere)" are outdated. + +1. `Reflect` doesn't seem to exist. +2. using the documented `impl WrapMessage { ... }` doesn't seem to work anymore. Updated doc that I believe accomplishes the same thing is below. + +The `service` mod, being decorated by the `service` attribute macro, has a lot of magic go on when it processes the mod. It can be very non-obvious how to split out tables and lead to a dev putting all their code in the module. Obviously, we need a way to separate tables out from the module for better code splitting, readability, and encapsulation. + +First you can move the table's struct (record definition) out of the module by structuring it as follows: + +``` +use psibase::{Fracpack, TableRecord}; +use serde::{Deserialize, Serialize}; + +impl TableRecord for MyTableRec { + type PrimaryKey = u64; + + const SECONDARY_KEYS: u8 = 0; + + fn get_primary_key(&self) -> Self::PrimaryKey { + self.event_id + } +} + +#[derive(Debug, Fracpack, Serialize, Deserialize)] +pub struct MyTableRec { + pub field1: u64, + pub field2: String, +} +``` + +Then define only the table itself in the `service` mod: + +``` + use psibase::{Fracpack, Table}; + use serde::{Deserialize, Serialize}; + ... + #[table(name = "MyTable", record = "MyTableRec")] + #[derive(Debug, Fracpack, Serialize, Deserialize)] + struct MyTable; + ... +``` + +## Dev Challenges + +## Bugs + +`service` macro's hygiene could use some cleanup. + +1. `anyhow` must be imported for the macro to be happy (need to clarify under what circumstances this is the case to fix it properly) +2. `Table` must be imported for the macro to be happy (need to figure out which table def exactly require it. Maybe just be when record = "" is specified in the `table` macro). + `Table` is a Trait that implements things like ::new(). Getting a reference to a table won't work without bringing the Trait into scope. We could bring it in scope whenever the named table shows up in an #[action]. We could also have the macro look for any element of the Trait and include `Table` only if it finds it being used. Perhaps there's a way to define tables that naturally pulls it into scope? +3. `asyncgraphql_*` need to be `use`d in some cases. should come along with Query definitions. + +```svgbob ++-------------+ +---------+ +---------+ +| http-server | | | | HTTP | +| service |<---- | psinode |<---- | Request | +| | | | | | ++-------------+ +---------+ +---------+ + | + | + v + +--------------+ +-----------------+ + / \ yes | common-api | +/ target begins \ ------> | service's | +\ with "/common?" / | serveSys action | + \ / +-----------------+ + +--------------+ ^ + | no | + | +-----------+ + v | + +----------------+ | +-----------------+ + / \ no | | sites | +/ on a subdomain? \ ---+ | service's | +\ / | serveSys action | + \ / +-----------------+ + +----------------+ ^ + | yes | + | +----------------+ + v | + +------------+ no | +-----------------+ + / \ ---+ | registered | +/ registered? \ | service's | +\ / yes +-->| serveSys action | + \ / -------+ +-----------------+ + +------------+ +``` + +`psinode` passes most HTTP requests to the [SystemService::HttpServer] service, which then routes requests to the appropriate service's [serveSys](https://docs.rs/psibase/latest/psibase/server_interface/struct.ServerActions.html#method.serveSys) action (see diagram). The services run in RPC mode; this prevents them from writing to the database, but allows them to read data they normally can't. See [psibase::DbId](https://docs.rs/psibase/latest/psibase/enum.DbId.html). + +[SystemService::CommonApi] provides services common to all domains under the `/common` tree. It also serves the chain's main page. + +[SystemService::Sites] provides web hosting for non-service accounts or service accounts that did not [register](#registration) for HTTP handling. + +`psinode` directly handles requests which start with `/native`, e.g. `/native/push_transaction`. Services don't serve these. + +## Registration + +Services which wish to serve HTTP requests need to register using the [SystemService::HttpServer] service's [SystemService::HttpServer::registerServer] action. This is usually done by setting the `package.metadata.psibase.server` field in `Cargo.toml` to add this action to the package installation process. + +A service doesn't have to serve HTTP requests itself; it may delegate this to another service during registration. + +## HTTP Interfaces + +Services which serve HTTP implement these interfaces: + +- [psibase::server_interface](https://docs.rs/psibase/latest/psibase/server_interface/index.html) (required) + - [psibase::HttpRequest](https://docs.rs/psibase/latest/psibase/struct.HttpRequest.html) + - [psibase::HttpReply](https://docs.rs/psibase/latest/psibase/struct.HttpReply.html) +- [psibase::storage_interface](https://docs.rs/psibase/latest/psibase/storage_interface/index.html) (optional) + +## Helpers + +These help implement basic functionality: + +- [psibase::serve_simple_ui](https://docs.rs/psibase/latest/psibase/fn.serve_simple_ui.html) +- [psibase::serve_simple_index](https://docs.rs/psibase/latest/psibase/fn.serve_simple_index.html) +- [psibase::serve_action_templates](https://docs.rs/psibase/latest/psibase/fn.serve_action_templates.html) +- [psibase::serve_schema](https://docs.rs/psibase/latest/psibase/fn.serve_schema.html) +- [psibase::serve_pack_action](https://docs.rs/psibase/latest/psibase/fn.serve_pack_action.html) + +Here's a common pattern for using these functions. +[`#[psibase::service]`](https://docs.rs/psibase/latest/psibase/attr.service.html) defines `Wrapper`; +the `serve_*` functions fetch action definitions from Wrapper. + +```rust +#[psibase::service] +#[allow(non_snake_case)] +mod service { + use psibase::*; + + #[action] + fn serveSys(request: HttpRequest) -> Option { + if request.method == "GET" + && (request.target == "/" || request.target == "/index.html") + { + return Some(HttpReply { + contentType: "text/html".into(), + body: "This is my UI".into(), + headers: vec![], + }); + } + + None.or_else(|| serve_schema::(&request)) + .or_else(|| serve_action_templates::(&request)) + .or_else(|| serve_pack_action::(&request)) + } +} +``` diff --git a/services/user/Chainmail/Cargo.lock b/services/user/Chainmail/Cargo.lock index 5ec65dd5b..21e281b3c 100644 --- a/services/user/Chainmail/Cargo.lock +++ b/services/user/Chainmail/Cargo.lock @@ -1443,6 +1443,8 @@ dependencies = [ "chainmail", "chainmail_package", "psibase", + "serde", + "serde_json", "wit-bindgen-rt", ] @@ -1833,9 +1835,9 @@ dependencies = [ [[package]] name = "serde" -version = "1.0.209" +version = "1.0.210" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "99fce0ffe7310761ca6bf9faf5115afbc19688edd00171d81b1bb1b116c63e09" +checksum = "c8e3592472072e6e22e0a54d5904d9febf8508f65fb8552499a1abc7d1078c3a" dependencies = [ "serde_derive", ] @@ -1873,9 +1875,9 @@ dependencies = [ [[package]] name = "serde_derive" -version = "1.0.209" +version = "1.0.210" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a5831b979fd7b5439637af1752d535ff49f4860c0f341d1baeb6faf0f4242170" +checksum = "243902eda00fad750862fc144cea25caca5e20d615af0a81bee94ca738f1df1f" dependencies = [ "proc-macro2", "quote", diff --git a/services/user/Chainmail/plugin/Cargo.toml b/services/user/Chainmail/plugin/Cargo.toml index 1cba67759..6ecbf05cd 100644 --- a/services/user/Chainmail/plugin/Cargo.toml +++ b/services/user/Chainmail/plugin/Cargo.toml @@ -11,6 +11,8 @@ publish = false wit-bindgen-rt = { version = "0.28.0", features = ["bitflags"] } psibase = { path = "../../../../rust/psibase/" } chainmail = { path = "../service/" } +serde_json = "1.0.128" +serde = "1.0.210" [lib] crate-type = ["cdylib"] diff --git a/services/user/Chainmail/plugin/src/errors.rs b/services/user/Chainmail/plugin/src/errors.rs new file mode 100644 index 000000000..187a25c74 --- /dev/null +++ b/services/user/Chainmail/plugin/src/errors.rs @@ -0,0 +1,25 @@ +use crate::bindings::host::common::types::{Error, PluginId}; + +#[derive(PartialEq, Eq, Hash)] +pub enum ErrorType { + QueryResponseParseError, +} + +fn my_plugin_id() -> PluginId { + return PluginId { + service: "chainmail".to_string(), + plugin: "plugin".to_string(), + }; +} + +impl ErrorType { + pub fn err(self, msg: &str) -> Error { + match self { + ErrorType::QueryResponseParseError => Error { + code: self as u32, + producer: my_plugin_id(), + message: format!("Query response parsing error: {}", msg), + }, + } + } +} diff --git a/services/user/Chainmail/plugin/src/lib.rs b/services/user/Chainmail/plugin/src/lib.rs index 485138099..0531d3274 100644 --- a/services/user/Chainmail/plugin/src/lib.rs +++ b/services/user/Chainmail/plugin/src/lib.rs @@ -1,14 +1,24 @@ #[allow(warnings)] mod bindings; +mod errors; +use crate::bindings::host::common::server; use bindings::exports::chainmail::plugin::api::{Error, Guest as API}; use bindings::transact::plugin::intf as Transact; +use errors::ErrorType; use psibase::fracpack::Pack; -use psibase::services::chainmail; use psibase::AccountNumber; +use serde::Deserialize; struct ChainmailPlugin; +#[derive(Deserialize, Debug)] +struct Message { + receiver: AccountNumber, + subject: String, + body: String, +} + impl API for ChainmailPlugin { fn send(receiver: String, subject: String, body: String) -> Result<(), Error> { Transact::add_action_to_transaction( @@ -22,6 +32,35 @@ impl API for ChainmailPlugin { )?; Ok(()) } + + fn archive(event_id: u64) -> Result<(), Error> { + Transact::add_action_to_transaction( + "archive", + &chainmail::action_structs::archive { event_id }.packed(), + )?; + Ok(()) + } + + fn save(event_id: u64) -> Result<(), Error> { + // look up message details via event_id + // let (sender, receiver, subject, body) = fetch.get(/rest/message by id); + let res = server::get_json(&format!("/messages?id={}", event_id))?; + + let msg = serde_json::from_str::(&res) + .map_err(|err| ErrorType::QueryResponseParseError.err(err.to_string().as_str()))?; + + // save the message to state + Transact::add_action_to_transaction( + "save", + &chainmail::action_structs::save { + receiver: msg.receiver, + subject: msg.subject, + body: msg.body, + } + .packed(), + )?; + Ok(()) + } } bindings::export!(ChainmailPlugin with_types_in bindings); diff --git a/services/user/Chainmail/plugin/wit/world.wit b/services/user/Chainmail/plugin/wit/world.wit index 26a448b72..b9eeb1a8f 100644 --- a/services/user/Chainmail/plugin/wit/world.wit +++ b/services/user/Chainmail/plugin/wit/world.wit @@ -5,6 +5,12 @@ interface api { // Send an email send: func(receiver: string, subject: string, body: string) -> result<_, error>; + + // Archive an email + archive: func(event-id: u64) -> result<_, error>; + + // Save an email + save: func(event-id: u64) -> result<_, error>; } world imports { diff --git a/services/user/Chainmail/service/src/event_query_helpers.rs b/services/user/Chainmail/service/src/event_query_helpers.rs new file mode 100644 index 000000000..cd7c1c82e --- /dev/null +++ b/services/user/Chainmail/service/src/event_query_helpers.rs @@ -0,0 +1,111 @@ +use std::collections::HashMap; + +use crate::helpers::validate_user; + +use psibase::services::r_events::Wrapper as REventsSvc; + +use psibase::{HttpReply, HttpRequest}; + +fn make_query(req: &HttpRequest, sql: String) -> HttpRequest { + return HttpRequest { + host: req.host.clone(), + rootHost: req.rootHost.clone(), + method: String::from("POST"), + target: String::from("/sql"), + contentType: String::from("application/sql"), + body: sql.into(), + }; +} + +fn parse_query(query: &str) -> HashMap { + let mut params: HashMap = HashMap::new(); + + let itr = query.split("&"); + for p in itr { + let kv = p.split_once("=").unwrap(); + params.insert(kv.0.to_string(), kv.1.trim_start_matches('=').to_string()); + } + params +} + +fn get_where_clause_from_sender_receiver_params(params: HashMap) -> Option { + let mut s_clause = String::new(); + let s_opt = params.get("sender"); + if let Some(s) = s_opt { + if !validate_user(s) { + return None; + } + s_clause = format!("sender = '{}'", s); + } + + let mut r_clause = String::new(); + let r_opt = params.get(&String::from("receiver")); + if let Some(r) = r_opt { + if !validate_user(r) { + return None; + } + r_clause = format!("receiver = '{}'", r); + } + + if s_opt.is_none() && r_opt.is_none() { + return None; + } + + let mut where_clause: String = String::from("WHERE "); + if s_opt.is_some() { + where_clause += s_clause.as_str(); + } + if s_opt.is_some() && r_opt.is_some() { + where_clause += " AND "; + } + if r_opt.is_some() { + where_clause += r_clause.as_str(); + } + + Some(where_clause) +} + +pub fn serve_rest_api(request: &HttpRequest) -> Option { + if request.method == "GET" { + if !request.target.starts_with("/messages") { + return None; + } + + let query_start = request.target.find('?'); + if query_start.is_none() { + return None; + } + let query_start = query_start.unwrap(); + + let query = request.target.split_at(query_start + 1).1; + let params = parse_query(query); + + let where_clause: String; + let mq: HttpRequest; + // handle id param requests (for specific message) + if params.contains_key("id") { + where_clause = format!("WHERE CONCAT(receiver, rowid) = {}", params.get("id")?); + + mq = make_query( + request, + format!( + "SELECT * + FROM \"history.chainmail.sent\" {} ORDER BY ROWID", + where_clause + ), + ); + // handle receiver or sender param requests + } else { + where_clause = get_where_clause_from_sender_receiver_params(params)?; + + mq = make_query( + request, + format!("SELECT * + FROM \"history.chainmail.sent\" AS sent + LEFT JOIN \"history.chainmail.archive\" AS archive ON CONCAT(sent.receiver, sent.rowid) = archive.event_id {} ORDER BY ROWID", where_clause), + ); + } + return REventsSvc::call().serveSys(mq); + } + return None; +} diff --git a/services/user/Chainmail/service/src/helpers.rs b/services/user/Chainmail/service/src/helpers.rs new file mode 100644 index 000000000..74a2b2ca9 --- /dev/null +++ b/services/user/Chainmail/service/src/helpers.rs @@ -0,0 +1,10 @@ +use psibase::{services::accounts::Wrapper as AccountsSvc, AccountNumber}; + +pub fn validate_user(user: &str) -> bool { + let acc = AccountNumber::from(user); + if acc.to_string() != user { + return false; + } + + AccountsSvc::call().exists(acc) +} diff --git a/services/user/Chainmail/service/src/lib.rs b/services/user/Chainmail/service/src/lib.rs index e463c5abb..53ea22fb5 100644 --- a/services/user/Chainmail/service/src/lib.rs +++ b/services/user/Chainmail/service/src/lib.rs @@ -1,113 +1,21 @@ -use std::collections::HashMap; - -use psibase::services::accounts::Wrapper as AccountsSvc; -use psibase::services::r_events::Wrapper as REventsSvc; -use psibase::AccountNumber; -use psibase::HttpReply; -use psibase::HttpRequest; - -fn validate_user(user: &str) -> bool { - let acc = AccountNumber::from(user); - if acc.to_string() != user { - return false; - } - - AccountsSvc::call().exists(acc) -} - -fn make_query(req: &HttpRequest, sql: String) -> HttpRequest { - return HttpRequest { - host: req.host.clone(), - rootHost: req.rootHost.clone(), - method: String::from("POST"), - target: String::from("/sql"), - contentType: String::from("application/sql"), - body: sql.into(), - }; -} - -fn parse_query(query: &str) -> HashMap { - let mut params: HashMap = HashMap::new(); - - let itr = query.split("&"); - for p in itr { - let kv = p.split_once("=").unwrap(); - params.insert(kv.0.to_string(), kv.1.trim_start_matches('=').to_string()); - } - params -} - -fn serve_rest_api(request: &HttpRequest) -> Option { - if request.method == "GET" { - if !request.target.starts_with("/messages") { - return None; - } - - let query_start = request.target.find('?'); - if query_start.is_none() { - return None; - } - let query_start = query_start.unwrap(); - - let query = request.target.split_at(query_start + 1).1; - let params = crate::parse_query(query); - - let mut s_clause = String::new(); - let s_opt = params.get("sender"); - if let Some(s) = s_opt { - if !validate_user(s) { - return None; - } - s_clause = format!("sender = '{}'", s); - } - - let mut r_clause = String::new(); - let r_opt = params.get(&String::from("receiver")); - if let Some(r) = r_opt { - if !validate_user(r) { - return None; - } - r_clause = format!("receiver = '{}'", r); - } - - if s_opt.is_none() && r_opt.is_none() { - return None; - } - - let mut where_clause: String = String::from("WHERE "); - if s_opt.is_some() { - where_clause += s_clause.as_str(); - } - if s_opt.is_some() && r_opt.is_some() { - where_clause += " AND "; - } - if r_opt.is_some() { - where_clause += r_clause.as_str(); - } - - let mq = make_query( - request, - format!( - "SELECT * FROM \"history.chainmail.sent\" {} ORDER BY ROWID", - where_clause - ), - ); - return REventsSvc::call().serveSys(mq); - } - return None; -} +mod event_query_helpers; +mod helpers; +mod tables; #[psibase::service] mod service { + use crate::event_query_helpers::serve_rest_api; + use crate::tables::SavedMessage; use psibase::services::accounts::Wrapper as AccountsSvc; use psibase::{ anyhow, check, get_sender, get_service, serve_content, serve_simple_ui, store_content, AccountNumber, HexBytes, HttpReply, HttpRequest, Table, WebContentRow, }; - use crate::serve_rest_api; + #[table(name = "SavedMessagesTable", record = "SavedMessage")] + struct SavedMessagesTable; - #[table(record = "WebContentRow")] + #[table(record = "WebContentRow", index = 1)] struct WebContentTable; #[action] @@ -121,8 +29,44 @@ mod service { .sent(get_sender(), receiver, subject, body); } + #[action] + fn archive(event_id: u64) { + Wrapper::emit() + .history() + .archive(get_sender().to_string() + &event_id.to_string()); + } + + #[action] + fn save(event_id: u64, sender: AccountNumber, subject: String, body: String) { + let saved_messages_table = SavedMessagesTable::new(); + + saved_messages_table + .put(&SavedMessage { + event_id, + sender, + subject, + body, + }) + .unwrap(); + () + } + + #[action] + fn unsave(event_id: u64, sender: AccountNumber, subject: String, body: String) { + let saved_messages_table = SavedMessagesTable::new(); + + saved_messages_table.remove(&SavedMessage { + event_id, + sender, + subject, + body, + }) + } + #[event(history)] pub fn sent(sender: AccountNumber, receiver: AccountNumber, subject: String, body: String) {} + #[event(history)] + pub fn archive(event_id: String) {} #[action] #[allow(non_snake_case)] diff --git a/services/user/Chainmail/service/src/tables.rs b/services/user/Chainmail/service/src/tables.rs new file mode 100644 index 000000000..7bc2db7ed --- /dev/null +++ b/services/user/Chainmail/service/src/tables.rs @@ -0,0 +1,20 @@ +use psibase::{AccountNumber, Fracpack, TableRecord}; +use serde::{Deserialize, Serialize}; + +impl TableRecord for SavedMessage { + type PrimaryKey = u64; + + const SECONDARY_KEYS: u8 = 0; + + fn get_primary_key(&self) -> Self::PrimaryKey { + self.event_id + } +} + +#[derive(Debug, Fracpack, Serialize, Deserialize)] +pub struct SavedMessage { + pub event_id: u64, + pub sender: AccountNumber, + pub subject: String, + pub body: String, +}