Skip to content

Commit

Permalink
feat(cc-tx-visualization): initial commit
Browse files Browse the repository at this point in the history
Signed-off-by: Iulia Mihaiu <[email protected]>
Signed-off-by: Rafael Belchior <[email protected]>

feat(cc-tx-viz): add methods to fabric connector

Signed-off-by: Iulia Mihaiu <[email protected]>

fix(prometheus): metrics.ts leaks to global registry #1202

1. Specified a `register` property of the gauges as an empty
array so that it does not pollute the global namespace. This
is how this is supposed to be done as per the docs of prom-client.

2. Once the change from 1) took place, the issue became that
the metrics gathering code was still trying to hit up the
global scope for the metrics, e.g. calling the get metrics
methods directly on the promClient object instead of the
registry that we create for each prmoetheus exporter object
separately. So a little additional refactor ensued to fix this
as well by making sure that we grab a reference of the registry
object at construction time and then re-use that wherever needed
instead of going through the global promClient object.

3. Added missing .startMetricsCollection calls in the plugin
constructors to ensure that the prometheus exporter object
gets initialized properly (this is where the registry gets
created as well so without this there are crashes happening
when one tries to access the metrics through the registry)

Why though?
The problem was that the metrics.ts file that we have for all the
plugin's metrics constructs a new Metric (Gauge) object at import
time which then defaults to registering the metric in the global
registry of prom-client by default.

The latter was causing crashes when separate versions of the same
metrics.ts file are imported in a scenario were the API server
imports plugins from a different directory (this issue is coming
from the branch where I'm working on plugin sandboxing via the
live-plugin-manager).

Fixes #1202

Signed-off-by: Peter Somogyvari <[email protected]>

feat(cc-tx-viz): added test for multiple prometheus metrics on fabric

Signed-off-by: Iulia Mihaiu <[email protected]>

feat(cc-tx-viz): add metric model

Signed-off-by: Iulia Mihaiu <[email protected]>

feat(cc-tx-viz): added croschainEventLog models

Signed-off-by: Iulia Mihaiu <[email protected]>

feat(fabric): added endpoint for list of transaction receipts

Signed-off-by: Iulia Mihaiu <[email protected]>

feat(test-tooling): add RabbitMQ test server

feat(cc-tx-viz): add RabbitMQ support; update model

feat(cc-tx-viz): add preliminary support to Fabric and Besu

build(deps): upgrade axios to latest to fix CVE

Details
CVE-2021-3749
high severity
Vulnerable versions: <= 0.21.1
Patched version: 0.21.2
axios is vulnerable to Inefficient Regular Expression Complexity

Signed-off-by: Peter Somogyvari <[email protected]>

feat(cc-tx-viz): add methods to fabric connector

Signed-off-by: Iulia Mihaiu <[email protected]>

feat(cc-tx-viz): add metric model

Signed-off-by: Iulia Mihaiu <[email protected]>

xfix(cc-tx-viz): error implementation of IsVisualizable

feat(cc-tx-viz): add methods to fabric and besu; update models

Signed-off-by: Iulia Mihaiu <[email protected]>

feat(cc-tx-viz): update cross chain events log for fabric transactions

Signed-off-by: Iulia Mihaiu <[email protected]>

feat(cctxviz): fix fabric prometheus

feat(cctxviz): fix besu prometheus

feat(cctxviz): fix fabric test

feat(cctxviz): update tx receipt

feat(cctxviz): add test script

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add test receipts

feat(cctxviz): refactor CrossChainEventLog

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): refactor tests

feat(cc-tx-viz): update model, add tests

chore(cc-tx-viz): update dependencies

Signed-off-by: Rafael Belchior <[email protected]>

feat(cc-tx-viz): persist logs

Signed-off-by: Rafael Belchior <[email protected]>

test(cc-tx-viz): update test

Signed-off-by: Rafael Belchior <[email protected]>

feat(cc-tx-viz): update strinfify to csv

Signed-off-by: Rafael Belchior <[email protected]>

chore(cc-tx-viz): add csv folder

Signed-off-by: Rafael Belchior <[email protected]>

chore(cc-tx-viz): move csv folder

Signed-off-by: Rafael Belchior <[email protected]>

feat(cc-tx-viz): add test

feat(cctxviz): add csv file, process mining script

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): proof of concept v1

Signed-off-by: Rafael Belchior <[email protected]>

chore(cctxviz): update csv

Signed-off-by: Rafael Belchior <[email protected]>

docs(cctxviz): update readme

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update tests

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add carbon footprint

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update tests

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update model

Signed-off-by: Rafael Belchior <[email protected]>

feat(fabric-connector): update default queue

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): fabric test

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add parameters to fabric calls

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): support besu

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update metrics

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxvi): update fabric and besu tests

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): besu and fabric

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): support call transactions on fabric

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add aggregation basis

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): aggregate cctx

Signed-off-by: Rafael Belchior <[email protected]>

chore: update yarn and remove package.lock

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): containerize draft

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add revenue

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update model playbook

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add dummy use case playbook

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): update persist test and dummy uc

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add timer

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): workload test

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add more timers

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): add timer to map on connectors

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update testing framework

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): vis plot wip

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): evaluate baseline

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update plots

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): add test files

Signed-off-by: Rafael Belchior <[email protected]>

feat(cctxviz): update testing framework

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): time-efficient polling receipts

Signed-off-by: Rafael Belchior <[email protected]>

chore(cctxviz): add test chaincode

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): update test

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): use case v1

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): update test framework

Signed-off-by: Rafael Belchior <[email protected]>

build(cc-tx-visualization): added missing fabric-contract-api

This will fix one compilation error.

Signed-off-by: Peter Somogyvari <[email protected]>

chore(deps): updated yarn.lock after rebase onto upstream/main

Signed-off-by: Peter Somogyvari <[email protected]>

build(cc-tx-visualization): deleted redundant test fixtures

Also added the test-tooling package as a dev dependency which
should fix a few of the compiler errors we were getting on account
of missing dependencies.

Signed-off-by: Peter Somogyvari <[email protected]>

fix: resolved build errors for cctxviz branch

- test-tooling package changed that needs to be reverted before commmit to main branch
- added import statement for test data generationt

xfix(cc-tx-viz): error implementation of IsVisualizable

chore: update yarn and remove package.lock

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): testing framework

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): test framework

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): update framework

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): add graphs

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): add test

Signed-off-by: Rafael Belchior <[email protected]>

fix(cctxviz): fix bug

Signed-off-by: Rafael Belchior <[email protected]>

test(cctxviz): add process conformance

Signed-off-by: Rafael Belchior <[email protected]>
  • Loading branch information
maramih authored and RafaelAPB committed Jun 4, 2024
1 parent ac5a55a commit c0d7c9e
Show file tree
Hide file tree
Showing 43 changed files with 5,324 additions and 35 deletions.
4 changes: 4 additions & 0 deletions .cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@
"cccs",
"ccep",
"ccid",
"cctx",
"cctxviz",
"celo",
"cids",
"clsx",
Expand Down Expand Up @@ -154,6 +156,7 @@
"qscc",
"recoverupdateackmessage",
"rogpeppe",
"rabbitmq",
"RUSTC",
"Rwset",
"satp",
Expand Down Expand Up @@ -184,6 +187,7 @@
"unixfs",
"Unmarshal",
"uuidv",
"Visualizable",
"vscc",
"vuln",
"wasm",
Expand Down
8 changes: 8 additions & 0 deletions jest.config.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 16 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,22 @@
"build:dev:frontend": "lerna run build:dev:frontend --scope='@hyperledger/cactus-example-*-frontend' --scope='@hyperledger/cacti-ledger-browser'",
"build:dev:common": "lerna exec --stream --scope '*/*common' -- 'del-cli dist/** && tsc --project ./tsconfig.json && webpack --env=dev --target=node --config ../../webpack.config.js'",
"build:dev:backend:postbuild": "lerna run build:dev:backend:postbuild",
"test:cmd-api-server": "tap --ts --timeout=600 \"packages/cactus-*cmd-api-server/src/test/typescript/{unit,integration}/\"",
"test:plugin-ledger-connector-besu": "tap --ts --jobs=1 --timeout=60 \"packages/cactus-*-besu/src/test/typescript/{unit,integration}/\"",
"test:plugin-htlc-besu-erc20": "tap --jobs=1 --timeout=600 \"packages/*htlc-eth-besu-erc20/src/test/typescript/{unit,integration}/\"",
"test:plugin": "tap --jobs=1 --timeout=600 \"packages/*test-plugin-htlc-eth-besu/src/test/typescript/{unit,integration}/\"",
"test:plugin-ledger-connector-quorum": "tap --ts --jobs=1 --timeout=60 \"packages/cactus-*-quorum/src/test/typescript/{unit,integration}/\"",
"test:cctxviz": "tap --jobs=1 --timeout=600 \"packages/cactus-plugin-cc-tx-visualization/src/test/typescript/{unit,integration}/\"",

"test:plugin-ledger-connector-iroha": "tap --ts --jobs=1 --timeout=600 \"packages/cactus-*-iroha/src/test/typescript/{unit,integration}/\"",
"test:plugin-htlc-besu": "tap --jobs=1 --timeout=600 \"packages/*htlc-eth-besu/src/test/typescript/{integration}/\"",
"build:dev:plugin-consortium-manual": "lerna exec --stream --scope '*/*manual-consortium' -- 'del-cli dist/** && tsc --project ./tsconfig.json && webpack --env=dev --target=node --config ../../webpack.config.js'",
"build:dev:plugin-cc-tx-visualization": "lerna exec --stream --scope '*/*cc-tx-visualization' -- 'del-cli dist/** && tsc --project ./tsconfig.json && webpack --env=dev --target=node --config ../../webpack.config.js'",
"build:dev:example-supply-chain-backend": "lerna exec --stream --scope '*/*example-supply-chain-b*' -- 'del-cli dist/** && tsc --project ./tsconfig.json && webpack --display-modules --env=dev --target=node --config ../../webpack.config.js'",
"build:dev:example-carbon-accounting-backend": "lerna exec --stream --scope '*/*carbon-accounting-b*' -- 'del-cli dist/** && tsc --project ./tsconfig.json && webpack --display-modules --env=dev --target=node --config ../../webpack.config.js' && cp -r examples/cactus-example-carbon-accounting-backend/src/utility-emissions-channel/ examples/cactus-example-carbon-accounting-backend/dist/lib/",
"build:dev:sdk": "lerna exec --stream --scope '*/*sdk' -- 'del-cli dist/** && tsc --project ./tsconfig.json && webpack --env=dev --target=node --config ../../webpack.config.js'",
"build:dev:plugin-ledger-connector-corda": "lerna exec --stream --scope '*/*connector-corda' -- 'del-cli dist/** && npm run tsc && webpack --env=dev --target=node --config ../../webpack.config.js'",
"test:plugin-ledger-connector-corda": "tap --ts --jobs=1 --timeout=600 \"packages/cactus-*-corda/src/test/typescript/{unit,integration}/\"",
"webpack": "lerna run webpack:dev",
"webpack:dev:web": "lerna run webpack:dev:web",
"webpack:dev:node": "lerna run webpack:dev:node",
Expand Down
4 changes: 4 additions & 0 deletions packages/cactus-plugin-cc-tx-visualization/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
cactus-openapi-spec-plugin-consortium-manual.json
src/main/typescript/generated/openapi/typescript-axios/.npmignore
src/test/csv
src/test/test-results/*.out
3 changes: 3 additions & 0 deletions packages/cactus-plugin-cc-tx-visualization/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# `@hyperledger/cactus-plugin-cctxviz`

The proposed plugin allows generating process models from arbitrary cross-chain use cases. Currently supports Fabric and Besu. More documentation to come soon.
92 changes: 92 additions & 0 deletions packages/cactus-plugin-cc-tx-visualization/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
{
"name": "@hyperledger/cactus-plugin-cc-tx-visualization",
"version": "1.0.0",
"description": "A web service plugin that provides management capabilities on cross-chain transactions visualization.",
"main": "dist/lib/main/typescript/index.js",
"mainMinified": "dist/cactus-plugin-cc-tx-visualization.node.umd.min.js",
"browser": "dist/cactus-plugin-cc-tx-visualization.web.umd.js",
"browserMinified": "dist/cactus-plugin-cc-tx-visualization.web.umd.min.js",
"module": "dist/lib/main/typescript/index.js",
"types": "dist/types/main/typescript/index.d.ts",
"files": [
"dist/*"
],
"scripts": {
"watch": "npm-watch",
"webpack": "npm-run-all webpack:dev webpack:prod",
"webpack:dev": "npm-run-all webpack:dev:node webpack:dev:web",
"webpack:dev:web": "webpack --env=dev --target=web --config ../../webpack.config.js",
"webpack:dev:node": "webpack --env=dev --target=node --config ../../webpack.config.js",
"webpack:prod": "npm-run-all webpack:prod:node webpack:prod:web",
"webpack:prod:web": "webpack --env=prod --target=web --config ../../webpack.config.js",
"webpack:prod:node": "webpack --env=prod --target=node --config ../../webpack.config.js"
},
"publishConfig": {
"access": "public"
},
"engines": {
"node": ">=10",
"npm": ">=6"
},
"repository": {
"type": "git",
"url": "git+https://github.com/hyperledger/cactus.git"
},
"keywords": [
"Hyperledger",
"Cactus",
"Integration",
"Blockchain",
"Distributed Ledger Technology"
],
"author": {
"name": "Hyperledger Cactus Contributors",
"email": "[email protected]",
"url": "https://www.hyperledger.org/use/cactus"
},
"contributors": [
{
"name": "Please add yourself to the list of contributors",
"email": "[email protected]",
"url": "https://example.com"
},
{
"name": "Iulia Mihaiu"
},
{
"name": "Sabrina Scuri"
},
{
"name": "Rafael Belchior"
}
],
"license": "Apache-2.0",
"bugs": {
"url": "https://github.com/hyperledger/cactus/issues"
},
"homepage": "https://github.com/hyperledger/cactus#readme",
"dependencies": {
"@hyperledger/cactus-common": "1.0.0",
"@hyperledger/cactus-core": "1.0.0",
"@hyperledger/cactus-core-api": "1.0.0",
"@hyperledger/cactus-plugin-ledger-connector-fabric": "1.0.0",
"@hyperledger/cactus-plugin-ledger-connector-besu": "1.0.0",
"amqp-ts": "1.8.0",
"axios": "0.21.1",
"body-parser": "1.19.0",
"csv-stringify": "6.0.5",
"express": "4.17.1",
"fabric-contract-api": "2.2.3",
"jose": "1.28.1",
"json-stable-stringify": "1.0.1",
"prom-client": "13.0.0",
"typescript-optional": "2.0.1",
"uuid": "8.3.2"
},
"devDependencies": {
"@hyperledger/cactus-test-tooling": "1.0.0",
"@types/express": "4.17.8",
"@types/json-stable-stringify": "1.0.32",
"@types/uuid": "8.3.0"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from "./public-api";
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export {};
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import { LedgerType } from "@hyperledger/cactus-core-api";

export function calculateGasPriceBesu(gasUsed: number): number {
if (!gasUsed) {
return 0;
}
return getGasPrice() * gasUsed;
}

// The conversion rate gwei-dollar can be dynamic
export function gweiToDollar(gwei: number): number | string {
if (!gwei) {
return 0;
}
return (gwei * 0.00000255).toFixed(2);
}

// This value can be retrieved dynamically; 1 gas = 30 gwei
export function getGasPrice(): number {
return 30;
}
// price per gwei
export function gasToDollar(gwei: number): number {
return gwei * 0.005899;
}

export function calculateCarbonFootPrintFabric(
peers: string[] | undefined,
): number {
if (!peers) {
return 0;
}
return peers.length * CarbonFootPrintConstants(LedgerType.Fabric2);
}
export function calculateCarbonFootPrintBesu(): number {
return CarbonFootPrintConstants(LedgerType.Besu2X);
}

export const CarbonFootPrintConstants = (ledger: LedgerType): number => {
switch (ledger) {
case LedgerType.Besu2X:
return 0.00018;

case LedgerType.Besu1X:
return 0.00018;

case LedgerType.Fabric2:
return 0.00018;

case LedgerType.Fabric14X:
return 0.00018;
default:
return 0;
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
export type CrossChainEvent = {
caseID: string;
receiptID: string;
timestamp: Date;
blockchainID: string;
invocationType: string;
methodName: string;
parameters: string[];
identity: string;
cost?: number | string;
carbonFootprint?: number | string;
latency?: number;
revenue?: number;
};

export interface ICrossChainEventLog {
name: string;
}

export class CrossChainEventLog {
private crossChainEvents: CrossChainEvent[] = [];
private creationDate: Date;
private lastUpdateDate: Date;
public readonly logName: string;
//TODO: add a pause boolean?

constructor(options: ICrossChainEventLog) {
this.creationDate = new Date();
this.lastUpdateDate = new Date();
this.logName = options.name;
}

get logEntries(): CrossChainEvent[] {
return this.crossChainEvents;
}

public numberEvents(): number {
return this.crossChainEvents.length;
}
public getCreationDate(): Date {
return this.creationDate;
}

public getLastUpdateDate(): Date {
return this.lastUpdateDate;
}

public purgeLogs(): void {
this.crossChainEvents = [];
}

public addCrossChainEvent(event: CrossChainEvent): void {
this.crossChainEvents.push(event);
this.lastUpdateDate = new Date();
}

public getCrossChainLogAttributes(): string[] {
return [
"caseID",
"receiptID",
"timestamp",
"blockchainID",
"invocationType",
"methodName",
"parameters",
"identity",
"cost",
"carbonFootprint",
"latency",
"revenue",
];
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import { v4 as uuidv4 } from "uuid";

export class CrossChainModel {
private modelType: CrossChainModelType | undefined;
private crossChainTransactions:
| Map<string, CrossChainTransactionSchema>
| undefined;
private models = new Map<CrossChainModelType, string>();
private id: string;
private lastAggregationDate: Date;

constructor() {
this.id = uuidv4();
this.crossChainTransactions = new Map<
string,
CrossChainTransactionSchema
>();
this.lastAggregationDate = new Date();
}
get lastAggregation(): Date {
return this.lastAggregationDate;
}

public setLastAggregationDate(date: Date): void {
this.lastAggregationDate = date;
}

public saveModel(type: CrossChainModelType, model: string): void {
this.models.set(type, model);
}

public getModel(type: CrossChainModelType): string | undefined {
if (this.models.has(type)) {
return this.models.get(type);
}
}

public getOneCCTx(txKey: string): CrossChainTransactionSchema | undefined {
if (this.crossChainTransactions && this.crossChainTransactions.has(txKey)) {
return this.crossChainTransactions.get(txKey);
}
}

public getCCTxs(): Map<string, CrossChainTransactionSchema> | undefined {
if (this.crossChainTransactions) {
return this.crossChainTransactions;
}
}

public setCCTxs(
key: string,
mapDefintion: CrossChainTransactionSchema,
): void {
this.crossChainTransactions?.set(key, mapDefintion);
}
}

export enum CrossChainModelType {
HeuristicMiner,
ProcessTree,
DirectFollowGraph,
}

export type CrossChainTransactionSchema = {
ccTxID: string;
// the receipt ids of each cross chain event
processedCrossChainEvents: string[];
latency: number;
carbonFootprint: number | undefined;
cost: number | undefined;
throughput: number;
latestUpdate: Date;
revenue: number;
};
Loading

0 comments on commit c0d7c9e

Please sign in to comment.