3D pricing metrics

[cloudspores]

This PR exposes the following metrics

1 Node's perception on the latest extraData contents (variable cost, fixed cost, eth gas price) as gauges under the PRICING_CONF category
1 Node's perception on Profitability levels of TxPool's contents (lo, hi, avg of transaction.priorityFeePerGas / TransactionProfitabilityCalculator.profitablePriorityFeePerGas(transaction)) as histogram under the TX_POOL_PROFITABILITY categoty
1 Same metrics as above, but in context of last sealed block (this is relevant only for Sequencer) as histogram under the SEQUENCER_PROFITABILITY category

Each metric category can be enabled indipendently, using the standard metrics-category option.

For histograms, it is possible to configure the buckets using the new conf option plugin-linea-profitability-metrics-buckets, that by default is set to 0.9, 1.0, 1.2, 2, 5, 10, 100, 1000

names of the new exported metrics

# HELP linea_tx_pool_profitability_ratio_min Lowest transaction pool profitability ratio
# TYPE linea_tx_pool_profitability_ratio_min gauge
linea_tx_pool_profitability_ratio_min +Inf
# HELP linea_pricing_conf_values Profitability configuration values at runtime
# TYPE linea_pricing_conf_values gauge
linea_pricing_conf_values{field="fixed_cost_wei",} 3.0E7
linea_pricing_conf_values{field="eth_gas_price_wei",} 1.53525E8
linea_pricing_conf_values{field="variable_cost_wei",} 8.226788E9
# HELP linea_sequencer_profitability_ratio_max Highest sequencer profitability ratio
# TYPE linea_sequencer_profitability_ratio_max gauge
linea_sequencer_profitability_ratio_max{phase="pre_processing",} 16572.31055926643
linea_sequencer_profitability_ratio_max{phase="post_processing",} 0.0
# HELP linea_sequencer_profitability_ratio Sequencer profitability ratio buckets
# TYPE linea_sequencer_profitability_ratio histogram
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="0.9",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="1.0",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="1.2",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="2.0",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="5.0",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="10.0",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="100.0",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="1000.0",} 0.0
linea_sequencer_profitability_ratio_bucket{phase="pre_processing",le="+Inf",} 1.0
linea_sequencer_profitability_ratio_count{phase="pre_processing",} 1.0
linea_sequencer_profitability_ratio_sum{phase="pre_processing",} 16572.31055926643
# HELP linea_sequencer_profitability_ratio_min Lowest sequencer profitability ratio
# TYPE linea_sequencer_profitability_ratio_min gauge
linea_sequencer_profitability_ratio_min{phase="pre_processing",} 16572.31055926643
linea_sequencer_profitability_ratio_min{phase="post_processing",} +Inf
# HELP linea_tx_pool_profitability_ratio Transaction pool profitability ratio buckets
# TYPE linea_tx_pool_profitability_ratio histogram
linea_tx_pool_profitability_ratio_bucket{le="0.9",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="1.0",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="1.2",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="2.0",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="5.0",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="10.0",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="100.0",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="1000.0",} 0.0
linea_tx_pool_profitability_ratio_bucket{le="+Inf",} 0.0
linea_tx_pool_profitability_ratio_count 0.0
linea_tx_pool_profitability_ratio_sum 0.0
# HELP linea_tx_pool_profitability_ratio_max Highest transaction pool profitability ratio
# TYPE linea_tx_pool_profitability_ratio_max gauge
linea_tx_pool_profitability_ratio_max 0.0
# HELP linea_sequencer_profitability_ratio_created Sequencer profitability ratio buckets
# TYPE linea_sequencer_profitability_ratio_created gauge
linea_sequencer_profitability_ratio_created{phase="pre_processing",} 1.730398405758E9
# HELP linea_tx_pool_profitability_ratio_created Transaction pool profitability ratio buckets
# TYPE linea_tx_pool_profitability_ratio_created gauge
linea_tx_pool_profitability_ratio_created 1.730398404348E9

fixes https://github.com/Consensys/protocol-misc/issues/953

[garyschulte]

Non-blockion suggestions. LGTM

[garyschulte]

This description could be more informative. They are buckets of what? profit for the sequencer in wei, gwei, or ? is it the tx fee revenue, or the net profit between posting to L1 vs fee on linea, etc.

I can find it in the code, but I think the option description could clear it up

[fab-10]

updated, to make it clearer

[garyschulte]

are we guaranteed that profitablePriorityFeePerGas is never zero.

[fab-10]

it could be zero only the pricing conf for variable and fixed cost is zero, that seems unreasonable, and according to the Java rules that operation will result in Double.POSITIVE_INFINITY

[Filter94]

Why do we need it as a separate method? Do you think that O(N) additional comparisons in track will impact the performance?

[fab-10]

I will review if it is possible to perform all the updates in a single method

[Filter94]

Similarly to how it's done for ProfitableTransactionSelector maybe instead of evaluating each transaction from txpool directly, maybe it would be more efficient and less invasive if we capture txpool profitability metrics in ProfitabilityValidator?

[fab-10]

we thought about that, but since a tx could stay in the pool for a long time, the value calculated at ProfitabilityValidator time could be different from the current value, due to the changes to the market conditions and the extra data pricing conf, so basically it will be a different metric, that represents the values at validation time, instead of the current status in the txpool.
If you think that collecting metrics in the ProfitabilityValidator is good enough, then yes it will be more efficient.

[Filter94]

I see what you mean. Let's leave it the way you did and if it causes any performance issues, we can always disable it

[fab-10]

since the txpool should have a similar content on every node, it is possible to enable this metric only on dedicated or not critical nodes.