Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smart Contract listing endpoints and GraphQl duration metrics #97

Merged
82 commits merged into from
Sep 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
d5e8afa
Made entities and import flow
Aug 17, 2023
cdf8265
Added test to aggregate which validates events are stored
Aug 18, 2023
bc505e9
Merge branch 'main' into cbw-1248/implementation-smart-contract-aggre…
Aug 18, 2023
fbf296d
Added test for other read cases
Aug 18, 2023
aa922e0
Added smart contract entity
Aug 18, 2023
81b34ab
Added Smart Contract Aggregate to Startup
Aug 18, 2023
58156e1
Added import job
Aug 21, 2023
dcb5256
Finished repository test
Aug 21, 2023
ca0522a
Fixed aggregate test
Aug 21, 2023
b73da60
Added data source
Aug 21, 2023
91b0eaf
Added awaits on jobs on node import
Aug 21, 2023
fc47cb3
Added contract version to contract initialize and update events
Aug 22, 2023
50f6167
Added tests which validates uniqueness contrains
Aug 22, 2023
293329a
Remove todos and added simple resilience in import loop
Aug 22, 2023
1cfb11c
Fixed dependency injection issue
Aug 22, 2023
aa91b17
Fix issues when testing job
Aug 22, 2023
d575289
Small updates
Aug 22, 2023
f63bf17
Updated changelog
Aug 22, 2023
ef4550d
Moved classes to own files
Aug 22, 2023
eae5cb1
Added logic to Smart Contract Repository which can fetch batch
Aug 24, 2023
bb162ca
Made batch import instead of processing single rows
Aug 24, 2023
952e48b
Clean up unused code
Aug 24, 2023
4a40b00
Fix limit on smart contract database job
Aug 24, 2023
e708617
Remove unnecessary batch
Aug 24, 2023
40e474d
Added duration- and height metrics
Aug 24, 2023
d565919
Added health states to application
Aug 25, 2023
b3cc30b
Fixed health output
Aug 25, 2023
5411768
Adding temp CI
Aug 25, 2023
ea18072
Updating change log
Aug 25, 2023
75f2851
Remove some debugging and added nugets
Aug 25, 2023
54ab1c6
Fix test
Aug 25, 2023
b71562d
Merge branch 'main' into cbw-1248/implementation-smart-contract-aggre…
Aug 25, 2023
2c26643
Merge branch 'cbw-1248/implementation-smart-contract-aggregate' into …
Aug 25, 2023
915777e
Refactor
Aug 28, 2023
9857710
Merge branch 'cbw-1248/implementation-smart-contract-aggregate' into …
Aug 28, 2023
ce21ac8
Clean up after merge
Aug 28, 2023
707c401
Added smart contract paging query
Aug 31, 2023
7e5f664
Added metrics to graphql endpoints
Sep 1, 2023
2171a83
Updated changelog
Sep 1, 2023
09d7102
Added contract address to Smart Contract query
Sep 1, 2023
ee4acf7
Updated naming from Smart Contract to Contract
Sep 1, 2023
9ba8c24
Renamed test and tables
Sep 1, 2023
68db31f
Merge branch 'cbw-1248/implementation-smart-contract-aggregate' into …
Sep 1, 2023
bc4440a
Fix naming conflicts
Sep 1, 2023
0faea97
Merge branch 'cbw-1249/add-observability' into cbw-1261/sc-listing-en…
Sep 1, 2023
c595e17
Resolve conflicts after merge
Sep 1, 2023
aa2f083
Updated with add and remove link entities
Sep 1, 2023
f143293
Fixed off by one error
Sep 1, 2023
4ec857d
Merge branch 'cbw-1248/implementation-smart-contract-aggregate' into …
Sep 4, 2023
8c96ed8
Merge branch 'cbw-1249/add-observability' into cbw-1261/sc-listing-en…
Sep 4, 2023
f1f8c70
Update Program.cs
Sep 4, 2023
c33b7e6
Merge branch 'cbw-1249/add-observability' into cbw-1261/sc-listing-en…
Sep 4, 2023
8441f6c
Using activity and better exception handling in diagnostic handler
Sep 4, 2023
5197ede
Enable rewind of request body
Sep 4, 2023
ffaa221
Added tracing to logs
Sep 4, 2023
c42ada0
Updated log formats
Sep 4, 2023
0944d3d
remove extra from console
Sep 4, 2023
bff53f4
Corrected naming
Sep 5, 2023
5bf501e
Made repository for contract jobs
Sep 5, 2023
5843bdc
Migrated to options for feature flags
Sep 5, 2023
37b979e
Moved interfaces to same file as classes where only one implementatio…
Sep 5, 2023
e9bc1c1
Changed to date time offset
Sep 5, 2023
152c486
Added warnings not to change job identifier
Sep 5, 2023
3c682d0
Update range function with comments
Sep 5, 2023
6769b70
Follow standards from EF regarding CS8618 compiler warning for C# 10
Sep 5, 2023
3e7126e
Refactored Contract Aggregate Node Import job to be more readable
Sep 5, 2023
8048a49
Merge branch 'cbw-1248/implementation-smart-contract-aggregate' into …
Sep 5, 2023
a30fb7c
Resolve comments after merge
Sep 5, 2023
9f028f2
Merge branch 'main' into cbw-1249/add-observability
Sep 5, 2023
05dfeb9
Resolve issues after merge
Sep 5, 2023
0c43cc3
Merge branch 'cbw-1249/add-observability' into cbw-1261/sc-listing-en…
Sep 5, 2023
77ec045
Resolve issues after merge
Sep 5, 2023
e26cb54
Resolve comments
Sep 5, 2023
de8e391
Minor refactoring after final review
Sep 5, 2023
d47a38c
Merge branch 'cbw-1249/add-observability' into cbw-1261/sc-listing-en…
Sep 5, 2023
36467a9
Merge branch 'main' into cbw-1261/sc-listing-endpoints
Sep 5, 2023
f8fa896
Updates after merge
Sep 5, 2023
f483588
Fix exceptions handling in diagnostics
Sep 8, 2023
0b962a5
Fix missing events when money CCD was transferred between account
Sep 8, 2023
282839d
Remove unused exception
Sep 8, 2023
3492061
Added block slot time to all events
Sep 8, 2023
4838845
Updated graphql schema
Sep 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions backend/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -447,3 +447,5 @@ $RECYCLE.BIN/
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json

/Logs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
using System.Threading;
using System.Threading.Tasks;
using Application.Aggregates.Contract.Jobs;
using Application.Observability;
using Application.Configurations;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Options;
Expand Down Expand Up @@ -33,6 +34,8 @@ IOptions<FeatureFlagOptions> featureFlagsOptions

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
using var _ = TraceContext.StartActivity(nameof(ContractJobsBackgroundService));

if (!_featureFlags.ConcordiumNodeImportEnabled)
{
_logger.Information("Import data from Concordium node is disabled. This controller will not run!");
Expand Down Expand Up @@ -73,4 +76,4 @@ private async Task RunJob(IContractJob job, CancellationToken token)
throw;
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using Application.Aggregates.Contract.Jobs;
using Application.Aggregates.Contract.Observability;
using Application.Api.GraphQL.EfCore;
using Application.Observability;
using Application.Configurations;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Hosting;
Expand Down Expand Up @@ -46,6 +47,8 @@ public ContractNodeImportBackgroundService(

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
using var _ = TraceContext.StartActivity(nameof(ContractNodeImportBackgroundService));

if (!_featureFlags.ConcordiumNodeImportEnabled)
{
_logger.Information("Import data from Concordium node is disabled. This controller will not run!");
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
using Application.Aggregates.Contract.Entities;
using Application.Api.GraphQL.EfCore.Converters.EfCore;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
Expand All @@ -11,9 +12,6 @@ public void Configure(EntityTypeBuilder<Entities.Contract> builder)
builder.ToTable("graphql_contracts");
builder.HasKey(x => new
{
x.BlockHeight,
x.TransactionIndex,
x.EventIndex,
x.ContractAddressIndex,
x.ContractAddressSubIndex
});
Expand All @@ -34,7 +32,14 @@ public void Configure(EntityTypeBuilder<Entities.Contract> builder)
.HasConversion<AccountAddressConverter>();
builder.Property(x => x.Source)
.HasColumnName("source");
builder.Property(x => x.BlockSlotTime)
.HasColumnName("block_slot_time");
builder.Property(x => x.CreatedAt)
.HasColumnName("created_at");
.HasColumnName("created_at");

builder
.HasMany<ContractEvent>(sm => sm.ContractEvents)
.WithOne()
.HasForeignKey(sme => new { sme.ContractAddressIndex, sme.ContractAddressSubIndex });
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@ public void Configure(EntityTypeBuilder<ContractEvent> builder)
.HasConversion<TransactionResultEventToJsonConverter>();
builder.Property(x => x.Source)
.HasColumnName("source");
builder.Property(x => x.BlockSlotTime)
.HasColumnName("block_slot_time");
builder.Property(x => x.CreatedAt)
.HasColumnName("created_at");
.HasColumnName("created_at");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ public void Configure(EntityTypeBuilder<ModuleReferenceContractLinkEvent> builde
x.EventIndex,
x.ModuleReference,
x.ContractAddressIndex,
x.ContractAddressSubIndex
x.ContractAddressSubIndex,
x.LinkAction
});
builder.Property(x => x.BlockHeight)
.HasColumnName("block_height");
Expand All @@ -36,6 +37,8 @@ public void Configure(EntityTypeBuilder<ModuleReferenceContractLinkEvent> builde
.HasColumnName("source");
builder.Property(x => x.LinkAction)
.HasColumnName("link_action");
builder.Property(x => x.BlockSlotTime)
.HasColumnName("block_slot_time");
builder.Property(x => x.CreatedAt)
.HasColumnName("created_at");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,6 @@ public void Configure(EntityTypeBuilder<ModuleReferenceEvent> builder)
builder.ToTable("graphql_module_reference_events");
builder.HasKey(x => new
{
x.BlockHeight,
x.TransactionIndex,
x.EventIndex,
x.ModuleReference
});
builder.Property(x => x.BlockHeight)
Expand All @@ -28,6 +25,8 @@ public void Configure(EntityTypeBuilder<ModuleReferenceEvent> builder)
.HasColumnName("module_reference");
builder.Property(x => x.Source)
.HasColumnName("source");
builder.Property(x => x.BlockSlotTime)
.HasColumnName("block_slot_time");
builder.Property(x => x.CreatedAt)
.HasColumnName("created_at");
}
Expand Down
97 changes: 73 additions & 24 deletions backend/Application/Aggregates/Contract/ContractAggregate.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
using Application.Aggregates.Contract.Observability;
using Application.Aggregates.Contract.Types;
using Application.Api.GraphQL.Transactions;
using Application.Observability;
using Concordium.Sdk.Types;
using AccountAddress = Application.Api.GraphQL.Accounts.AccountAddress;
using ContractAddress = Application.Api.GraphQL.ContractAddress;
Expand All @@ -19,6 +20,8 @@ internal sealed class ContractAggregate
private readonly IContractRepositoryFactory _repositoryFactory;
private readonly ContractAggregateOptions _options;
private readonly ILogger _logger;
private const string NodeImportJobActivity = "NodeImportJobActivity";
private const string NodeImportJobLoopActivity = "NodeImportJobLoopActivity";

public ContractAggregate(
IContractRepositoryFactory repositoryFactory,
Expand All @@ -32,6 +35,8 @@ ContractAggregateOptions options

internal async Task NodeImportJob(IContractNodeClient client, CancellationToken token = default)
{
using var _ = TraceContext.StartActivity(NodeImportJobActivity);

var retryCount = 0;
while (!token.IsCancellationRequested)
{
Expand Down Expand Up @@ -102,6 +107,7 @@ await StoreEvent(
transactionResultEvent,
AccountAddress.From(details.Sender),
blockInfo.BlockHeight,
blockInfo.BlockSlotTime,
transactionHash,
blockItemSummary.Index,
eventIndex
Expand Down Expand Up @@ -144,6 +150,7 @@ internal static async Task StoreEvent(
TransactionResultEvent transactionResultEvent,
AccountAddress sender,
ulong blockHeight,
DateTimeOffset blockSlotTime,
string transactionHash,
ulong transactionIndex,
uint eventIndex
Expand All @@ -159,7 +166,8 @@ await repository.AddAsync(new Entities.Contract(
eventIndex,
contractInitialized.ContractAddress,
sender,
source
source,
blockSlotTime
));
await repository
.AddAsync(new ContractEvent(
Expand All @@ -169,7 +177,8 @@ await repository
eventIndex,
contractInitialized.ContractAddress,
contractInitialized,
source
source,
blockSlotTime
));
await repository
.AddAsync(new ModuleReferenceContractLinkEvent(
Expand All @@ -180,7 +189,8 @@ await repository
contractInitialized.ModuleRef,
contractInitialized.ContractAddress,
source,
ModuleReferenceContractLinkEvent.ModuleReferenceContractLinkAction.Added
ModuleReferenceContractLinkEvent.ModuleReferenceContractLinkAction.Added,
blockSlotTime
));
break;
case ContractInterrupted contractInterrupted:
Expand All @@ -192,7 +202,8 @@ await repository
eventIndex,
contractInterrupted.ContractAddress,
contractInterrupted,
source
source,
blockSlotTime
));
break;
case ContractResumed contractResumed:
Expand All @@ -204,7 +215,8 @@ await repository
eventIndex,
contractResumed.ContractAddress,
contractResumed,
source
source,
blockSlotTime
));
break;
case ContractUpdated contractUpdated:
Expand All @@ -216,8 +228,27 @@ await repository
eventIndex,
contractUpdated.ContractAddress,
contractUpdated,
source
));
source,
blockSlotTime
));
if (contractUpdated.Instigator is ContractAddress contractInstigator && contractUpdated.Amount != 0)
{
await repository
.AddAsync(new ContractEvent(
blockHeight,
transactionHash,
transactionIndex,
eventIndex,
contractInstigator,
new Transferred(
contractUpdated.Amount,
contractInstigator,
contractUpdated.ContractAddress
),
source,
blockSlotTime
));
}
break;
case ContractUpgraded contractUpgraded:
await repository
Expand All @@ -228,7 +259,8 @@ await repository
eventIndex,
contractUpgraded.ContractAddress,
contractUpgraded,
source
source,
blockSlotTime
));
await repository
.AddAsync(new ModuleReferenceContractLinkEvent(
Expand All @@ -239,7 +271,8 @@ await repository
contractUpgraded.To,
contractUpgraded.ContractAddress,
source,
ModuleReferenceContractLinkEvent.ModuleReferenceContractLinkAction.Added
ModuleReferenceContractLinkEvent.ModuleReferenceContractLinkAction.Added,
blockSlotTime
));
await repository
.AddAsync(new ModuleReferenceContractLinkEvent(
Expand All @@ -250,25 +283,39 @@ await repository
contractUpgraded.From,
contractUpgraded.ContractAddress,
source,
ModuleReferenceContractLinkEvent.ModuleReferenceContractLinkAction.Removed
ModuleReferenceContractLinkEvent.ModuleReferenceContractLinkAction.Removed,
blockSlotTime
));
break;
case Transferred transferred:
if (transferred.From is not ContractAddress contractAddress ||
transferred.To is not AccountAddress)
if (transferred.From is ContractAddress contractAddressFrom)
{
break;
await repository
.AddAsync(new ContractEvent(
blockHeight,
transactionHash,
transactionIndex,
eventIndex,
contractAddressFrom,
transferred,
source,
blockSlotTime
));
}
if (transferred.To is ContractAddress contractAddressTo)
{
await repository
.AddAsync(new ContractEvent(
blockHeight,
transactionHash,
transactionIndex,
eventIndex,
contractAddressTo,
transferred,
source,
blockSlotTime
));
}
await repository
.AddAsync(new ContractEvent(
blockHeight,
transactionHash,
transactionIndex,
eventIndex,
contractAddress,
transferred,
source
));
break;
case ContractModuleDeployed contractModuleDeployed:
await repository
Expand All @@ -278,7 +325,8 @@ await repository
transactionIndex,
eventIndex,
contractModuleDeployed.ModuleRef,
source
source,
blockSlotTime
));
break;
}
Expand All @@ -294,6 +342,7 @@ private async Task NodeImportRange(IContractNodeClient client, ulong fromBlockHe
{
for (var height = fromBlockHeight; height <= toBlockHeight; height++)
{
using var __ = TraceContext.StartActivity(NodeImportJobLoopActivity);
using var durationMetric = new ContractMetrics.DurationMetric(ImportSource.NodeImport);
try
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ public async Task<IList<TransactionResultEventDto>> FromBlockHeightRangeGetContr
const string sql = @"
SELECT
gb.block_height as BlockHeight,
gb.block_slot_time as BlockSlotTime,
gt.transaction_type as TransactionType,
gt.sender as TransactionSender,
gt.transaction_hash as TransactionHash,
Expand All @@ -81,7 +82,7 @@ graphql_transaction_events te
graphql_blocks gb ON gt.block_id = gb.id
WHERE
gb.block_height >= @FromHeight AND gb.block_height <= @ToHeight
AND te.event->>'tag' IN ('1', '16', '18', '17', '34', '35', '36');
AND te.event->>'tag' IN ('1', '16', '17', '18', '34', '35', '36');
";
var queryAsync = await _context.Database.GetDbConnection()
.QueryAsync<TransactionResultEventDto>(sql, new { FromHeight = (long)heightFrom, ToHeight = (long)heightTo });
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ namespace Application.Aggregates.Contract.Dto;
public class TransactionResultEventDto
{
public int BlockHeight { get; init; }
public DateTimeOffset BlockSlotTime { get; init; }
public TransactionTypeUnion TransactionType { get; init; }
public AccountAddress? TransactionSender { get; init; }
public string TransactionHash { get; init; }
Expand Down
Loading
Loading