Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Data reports #23

Open
wants to merge 74 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
474fc24
Refactor processing and fix caching and fix tests
sophialittlejohn Nov 20, 2024
c3ab9ad
Clean up processor and add tests and mocks
sophialittlejohn Nov 20, 2024
89cca6f
Clean up types
sophialittlejohn Nov 20, 2024
562e14c
Add cashflow report and tests
sophialittlejohn Nov 21, 2024
8625327
Remove defer and use querywith poolid
sophialittlejohn Nov 21, 2024
710571f
Refactor tests and add comments
sophialittlejohn Nov 21, 2024
0cc968f
Add query for metadata and utils around it
sophialittlejohn Nov 22, 2024
62a6fff
Add data for pool fee snapshots and use in cashflow statments
sophialittlejohn Nov 22, 2024
bd50461
Group tranche data by id to prevent double processing
sophialittlejohn Nov 22, 2024
efc2e37
fix zero case in fromFloat wrapper
sophialittlejohn Nov 22, 2024
2580226
Prevent fees and endCashBalance from being aggregated
sophialittlejohn Nov 23, 2024
3c4b0b6
Add fee snapshot mocks and improve fee testing
sophialittlejohn Nov 23, 2024
a059805
Add a few more fee test
sophialittlejohn Nov 23, 2024
a205c3d
Differentiate public and private credit for cashflows
sophialittlejohn Nov 25, 2024
c4b9997
Remove identifying info
sophialittlejohn Nov 25, 2024
d44e2d8
Fix data aggregation and improve test coverage with real data
sophialittlejohn Nov 25, 2024
fd88d5a
Add code coverage support
sophialittlejohn Nov 25, 2024
2396604
Add coverage summary
sophialittlejohn Nov 26, 2024
452816e
Exclude types from coverage
sophialittlejohn Nov 26, 2024
a44d576
And P&L report and processor tests
sophialittlejohn Nov 26, 2024
cc8926a
Add tests for applyGrouping
sophialittlejohn Nov 26, 2024
9f1c6b5
Add readme for reports
sophialittlejohn Nov 26, 2024
22ee830
Add test to applyGrouping
sophialittlejohn Nov 27, 2024
fabe96e
Fix test
sophialittlejohn Nov 27, 2024
df1c10d
Merge branch 'main' of github.com:centrifuge/centrifuge-sdk into reports
sophialittlejohn Nov 28, 2024
01879fd
Fix BigInt tests
sophialittlejohn Nov 29, 2024
97b6efc
Clean up metadata fetching
sophialittlejohn Nov 29, 2024
db48c58
Remove cache time
sophialittlejohn Nov 29, 2024
3c672d3
Remove filtering in keys
sophialittlejohn Nov 29, 2024
25fa8bf
Fix import so that decimal.js-light works in browser and test env
sophialittlejohn Nov 29, 2024
0a78c08
Update README.md
sophialittlejohn Dec 2, 2024
037fa20
Update README.md
sophialittlejohn Dec 2, 2024
35908e5
Merge branch 'main' of github.com:centrifuge/centrifuge-sdk into reports
sophialittlejohn Dec 2, 2024
d6ca3fe
Use purchases and financings instead of acquisistions
sophialittlejohn Dec 2, 2024
b380c98
Remove coverage in favor of nyc in ci PR
sophialittlejohn Dec 2, 2024
30bb9a4
Remove c8 config
sophialittlejohn Dec 2, 2024
1ce0bbc
Merge branch 'main' of github.com:centrifuge/centrifuge-sdk into reports
sophialittlejohn Dec 2, 2024
bd394c0
Restore sinon stub to fix error
sophialittlejohn Dec 2, 2024
bbe15a8
Add fix to P&L report
sophialittlejohn Dec 2, 2024
59dd96c
Add investor tx data and report
sophialittlejohn Dec 2, 2024
6a31f46
Provide access to P&L
sophialittlejohn Dec 2, 2024
b66710f
Merge branch 'reports' of github.com:centrifuge/centrifuge-sdk into m…
sophialittlejohn Dec 2, 2024
9605fba
Add tests
sophialittlejohn Dec 2, 2024
4e1645d
Merge branch 'main' of github.com:centrifuge/centrifuge-sdk into more…
sophialittlejohn Dec 3, 2024
54c796e
Add filters and tests to investor transaction report
sophialittlejohn Dec 3, 2024
f6afede
Add more test cases
sophialittlejohn Dec 3, 2024
91476bc
Test investor report generation
sophialittlejohn Dec 3, 2024
7d2162f
Optimize processing
sophialittlejohn Dec 3, 2024
3e3ed1e
Add asset transaction report
sophialittlejohn Dec 3, 2024
36f25a0
Fix typo
sophialittlejohn Dec 3, 2024
19a40f3
Rename so it doesn't appear as main README in repo
sophialittlejohn Dec 4, 2024
20e39d3
Merge branch 'main' of github.com:centrifuge/centrifuge-sdk into more…
sophialittlejohn Dec 4, 2024
c26a47f
Fix asset transaction filtering
sophialittlejohn Dec 4, 2024
ebaf007
Add fee transactions report
sophialittlejohn Dec 4, 2024
5bc3b42
Make filtering more efficient
sophialittlejohn Dec 4, 2024
0c767ce
Add token price report
sophialittlejohn Dec 5, 2024
4bb2474
Move types for reports so they're excluded from coverage
sophialittlejohn Dec 5, 2024
4c8d6a1
Convert date to UTC to prevent timezone bugs in grouping
sophialittlejohn Dec 5, 2024
7b4550c
Fix filtering by type for fee transactions
sophialittlejohn Dec 5, 2024
49b72e6
Improve test coverage and fix some filtering bugs
sophialittlejohn Dec 5, 2024
473aa99
Remove log and comment
sophialittlejohn Dec 5, 2024
b64ceea
More test coverage improvements
sophialittlejohn Dec 5, 2024
05f95bb
Move indexer querying into separate class
sophialittlejohn Dec 5, 2024
23414f5
Add asset list report
sophialittlejohn Dec 5, 2024
9286daa
Add tests for asset list
sophialittlejohn Dec 6, 2024
291b47d
Add investor list report
sophialittlejohn Dec 6, 2024
654a7bb
Merge branch 'main' into more-reports
gpmayorga Dec 20, 2024
1644754
ci: fix issue with pr_number check
gpmayorga Dec 20, 2024
5cc5c8e
ci: fix release script logic
gpmayorga Dec 20, 2024
5bab699
ci: debugging pr-check script
gpmayorga Dec 20, 2024
b0f1963
ci: final fix for PR checks on release events
gpmayorga Dec 20, 2024
df6af85
[bot] New pkg version: 0.0.0-alpha.3
actions-user Dec 20, 2024
27c3484
ci: final cleanup
gpmayorga Dec 20, 2024
bc75ece
Merge branch 'main' into more-reports
gpmayorga Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ This workflow is responsible for versioning and preparing releases based on pull

- **Alpha Versioning**: When a PR includes the `alpha` label, versions are created with an `-alpha.N` suffix (e.g., `1.2.3-alpha.1`, `1.2.3-alpha.2`). This continues until the alpha label is removed, at which point the version returns to standard semantic versioning.


### 2. `build-test-report.yml`

This workflow handles building, testing, and reporting.
Expand Down Expand Up @@ -56,4 +55,3 @@ This workflow handles the publishing of releases to NPM.
2. **Draft Release Creation**: The `create-release.yml` workflow creates a draft release if `package.json` contains a new version on the `main` branch.

3. **Publishing**: When a release is published on GitHub (moved from draft to released), the `publish-release.yml` workflow is triggered. It builds the project and publishes it to NPM, ensuring the package is available for public use.

13 changes: 9 additions & 4 deletions .github/ci-scripts/pr-label-check.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
#!/bin/bash

# Check if PR number is provided
if [ -z "$1" ]; then
echo "Error: PR number is required."
exit 1
fi

PR_NUMBER="$1"

# Function to check if a specific label exists in a list
contains_label() {
local label="$1"
Expand All @@ -12,11 +20,8 @@ contains_label() {
return 1
}

# Fetch the most recent merged PR number
LAST_PR_NUMBER=$(gh pr list --state merged --json number --jq '.[0].number')

# Fetch the labels of the PR
labels=$(gh pr view "$LAST_PR_NUMBER" --json labels --jq '.labels[].name')
labels=$(gh pr view "$PR_NUMBER" --json labels --jq '.labels[].name')

# Convert labels to an array
IFS=$'\n' read -rd '' -a label_array <<<"$labels"
Expand Down
22 changes: 15 additions & 7 deletions .github/workflows/create-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: |
if [ "${{ github.event_name }}" == "pull_request" ]; then
if [ "${{ github.event_name }}" == "pull_request_review" ]; then
PR_NUMBER=${{ github.event.pull_request.number }}
else
PR_NUMBER=$(gh pr list --state merged --json number --jq '.[0].number')
Expand All @@ -54,19 +54,27 @@ jobs:
echo "::notice::PR used to check version bump: #$PR_NUMBER"
echo "::notice::PR labels: "[${LABELS//$'\n'/,}]""

RELEASE_TYPE=$(./.github/ci-scripts/pr-label-check.sh) || exit_code=$?
if [ "$exit_code" -ne 0 ]; then
echo "::error::PR #"$PR_NUMBER" has more than one release label"
exit $exit_code
# Make script executable
chmod +x ./.github/ci-scripts/pr-label-check.sh

# Run the script and capture both output and exit code
RELEASE_TYPE=$(./.github/ci-scripts/pr-label-check.sh $PR_NUMBER 2>&1) || {
# If script exits with non-zero, output the error and exit
echo "::error::Script failed with output: $RELEASE_TYPE"
exit 1
}

# Only proceed if we have a valid release type
if [ -z "$RELEASE_TYPE" ]; then
echo "::error::No valid release type determined"
exit 1
fi

echo "version=$RELEASE_TYPE" >> $GITHUB_OUTPUT
echo "::group::Output version"
echo "::notice::Release type: $RELEASE_TYPE"
if [ "$RELEASE_TYPE" == "no-release" ]; then
echo "::notice::PR is not flagged for release, skipping other steps"
fi
echo "::endgroup::"

- name: Bump version
id: bump
Expand Down
161 changes: 161 additions & 0 deletions src/IndexerQueries/assetSnapshots.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
import { Currency, Rate } from '../utils/BigInt.js'

export type AssetSnapshotFilter = Partial<Record<keyof SubqueryAssetSnapshots['assetSnapshots']['nodes'][0], any>>

export type AssetSnapshot = {
actualMaturityDate: string | undefined
actualOriginationDate: number | undefined
advanceRate: Rate | undefined
assetId: string
collateralValue: Currency | undefined
currentPrice: Currency | undefined
discountRate: Rate | undefined
faceValue: Currency | undefined
lossGivenDefault: Rate | undefined
name: string
outstandingDebt: Currency | undefined
outstandingInterest: Currency | undefined
outstandingPrincipal: Currency | undefined
outstandingQuantity: Currency | undefined
presentValue: Currency | undefined
probabilityOfDefault: Rate | undefined
status: string
sumRealizedProfitFifo: Currency | undefined
timestamp: string
totalRepaidInterest: Currency | undefined
totalRepaidPrincipal: Currency | undefined
totalRepaidUnscheduled: Currency | undefined
unrealizedProfitAtMarketPrice: Currency | undefined
valuationMethod: string | undefined
}

export type SubqueryAssetSnapshots = {
assetSnapshots: {
nodes: {
asset: {
pool: {
currency: {
decimals: number
}
}
actualOriginationDate: number
advanceRate: string | undefined
collateralValue: string | undefined
discountRate: string | undefined
id: string
lossGivenDefault: string | undefined
actualMaturityDate: string | undefined
name: string
probabilityOfDefault: string | undefined
status: string
sumRealizedProfitFifo: string | undefined
unrealizedProfitAtMarketPrice: string | undefined
valuationMethod: string
notional: string | undefined
}
timestamp: string
assetId: string
presentValue: string | undefined
currentPrice: string | undefined
outstandingPrincipal: string | undefined
outstandingInterest: string | undefined
outstandingDebt: string | undefined
outstandingQuantity: string | undefined
totalRepaidPrincipal: string | undefined
totalRepaidInterest: string | undefined
totalRepaidUnscheduled: string | undefined
}[]
}
}

export const assetSnapshotsPostProcess = (data: SubqueryAssetSnapshots): AssetSnapshot[] => {
return data!.assetSnapshots.nodes.map((tx) => {
const currencyDecimals = tx.asset.pool.currency.decimals
return {
...tx,
timestamp: tx.timestamp,
assetId: tx.assetId,
actualMaturityDate: tx.asset.actualMaturityDate || undefined,
actualOriginationDate: tx.asset.actualOriginationDate || undefined,
advanceRate: new Rate(tx.asset.advanceRate || '0'),
collateralValue: tx.asset?.collateralValue
? new Currency(tx.asset?.collateralValue, currencyDecimals)
: undefined,
currentPrice: tx.currentPrice ? new Currency(tx.currentPrice, currencyDecimals).mul(10n ** 18n) : undefined,
discountRate: tx.asset.discountRate ? new Rate(tx.asset.discountRate) : undefined,
faceValue:
tx.asset.notional && tx.outstandingQuantity
? new Currency(tx.asset.notional, currencyDecimals).mul(BigInt(tx.outstandingQuantity))
: undefined,
lossGivenDefault: tx.asset.lossGivenDefault ? new Rate(tx.asset.lossGivenDefault) : undefined,
name: tx.asset.name,
outstandingDebt: tx.outstandingDebt ? new Currency(tx.outstandingDebt, currencyDecimals) : undefined,
outstandingInterest: tx.outstandingInterest ? new Currency(tx.outstandingInterest, currencyDecimals) : undefined,
outstandingPrincipal: tx.outstandingPrincipal
? new Currency(tx.outstandingPrincipal, currencyDecimals)
: undefined,
outstandingQuantity: tx.outstandingQuantity ? new Currency(tx.outstandingQuantity, 18) : undefined,
presentValue: tx.presentValue ? new Currency(tx.presentValue, currencyDecimals) : undefined,
probabilityOfDefault: tx.asset.probabilityOfDefault ? new Rate(tx.asset.probabilityOfDefault) : undefined,
status: tx.asset.status,
sumRealizedProfitFifo: tx.asset.sumRealizedProfitFifo
? new Currency(tx.asset.sumRealizedProfitFifo, currencyDecimals)
: undefined,
totalRepaidInterest: tx.totalRepaidInterest ? new Currency(tx.totalRepaidInterest, currencyDecimals) : undefined,
totalRepaidPrincipal: tx.totalRepaidPrincipal
? new Currency(tx.totalRepaidPrincipal, currencyDecimals)
: undefined,
totalRepaidUnscheduled: tx.totalRepaidUnscheduled
? new Currency(tx.totalRepaidUnscheduled, currencyDecimals)
: undefined,
unrealizedProfitAtMarketPrice: tx.asset.unrealizedProfitAtMarketPrice
? new Currency(tx.asset.unrealizedProfitAtMarketPrice, currencyDecimals)
: undefined,
valuationMethod: tx.asset.valuationMethod,
}
}) satisfies AssetSnapshot[]
}

export const assetSnapshotsQuery = `
query($filter: AssetSnapshotFilter) {
assetSnapshots(
first: 1000,
orderBy: TIMESTAMP_ASC,
filter: $filter
) {
nodes {
assetId
timestamp
totalRepaidUnscheduled
outstandingInterest
totalRepaidInterest
currentPrice
outstandingPrincipal
totalRepaidPrincipal
outstandingQuantity
presentValue
outstandingDebt
asset {
pool {
currency {
decimals
}
}
actualMaturityDate
actualOriginationDate
advanceRate
collateralValue
discountRate
lossGivenDefault
name
notional
probabilityOfDefault
status
sumRealizedProfitFifo
unrealizedProfitAtMarketPrice
valuationMethod
}
}
}
}
`
Loading
Loading