Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: x-ar-io-root-transaction-id header #225

Merged
merged 1 commit into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export const headerNames = {
stable: 'X-AR-IO-Stable',
verified: 'X-AR-IO-Verified',
cache: 'X-Cache',
rootTransactionId: 'X-AR-IO-Root-Transaction-Id',
arnsTtlSeconds: 'X-ArNS-TTL-Seconds',
arnsResolvedId: 'X-ArNS-Resolved-Id',
arnsProcessId: 'X-ArNS-Process-Id',
Expand Down
8 changes: 4 additions & 4 deletions src/database/sql/core/data-attributes.sql
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
-- selectDataAttributes
SELECT *
FROM (
SELECT data_root, data_size, content_type, content_encoding, true AS stable
SELECT data_root, data_size, content_type, content_encoding, null AS root_transaction_id, true AS stable
FROM stable_transactions
WHERE id = @id
UNION
SELECT data_root, data_size, content_type, content_encoding, false AS stable
SELECT data_root, data_size, content_type, content_encoding, null AS root_transaction_id, false AS stable
FROM new_transactions
WHERE id = @id
UNION
SELECT null, data_size, content_type, content_encoding, true AS stable
SELECT null AS data_root, data_size, content_type, content_encoding, root_transaction_id, true AS stable
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to null AS data_root for readability

FROM bundles.stable_data_items
WHERE id = @id
UNION
SELECT null, data_size, content_type, content_encoding, false AS stable
SELECT null AS data_root, data_size, content_type, content_encoding, root_transaction_id, false AS stable
FROM bundles.new_data_items
WHERE id = @id
)
Expand Down
9 changes: 8 additions & 1 deletion src/database/standalone-sqlite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -981,12 +981,19 @@ export class StandaloneSqliteDatabaseWorker {
const hash = dataRow?.hash;
const dataRoot = coreRow?.data_root;

const rootTransactionId =
coreRow?.root_transaction_id !== null &&
coreRow?.root_transaction_id !== undefined
Comment on lines +985 to +986
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checking for null AND undefined because it can return null for transactions where root_transaction_id doesn't exist and return undefined for bundles/data items.

? toB64Url(coreRow.root_transaction_id)
: undefined;

return {
hash: hash ? toB64Url(hash) : undefined,
dataRoot: dataRoot ? toB64Url(dataRoot) : undefined,
size: coreRow?.data_size ?? dataRow?.data_size,
contentType,
contentEncoding: coreRow?.content_encoding,
contentType,
rootTransactionId,
isManifest: contentType === MANIFEST_CONTENT_TYPE,
stable: coreRow?.stable === true,
verified: dataRow?.verified === 1,
Expand Down
59 changes: 59 additions & 0 deletions src/routes/data/handlers.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,65 @@ st
});
});
});

describe('X-AR-IO-Root-Transaction-Id', () => {
it("shouldn't return root transaction id for transactions", async () => {
app.get(
'/:id',
createDataHandler({
log,
dataIndex,
dataSource,
blockListValidator,
manifestPathResolver,
}),
);

return request(app)
.get('/not-a-real-id')
.expect(200)
.then((res: any) => {
assert.equal(
res.headers['x-ar-io-root-transaction-id'],
undefined,
);
});
});

it('should return root transaction id for data items', async () => {
dataIndex.getDataAttributes = () =>
Promise.resolve({
size: 10,
contentType: 'application/octet-stream',
rootTransactionId: 'root-tx',
isManifest: false,
stable: true,
verified: true,
signature: null,
});

app.get(
'/:id',
createDataHandler({
log,
dataIndex,
dataSource,
blockListValidator,
manifestPathResolver,
}),
);

return request(app)
.get('/not-a-real-id')
.expect(200)
.then((res: any) => {
assert.equal(
res.headers['x-ar-io-root-transaction-id'],
'root-tx',
);
});
});
});
Comment on lines +676 to +733
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider adding more test cases for comprehensive coverage.

The new test cases for the X-AR-IO-Root-Transaction-Id header are well-implemented and cover the main scenarios. To further improve the test suite, consider adding the following test cases:

  1. Test when rootTransactionId is undefined or null in the data attributes.
  2. Test with different HTTP methods (e.g., HEAD request) to ensure consistent behavior.
  3. Test error scenarios, such as when getDataAttributes fails.

These additional tests would provide more comprehensive coverage and help catch potential edge cases.

});
});
});
7 changes: 4 additions & 3 deletions src/routes/data/handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,6 @@ const setDataHeaders = ({
dataAttributes: ContiguousDataAttributes | undefined;
data: ContiguousData;
}) => {
// TODO add etag
// TODO add header indicating stability
// TODO add header indicating whether data is verified
// TODO cached header for zero length data (maybe...)

// Allow range requests
Expand Down Expand Up @@ -123,6 +120,10 @@ const setDataHeaders = ({
res.header('Content-Encoding', dataAttributes.contentEncoding);
}

if (dataAttributes?.rootTransactionId !== undefined) {
res.header(headerNames.rootTransactionId, dataAttributes.rootTransactionId);
}

setDigestStableVerifiedHeaders({ res, dataAttributes, data });
};

Expand Down
3 changes: 2 additions & 1 deletion src/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -506,7 +506,8 @@ export interface ContiguousDataAttributes {
dataRoot?: string;
size: number;
contentEncoding?: string;
contentType: string | undefined;
contentType?: string;
rootTransactionId?: string;
isManifest: boolean;
stable: boolean;
verified: boolean;
Expand Down
56 changes: 56 additions & 0 deletions test/end-to-end/data.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ const tx3 = 'lbeIMUvoEqR2q-pKsT4Y5tz6mm9ppemReyLnQ8P7XpM';
// manifest with paths without trailing slash
const tx4 = 'sYaO7sklQ8FyObQNLy7kDbEvwUNKKes7mUnv-_Ri9bE';

const bundle1 = '73QwVewKc0hXmuiaahtGJqHEY5pb85SoqCC33VE0Teg';

describe('Data', function () {
let compose: StartedDockerComposeEnvironment;

Expand Down Expand Up @@ -277,6 +279,60 @@ describe('X-Cache header', function () {
});
});

describe('X-AR-IO-Root-Transaction-Id header', function () {
let compose: StartedDockerComposeEnvironment;

before(async function () {
await rimraf(`${projectRootPath}/data/sqlite/*.db*`, { glob: true });

compose = await new DockerComposeEnvironment(
projectRootPath,
'docker-compose.yaml',
)
.withEnvironment({
START_HEIGHT: '1',
STOP_HEIGHT: '1',
GET_DATA_CIRCUIT_BREAKER_TIMEOUT_MS: '100000',
ANS104_UNBUNDLE_FILTER: '{"always": true}',
ANS104_INDEX_FILTER: '{"always": true}',
ADMIN_API_KEY: 'secret',
})
.withBuild()
.withWaitStrategy('core-1', Wait.forHttp('/ar-io/info', 4000))
.up(['core']);

await axios.post(
'http://localhost:4000/ar-io/admin/queue-bundle',
{ id: bundle1 },
{
headers: {
'Content-Type': 'application/json',
Authorization: 'Bearer secret',
},
},
);

// hopefully enough time to unbundle it
await wait(10000);
});

after(async function () {
await compose.down();
});

it('Verifying header for trascation', async function () {
const bundleRes = await axios.head(`http://localhost:4000/raw/${bundle1}`);

assert.equal(bundleRes.headers['x-ar-io-root-transaction-id'], undefined);
});

it('Verifying header for data item', async function () {
const datasItemRes = await axios.head(`http://localhost:4000/raw/${tx1}`);

assert.equal(datasItemRes.headers['x-ar-io-root-transaction-id'], bundle1);
});
});

describe('X-AR-IO headers', function () {
describe('with ARNS_ROOT_HOST', function () {
let compose: StartedDockerComposeEnvironment;
Expand Down