Skip to content

Commit

Permalink
chore: add tests for help.sap.com grounding
Browse files Browse the repository at this point in the history
  • Loading branch information
ZhongpinWang committed Jan 21, 2025
1 parent 15b7fb8 commit cbbde01
Show file tree
Hide file tree
Showing 5 changed files with 84 additions and 12 deletions.
17 changes: 14 additions & 3 deletions sample-code/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,11 +228,11 @@ The chain performs RAG with the chat and embedding client.

### Document Grounding

#### E2E flow for Orchestration Grounding
#### E2E flow for Orchestration Grounding using `vector` as the Data Repository

`GET /document-grounding/invoke-orchestration-grounding`
`GET /document-grounding/invoke-orchestration-grounding-vector`

This scenario demonstrates the end-to-end flow for creating collections and documents using the document grounding service, and then using the orchestration grounding to get a chat completion response with a retrieved context.
This scenario demonstrates the end-to-end flow for creating collections and documents using the document grounding service, and then using the orchestration grounding module to get a chat completion response with a retrieved context based on `vector` data repository.

The flow will first create an empty collection and then add a document to it.
The document contains a statement with the current timestamp.
Expand All @@ -251,3 +251,14 @@ This scenario demonstrates the end-to-end flow for creating collections and docu
The response should contain chunks of documents similar to the query.

The created collection will be deleted at the end of the flow.

#### E2E flow for Orchestration Grounding using `help.sap.com` as the Data Repository

`GET /document-grounding/invoke-orchestration-grounding-help-sap-com`

This scenario uses the orchestration grounding module to get a chat completion response with a retrieved context based on `help.sap.com` data repository.

Orchestration service will send a chat completion request with the context to LLM.
The response should contain the same timestamp.

The created collection will be deleted at the end of the flow.
3 changes: 2 additions & 1 deletion sample-code/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ export {
orchestrationRequestConfig,
orchestrationCompletionMasking,
orchestrationFromJson,
orchestrationGrounding,
orchestrationGroundingVector,
orchestrationGroundingHelpSapCom,
orchestrationChatCompletionImage,
chatCompletionStreamWithJsonModuleConfig,
chatCompletionStream
Expand Down
38 changes: 37 additions & 1 deletion sample-code/src/orchestration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ export async function orchestrationFromJson(): Promise<
* Ask about a custom knowledge embedded in document grounding.
* @returns The orchestration service response.
*/
export async function orchestrationGrounding(): Promise<OrchestrationResponse> {
export async function orchestrationGroundingVector(): Promise<OrchestrationResponse> {
const orchestrationClient = new OrchestrationClient({
llm,
templating: {
Expand All @@ -326,6 +326,42 @@ export async function orchestrationGrounding(): Promise<OrchestrationResponse> {
});
}

/**
* Ask about Generative AI Hub in SAP AI Core and ground the response.
* @returns The orchestration service response.
*/
export async function orchestrationGroundingHelpSapCom(): Promise<OrchestrationResponse> {
const orchestrationClient = new OrchestrationClient({
llm,
templating: {
template: [
{
role: 'user',
content:
'UserQuestion: {{?groundingRequest}} Context: {{?groundingOutput}}'
}
]
},
grounding: buildDocumentGroundingConfig({
input_params: ['groundingRequest'],
output_param: 'groundingOutput',
filters: [
{
id: 'filter1',
data_repositories: ['*'],
data_repository_type: 'help.sap.com'
}
]
})
});

return orchestrationClient.chatCompletion({
inputParams: {
groundingRequest: 'Give me a short introduction of SAP AI Core.'
}
});
}

/**
* Ask about the image content using a template.
* @returns The orchestration service response.
Expand Down
24 changes: 20 additions & 4 deletions sample-code/src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@ import {
orchestrationRequestConfig,
chatCompletionStream as orchestrationChatCompletionStream,
orchestrationFromJson,
orchestrationGrounding,
orchestrationGroundingVector,
orchestrationChatCompletionImage,
chatCompletionStreamWithJsonModuleConfig as orchestrationChatCompletionStreamWithJsonModuleConfig
chatCompletionStreamWithJsonModuleConfig as orchestrationChatCompletionStreamWithJsonModuleConfig,
orchestrationGroundingHelpSapCom
} from './orchestration.js';
import {
getDeployments,
Expand Down Expand Up @@ -420,7 +421,7 @@ app.get('/langchain/invoke-rag-chain', async (req, res) => {

/* Document Grounding */
app.get(
'/document-grounding/invoke-orchestration-grounding',
'/document-grounding/invoke-orchestration-grounding-vector',
async (req, res) => {
try {
res.setHeader('Content-Type', 'text/event-stream');
Expand All @@ -437,7 +438,7 @@ app.get(
res.write(`Document created with timestamp:\t${timestamp}\n`);

// Send an orchestration chat completion request with grounding module configured.
const groundingResult = await orchestrationGrounding();
const groundingResult = await orchestrationGroundingVector();
res.write(
`Orchestration responded with timestamp:\t${groundingResult.getContent()}\n`
);
Expand Down Expand Up @@ -509,3 +510,18 @@ app.get('/document-grounding/invoke-retrieve-documents', async (req, res) => {
.send('Yikes, vibes are off apparently 😬 -> ' + error.message);
}
});

app.get(
'/document-grounding/invoke-orchestration-grounding-help-sap-com',
async (req, res) => {
try {
const groundingResult = await orchestrationGroundingHelpSapCom();
res.send(groundingResult.getContent());
} catch (error: any) {
console.error(error);
res
.status(500)
.send('Yikes, vibes are off apparently 😬 -> ' + error.message);
}
}
);
14 changes: 11 additions & 3 deletions tests/e2e-tests/src/document-grounding.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ import {
createCollection,
createDocumentsWithTimestamp,
deleteCollection,
orchestrationGrounding
orchestrationGroundingHelpSapCom,
orchestrationGroundingVector
} from '@sap-ai-sdk/sample-code';
import { loadEnv } from './utils/load-env.js';

Expand All @@ -22,8 +23,15 @@ describe('document grounding', () => {
await deleteCollection(collectionId);
});

it('should get the result based on grounding context via orchestration API', async () => {
const result = await orchestrationGrounding();
it('should get the result based on grounding context from vector data repository via orchestration API', async () => {
const result = await orchestrationGroundingVector();
expect(result.getContent()).toEqual(timestamp.toString());
});

it('should get the result based on grounding context from `help.sap.com` data respository via orchestration API', async () => {
const result = await orchestrationGroundingHelpSapCom();
expect(result.getContent()).toEqual(expect.any(String));
expect(result.data.module_results).toBeDefined();
expect(result.data.module_results.grounding!.data).toBeDefined();
});
});

0 comments on commit cbbde01

Please sign in to comment.