Apigee Proxy Coverage

This is a tool to determine test case coverage for Apigee proxies.

Given a fully functional API proxy, the tool produces an instrumented proxy file as output. The instrumented proxy file should be deployed to Apigee Edge against which the test cases should be run. Upon completion, the execution data is retrieved by the tool from Apigee Edge to produce the coverage report.

The tool determines coverage of single API proxy under test.

Coverage in case of Apigee proxies is the percentage of policies executed with respect to the total number of policies. Unlike, programming languages where the practical acceptable coverage is around 80%, in case of Apigee proxies the coverage must be 100% - there are no policies that need not be tested or have a remote possibility of execution.

Usage

Pre-requisties

The distribution provides shell scripts, a executable Java 8 jar file and python scripts - which makes bash, python, Java SE 8 as pre-requisites.

---

Steps: (Short Version)

1. Configure environment settings in config_vars
2. Run instr_prepare.sh
3. Run your test cases against the instrumented proxy deployed by previous script
4. Run instr_report.sh to obtain the coverage report.

---

Steps: (A bit more detail)

1. Enter your environment details in the file config_vars. The details of Apigee org, env, api proxy for coverage and the revision of the proxy to be used for instrumentation. Credentials to access the org is essential.
2. Run instr_prepare.sh. Following are steps performed by this script.
   • Download specified revision of proxy bundle from the org.
   • Instrument the downloaded bundle.
   • Deploy the instrumented bundle into the org and activate it.
   • Delete any instrumented data recorded against the proxy in the org.
3. Run the all tests against the deployed instrumented bundle. The instrument bundle has the same base path as the original bundle
4. Run instr_report.sh to obtain the coverage results. Following are the steps performed by this script.
   • Download the instrumented data recorded in the Key Value map in Apigee edge.
   • Generate the coverage report. The report has XML format - a summary.xml and one XML corresponding to every proxy and target endpoints are generated.
   • Custom stylesheets can be used convert the XMLs into desired report format.
   • A HTML version of the report is also generated by applying the default stylesheet.
   • Open the summary.html in the default browser.

Sample Report Screenshots

Summary

Policy Execution Drilldown

Coverage Numbers

Coverage is percentage of policies executed against the total number of policies that exists in the API proxy. With this definition it's possible to have overall coverage at 100%, while the flow level coverage be less than 100%. This is because all the policies would have been executed - but not on all paths.

It's best to achieve 100% flow level coverage in effect attain 100% overall coverage, than vice versa.

Limitations

All proxy endpoints are expected to have only one

• PreFlow tag
• PostFlow tag
• FaultRules tag

Apigee Edge allows multiple of the above tags, though the behaviour in certain cases is un-documented and not supported for instrumentation.

Internals

The tool has to determine the policies that executed in each flow to arrive at the final coverage number.

The tool instruments a given proxy using the following approach.

1. For each policy a new key-value map policy is added to record the execution.
2. The policy is added prior to the actual policy with the same condition as the one that exists on the actual.
3. The rationale is that if the key-value map policy executes so will the actual policy since the same condition is satisfied.
4. Further, its added prior to the actual policy to handle the scenario of fault policies in which case the policy following the fault will not be executed.
5. The generated policies have random unique names to avoid conflict. They log the information into a Key Value Map named instrument at the api proxy level.
6. After test execution, the tool uses the content of the key value map to reconstruct the execution information and calculate coverage.

About

The initial version is implemented by Srikanth Seshadri. Interesting feedback has been provided by the Apigee CS architects for enhancement. Please feel free to contribute.