A better storage format for postman collections
by Natarajan Kannan
Postman collections are created using the Postman UI application. These
collections are represented as a JSON
file
by the Postman UI application. It is a single file that contains all the
requests/tests/variables etc of the collection. It is designed to be edited
within the Postman UI. The newman tool allows teams to run Postman collections
as part of their CI pipelines by supplying the Postman collection file as an
argument. Typically, teams that rely on Postman for their API testing in CI
pipelines maintain a copy of the collection file in their source repository.
When teams maintain the Postman collection in their repository, they will have
the following workflow to make changes to it and push is upstream:
- Import the collection json file from the repo into Postman UI
- Make changes to the collection in Postman UI
- Export the collection to a file using Postman UI
- Commit and push the changes
There are a few challenges with the above workflow:
- Collection file diffs are very hard to review in pull requests
- Old format collection diff: https://github.com/juspay/hyperswitch/commit/7e29adb5c9dee8b03ef58ccbd85b07b106459380
- New format collection diff: https://github.com/juspay/hyperswitch/pull/2117/files
- Old format collection diff: https://github.com/juspay/hyperswitch/commit/7e29adb5c9dee8b03ef58ccbd85b07b106459380
- Developers cannot use their favourite editor to make changes, add/remove new tests
As a consequence of the above, collection maintenance suffers and often become stale.
The core reason for challenges presented above is the file format in which the Postman requests are maintained. A single JSON file that encapsulates the entire request code and sequencing does not lend itself for easy reviewing and be an all-editor friendly format.
This fork of newman attempts to address the challenge by representing the
Postman collection as a set of directories/files. It allows developers to
create Postman collection using the directory representation and also run them.
It achieves this by re-constructing the collection json from the directory/file
structure and leveraging the existing newman run implementation.
For example, this is the directory equivalent
representation of a sample
collection in the examples directory as
generated by the dir-export command.
The following is a tree structure of the directory that represents the Postman collection with annotations of what each file corresponds to in a Postman collection.
examples/Sample Postman Collection
├── .info.json # contains the info element from the top of collection json
├── .meta.json # contains the ordering of the request folders
├── .auth.json # contains the auth element from the top of the collection json
├── .variable.json # contains the variable element from the top of the collection json
├── .event.json # contains the event element from the top of the collection json
└── A simple GET request
├── .event.meta.json # contains the ordering of the scripts - prerequest / test
├── event.test.js # the test script that runs post the request
├── event.prerequest.js # the test script that runs before the request
├── request.json # the request json from the Postman collection
└── response.json # the response json
This concept of representing Postman collections as a directory opens up programmatic pre-processing of request data before running the requests. This could include things like the following:
- Re-using same data across requests
- Using other javascript libraries in testing code
The instructions for using the tool are given below. Please give it spin and let me know it you if it is useful.
One can install the newman executable in this fork using the command:
npm install -g 'git+ssh://[email protected]:knutties/newman.git#feature/newman-dir'
The following no-arg run shows the new commands added to newman to help manage Postman requests maintained as directories/files instead of a single json file.
$ newman
Usage: newman [options] [command]
Options:
-v, --version output the version number
-h, --help display help for command
Commands:
dir-add-folder [options] <folder-path> Add a folder to directory based Postman collection in the given path
dir-add-request [options] <request-path> Add a request to directory based Postman collection in the given path
dir-collection-create [options] <collection-path> Create a directory based Postman collection in the given path
dir-export [options] <postman-collection-file> Convert a Postman collection file into its directory representation
dir-export-import-check [options] <postman-collection-file> Check if an export followed by import results in same collection
dir-import [options] <collection-dir> Convert a Postman directory representation into a postman collection
dir-remove-folder <folder-path> Remove folder at given path from directory based Postman collection
dir-remove-request <request-path> Remove request at given path from directory based Postman collection
dir-run [options] <collection-dir> Runs the requests in collection-dir, with all the provided options
run [options] <collection> Initiate a Postman Collection run from a given URL or path
To get available options for a command:
newman <command> -h
The following sections show the invocation of the different commands added in this fork of newman.
newman dir-collection-create new-dir-collection
newman dir-export examples/sample-collection.json
newman dir-import examples/Sample\ Postman\ Collection/ -o examples/sample-collection.json
This command is typically used to test the tool itself to ensure it can handle all collection json use-cases.
newman dir-export-import-check examples/sample-collection.json
newman dir-run examples/Sample\ Postman\ Collection/
The dir-run command supports all the options that the stock run command of
newman supports. This is achieved by re-using the same set of command options
for both the commands.
Sample output of dir-run shown below
newman
Sample Postman Collection
→ A simple GET request
GET https://postman-echo.com/get?source=newman-sample-github-collection [200 OK, 847B, 1054ms]
✓ expect response be 200
✓ expect response json contain args
→ A simple POST request
POST https://postman-echo.com/post [200 OK, 1.06kB, 317ms]
→ A simple POST request with JSON body
POST https://postman-echo.com/post [200 OK, 1.17kB, 233ms]
┌─────────────────────────┬─────────────────────┬─────────────────────┐
│ │ executed │ failed │
├─────────────────────────┼─────────────────────┼─────────────────────┤
│ iterations │ 1 │ 0 │
├─────────────────────────┼─────────────────────┼─────────────────────┤
│ requests │ 3 │ 0 │
├─────────────────────────┼─────────────────────┼─────────────────────┤
│ test-scripts │ 1 │ 0 │
├─────────────────────────┼─────────────────────┼─────────────────────┤
│ prerequest-scripts │ 0 │ 0 │
├─────────────────────────┼─────────────────────┼─────────────────────┤
│ assertions │ 2 │ 0 │
├─────────────────────────┴─────────────────────┴─────────────────────┤
│ total run duration: 1647ms │
├─────────────────────────────────────────────────────────────────────┤
│ total data received: 2.13kB (approx) │
├─────────────────────────────────────────────────────────────────────┤
│ average response time: 534ms [min: 233ms, max: 1054ms, s.d.: 368ms] │
└─────────────────────────────────────────────────────────────────────┘
newman dir-add-folder examples/Sample\ Postman\ Collection/folder1
newman dir-add-request examples/Sample\ Postman\ Collection/test4
This command adds the request to the end of the requests already present in the folder. The order of the requests is stored in a separate file called .meta.json. The order of requests can be changed by re-ordering the requests in this file.
newman dir-remove-folder examples/Sample\ Postman\ Collection/folder1
newman dir-remove-request examples/Sample\ Postman\ Collection/test4