Skip to content

Capturing benchmark test data

Jump to bottom
Ben Clark edited this page May 5, 2022 · 2 revisions

Something new: Benchmark testing for the new GL-based renderer

We're trying a new experiment- running renderer tests to prove out the new GL-based renderer. These are very much a WIP which we'll talk about more when they're fully baked.

This wiki doc explains how to actually capture the 'canned' .json data that is fed into the benchmarker code.

How to 'can' data:

  • Navigate rapid to a place/area where you want to test the renderer.
  • Refresh the browser to ensure that you have the minimal set of graph information/renderable stuff to do the test.
  • Wait for the renderer to load everything.
  • Set a breakpoint in the modules/pixi/PixiLayerOsm.js file on the line that starts with if (this._saveCannedData ...
  • pan the map a tad- this forces a re-render and your breakpoint should be hit.
  • Modify the 'this._saveCannedData' to true. Hit Continue.
  • A file called something like [zoom][lat][lng]_canned_osm_data.json should get saved to disk.

What's in each canned data file?

This file contains all the renderable entities that we're displaying on the screen, plus every OSM entity in the deep graph that we have in memory. The renderable entities are a subset of the graph- the graph contains many more entities that might be offscreen, invisible (like relations) etc.

Also, we save some metadata about the scene- the zoom, lat, lng, window width and height, as well as the projection used at the time of the snapshot taken.

This is all saved in .json format- our benchmarking code will load and rehydrate all these entities later.

Hand-edit steps

Now we need to curate the data a bit for test consumption. For each file you save, you will want to do three things:

  1. rename the file with a placename at the front of the filename (For example, 16_35.6762_139.6503_canned_osm_data.json would become tokyo_16_35.6762_139.6503_canned_osm_data.json)
  2. Add a data member like placename: 'Tokyo' inside the file. This will allow us to have nice printouts in the benchmarker later!
  3. Modify the filename from .json to .js, and save the entire blob to a variable called ${placename}_${zoom}. For this file, that means sticking var tokyo_16 = at the very beginning of the file, assigning that variable to the json blob we captured.

Prep for use by benchmarking code

  1. You will need to move each canned data file into the RapiD folder test/benchmark.
  2. Ensure that each data file is included by the bench.html benchmarking page. This page should load each canned data script starting at the line commented with put all canned data here. For this example, add the following directive to the page:

<script src="tokyo_16_35.6762_139.6503_canned_osm_data.json"></script>

This will load the canned data into the global variable we specified, so that the benchmark test can use it for the render step.

The Benchmarking code

Some Benchmarking code is written to iterate over each .json file. We'll need to manually iterate over each variable's worth of

Clone this wiki locally