Enrich Firebase rule coverage reports with more metadata

[IchordeDionysos]

I've internally at @simpleclub developed a package that takes the official Firebase rules coverage report and converts this report to a LOCV file that can be uploaded to any coverage reporting tool.
We've now open-sourced the tool and uploaded it to NPM.

We find it a very helpful addition to monitor/understand how much of your rules are covered by tests!

There are a few things that are not ideal right now and I want to start a discussion on how that might be improved 😃
The conversion script can be found in this file: conversion script
Documentation of the LCOV file format can be found here: LCOV format

In the following I will use these definitions:
Firebase coverage file: The test report that can be downloaded from the emulator.
LCOV coverage file: The output coverage file in the LCOV format.

---

1. if true/if false rules coverage

The Firebase coverage file is generated similar to an AST. There is evaluation information available for whole rules and then it gets more and more detailed until information on what values on variable had.

However, the true/false statements will not be captured in the Firebase rules coverage leading to the following rule to not be captured in the LCOV coverage file:

match: if true;

That means that there is no information available if this rule was ever evaluated or tested.

So, I would propose to add something like this to the Firebase coverage report for a static boolean:

{
  "sourcePosition": { ... },
  "values": [{
    "value": { "boolValue": true/false },
    "count": 1
  }]
}

2. Branch coverage

In an LCOV file, there are multiple dimensions to report coverage on.
Coverage can be reported on a line-by-line basis, but also on how many branches or functions were covered.

For branch coverage, it would be super cool to have more metadata on the relation of the children.

For example, in this case it should probably have two branches:

match: if request.auth.uid == userId || request.auth.uid == 'foo';

One request.auth.uid == userId and the second request.auth.uid == 'foo'.

It may be even should have 4 branches if we include that there should be tests for allow and deny.

My initial idea was to maybe include something like:

{
  "sourcePosition": { ... },
  "values": [ ... ],
  "operator": "||",
  "children:" [ { ... }, { ... } ],
}

3. Function coverage

Related to 2. Branch coverage it would be cool to know if we are in a function to report on function coverage.

I could imagine something like this in the format:

{
  "sourcePosition": { ... },
  "values": [ ... ],
  "function": "isAdmin",
  "children:" [ { ... }, { ... } ],
}

---

Looking forward to a constructive discussion!
If this discussion fit better in a different place or should be split up, let me know :)

Dennis

[hiranya911]

This should be really discussed at https://github.com/firebase/firebase-tools

Admin SDK has no relevance to any of the above.