What is the relationship between WebRTC stats and the Working Group?

[henbos]

It was my understanding that 6-7 years ago, the WebRTC stats editors was asked to lead the stats work on behalf of the WG.
I've been committed to this work since at least February, 2017 and we've come a long way in our goal to achieve browser-compatible stats, finally being able to deprecate legacy getStats.

This was primarily achieved through bi-weekly meetings between me, @vr000m and @alvestrand. The WG has been involved regarding the mandatory to implement list, WebRTC stats has frequently been presented at TPAC (example) and occasionally WebRTC stats has been featured at Virtual Interims to widen the audience.

But the bulk of the work has happened in the bi-weekly stats meetings and metrics defined there, while not mandatory to implement unless the WG says so, have been considered ready to implement and ship, and the libwebrtc implementation has slowly evolved over many years based on this process. So I was quite surprised about this comment by @jan-ivar. The full comment is worth reading, but here is a quote from it:

My understanding of the role of editors includes discretion to craft and merge editorial PRs, maybe PRs for normative bug fixes where WG opinion can be inferred from earlier discussion, and PRs for issues that have been discussed in the WG that the WG agrees to proceed with.

According to this view, we should not have done much of anything without asking the WG. While the WG has been involved numerous times, stats work being editorial has not been practised since the inception of getStats(), though not all stats have been considered mandatory to implement either.

What should the process be going forward? @jan-ivar @aboba @youennf @alvestrand @vr000m

[henbos]

@dontcallmedom do you remember when auto publishing was turned on or how that decision was made?

[dontcallmedom]

The decision was taken in 2016 https://lists.w3.org/Archives/Public/public-webrtc/2016Mar/0031.html ; naturally, if using for post-CR documents is seen as too confusing, that decision can be revisited (although having editors draft and TR documents not in sync is a source of confusion too)

[youennf]

What should the process be going forward?

I agree with @jan-ivar quote here.
Ideally, the biweekly meeting would design proposals that would then be presented to the WebRTC WG for discussion if these are normative changes.

It does not have to be during the WebRTC WG meetings, it could be through any other available means.
If we do not anticipate a high flow of WebRTC stat changes, I would tend to process WebRTC stat issues similarly to other WebRTC issues.

If a high flow of changes is expected, we could of course be more creative (task force with decision making rights maybe) but it does not seem needed to me.

[henbos]

I thought we were this task force with decision making rights, but it appears not. In the past the flow of issues was really large, but today the spec has been relatively stable and the number of issues is usually pretty low in volume

[youennf]

I thought we were this task force with decision making rights

Maybe you are correct about this, I do not really know.
Given where we are and the stat spec being stable, do you anticipate downsides to moving to the regular WebRTC flow?

[henbos]

If we can get a regular spot at the editor's meeting to discuss new stats (if there are any that week/month) that may be fine. Perhaps we can tag PRs we think are ready for WG review as such and then schedule to join an upcoming editor's meeting to talk about them?

[henbos]

Most weeks we probably would not have any PRs, but when we do have PRs it would be nice not to have to wait a month to get a decision

[vr000m]

The reason for the lightweight process was that, historically, the metrics were well defined and if there was a good standards reference, it was quicker to include it into the spec, as webrtc observability was in its infancy and we needed a corpus of well defined metrics, for browsers to implement.

However, with the emergence of the experimental spec, the bar for the main webrtc-stats spec was raised from well defined metric to at least one implementation of the metric (the implicit assumption that it was well defined and/or was shown to be useful in an origin trial or similar). All metrics that do not meet the bar are moved to the experimental spec.

So perhaps a moment in time to re-think the above process. I would rather move towards a more rigorous PR process than have a combined editor's meeting (which would still not be an open meeting).

[jan-ivar]

I thought we were this task force with decision making rights

Our W3C process states (scroll down): "Task forces do not publish technical reports; the Working Group may choose to publish their results as part of a technical report."

[jan-ivar]

... though not all stats have been considered mandatory to implement either.

I read https://www.w3.org/TR/webrtc/#mandatory-to-implement-stats as a normative reference affecting the conformance criteria of https://www.w3.org/TR/webrtc, not this spec. See also w3c/webrtc-pc#2844.

However, with the emergence of the experimental spec, the bar for the main webrtc-stats spec was raised from well defined metric to at least one implementation of the metric ... All metrics that do not meet the bar are moved to the experimental spec.

The bar for webrtc-stats to exit CR is "two independent implementations of every feature defined in the specification". I've filed #749.

[henbos]

The conclusion here seems to be that substative changes such as adding new stats require WG approval. I'll close this issue.