Merge branch 'master' into ethernet-segment

.github/CODEOWNERS

@@ -4,6 +4,11 @@
 * @aashaikh @robshakir @dplore
 
 # the release/models directory (all YANG content)
-# is maintained by the public-writers OpenConfig team. 
+# and doc directory is maintained by the public-writers 
+# OpenConfig team. 
+/doc @openconfig/public-writers
 /release/models/ @openconfig/public-writers
+
+# subfolders may have additional codeowners who
+# may merge content
 /release/models/wifi @mike-albano

cloudbuild.yaml

@@ -1,3 +1,6 @@
+substitutions:
+    _OC_PYANG_VERSION: master
+
 steps:
 ############### GET CI REPO ###############
 # Decrypt the file containing the key
@@ -15,6 +18,7 @@ steps:
   - name: 'ssh'
     path: /root/.ssh
 # Set up git with key and domain
+# See https://cloud.google.com/build/docs/access-github-from-build#add_the_public_ssh_key_to_known_hosts
 - name: 'gcr.io/cloud-builders/git'
   entrypoint: 'bash'
   args:
@@ -48,7 +52,7 @@ steps:
     git clone [email protected]:openconfig/models-ci.git /go/src/github.com/openconfig/models-ci
     cd /go/src/github.com/openconfig/models-ci
     # Modify the major version to update models-ci version.
-    branch=$(git tag -l 'v8*' | sort -V | tail -1)
+    branch=$(git tag -l 'v9*' | sort -V | tail -1)
     git checkout $branch
   volumes:
   - name: 'ssh'
@@ -83,9 +87,9 @@ steps:
     -pr-head-repo-url=$_HEAD_REPO_URL
     -commit-sha=$COMMIT_SHA
     -pr-number=$_PR_NUMBER
-    -skipped-validators=confd
+    -skipped-validators=confd,yanglint
     -extra-pyang-versions=2.2.1
-    -compat-report=yanglint
+    #-compat-report=yanglint
     -branch=$BRANCH_NAME
   secretEnv: ['GITHUB_ACCESS_TOKEN']
   env:
@@ -126,7 +130,7 @@ steps:
   dir: '/go/src/github.com/openconfig/pattern-regex-tests'
   waitFor: ['regexp clone']
   id: 'regexp dep'
-- name: 'gcr.io/$PROJECT_ID/models-ci-image'
+- name: 'us-west1-docker.pkg.dev/$PROJECT_ID/models-ci/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', "/go/src/github.com/openconfig/models-ci/validators/regexp/test.sh"]
   secretEnv: ['GITHUB_ACCESS_TOKEN']
@@ -152,7 +156,7 @@ steps:
   args: ['cp', 'gs://openconfig/yanglint.deb', '/workspace/yanglint.deb']
   waitFor: ['validator prep']
   id: 'yanglint prep2'
-- name: 'gcr.io/$PROJECT_ID/models-ci-image'
+- name: 'us-west1-docker.pkg.dev/$PROJECT_ID/models-ci/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', "/go/src/github.com/openconfig/models-ci/validators/yanglint/test.sh"]
   secretEnv: ['GITHUB_ACCESS_TOKEN']
@@ -190,7 +194,7 @@ steps:
   id: 'misc-checks'
 
 ############### OC-PYANG ###############
-- name: 'gcr.io/$PROJECT_ID/models-ci-image'
+- name: 'us-west1-docker.pkg.dev/$PROJECT_ID/models-ci/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', '/go/src/github.com/openconfig/models-ci/validators/oc-pyang/test.sh']
   secretEnv: ['GITHUB_ACCESS_TOKEN']
@@ -201,6 +205,7 @@ steps:
   - 'COMMIT_SHA=$COMMIT_SHA'
   - '_REPO_SLUG=$_REPO_SLUG'
   - 'BRANCH_NAME=$BRANCH_NAME'
+  - '_OC_PYANG_VERSION=$_OC_PYANG_VERSION'
   volumes:
   - name: 'gopath'
     path: /go
@@ -218,7 +223,7 @@ steps:
   - 'GOPATH=/go'
   waitFor: ['go path creation']
   id: 'goyang-ygot prep'
-- name: 'gcr.io/$PROJECT_ID/models-ci-image'
+- name: 'us-west1-docker.pkg.dev/$PROJECT_ID/models-ci/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', "/go/src/github.com/openconfig/models-ci/validators/goyang-ygot/test.sh"]
   secretEnv: ['GITHUB_ACCESS_TOKEN']
@@ -236,7 +241,7 @@ steps:
   id: 'goyang-ygot'
 
 ############### PYANG ###############
-- name: 'gcr.io/$PROJECT_ID/models-ci-image'
+- name: 'us-west1-docker.pkg.dev/$PROJECT_ID/models-ci/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', '/go/src/github.com/openconfig/models-ci/validators/pyang/test.sh']
   secretEnv: ['GITHUB_ACCESS_TOKEN']
@@ -254,7 +259,7 @@ steps:
   id: 'pyang'
 
 ############### PYANGBIND ###############
-- name: 'gcr.io/$PROJECT_ID/models-ci-image'
+- name: 'us-west1-docker.pkg.dev/$PROJECT_ID/models-ci/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', '/go/src/github.com/openconfig/models-ci/validators/pyangbind/test.sh']
   secretEnv: ['GITHUB_ACCESS_TOKEN']
@@ -272,7 +277,7 @@ steps:
   id: 'pyangbind'
 
 ############### COMPATIBILITY REPORT ###############
-- name: 'gcr.io/$PROJECT_ID/models-ci-image'
+- name: 'us-west1-docker.pkg.dev/$PROJECT_ID/models-ci/models-ci-image'
   entrypoint: 'bash'
   args: ['-c', '/go/src/github.com/openconfig/models-ci/validators/compat_report.sh']
   secretEnv: ['GITHUB_ACCESS_TOKEN']

doc/releases.md

@@ -1,17 +1,27 @@
 # OpenConfig Release Versioning
 
-## Rationale
-
-As the OpenConfig project matures and is adopted by more implementors and
-network operators, model changes are becoming increasingly frequent. This has
-made identifying compatible and well-known sets of models more difficult.
-Furthermore, uncontrolled and frequent breaking changes can create excessive
-burden on implementors and operators alike.
-
-This revision proposal aims to address these problems by introducing
-[semantic versioning](https://semver.org/) to the set of OpenConfig models as a
-whole along with some OpenConfig-specific guidelines. As a side note, these
-releases are compatible with the notion of
+## Background and Motivation
+
+While each individual OpenConfig model can be tagged with a semantic version
+(see [semver.md](semver.md)), models are often interdependent, or need to be
+used together, for example when managing a full device. It is therefore useful
+to define OpenConfig "releases" that contain a set of models that are designed
+to work together. This also enables tracking breaking changes at the repository
+level, as well as allowing public users to view and download tagged collections
+of self-consistent models (see also the description of
+[GitHub releases](https://help.github.com/articles/creating-releases/)).
+
+In light of the above, this proposal introduces tagging
+[semantic versions](https://semver.org/) to the set of all OpenConfig models as
+a whole along with some OpenConfig-specific guidelines. Each release is
+therefore the cumulative set of models committed to the master branch at a
+certain point in time, and is tied to a specific commit in the OpenConfig
+repository. YANG validators ensure that each release consists of collection of
+published OpenConfig models that work together: that is, interdependencies
+(e.g., imports, augments) and cross-references (e.g., leafrefs) are all
+resolved.
+
+As a side note, these releases are compatible with the notion of
 [YANG release bundles](https://github.com/openconfig/public/blob/master/release/models/catalog/openconfig-module-catalog.yang).
 
 ## Policy
@@ -43,15 +53,16 @@ releases are compatible with the notion of
     breaking change since it would cause featureprofile tests that test for the
     behaviour of the default value without explicitly setting this leaf to begin
     to fail. As a result, the pull request for this change is not merged until
-    the end of the quarter in December. In January 2022, a new release is
+    the end of the quarter in December. In January 2023, a new release is
     created, versioned `v3.0.0` containing this update.
 
 3.  Any non-backward compatible change for a pre-`v1.0.0` YANG module does NOT
     by itself necessitate a major revision change for the overall models
     repository.
 
 4.  [Patch releases](https://semver.org/#spec-item-6) may be created at any time
-    for backward compatible bug fixes.
+    for backward compatible bug fixes, or equivalently, where only patch number
+    increases occurred in models.
 
 5.  [Pre-releases](https://semver.org/#spec-item-9) may be created at anytime to
     quickly introduce new changes to the models. These are not intended to be
@@ -67,13 +78,20 @@ releases are compatible with the notion of
     This guideline may change once OpenConfig operators gain more experience
     managing breaking changes.
 
+7.  Release documentation should include the list of models and their version
+    numbers contained in the corresponding release.
+
 Each release `vx.y.z` (e.g. `v1.2.0`) MAY be given a name for easier human
 identification, e.g. "September 2022". A client can thus assert they are
 compatible with the "September 2022" release of OpenConfig. It's expected that
 vendors will have some deviations and augments from the baseline; further, some
 vendors may offer the ability to configure their NOS (network operating system)
 to support different releases of https://github.com/openconfig/public.
 
+Note that release version numbers need not be a function of individual model
+release numbers (e.g., the max version number of all of the models in the
+release).
+
 ### Corner-Case Guidelines
 
 For non-backward compatible changes involving changing the type of a leaf, the

doc/semver.md

@@ -0,0 +1,113 @@
+# Versioning Individual OpenConfig models
+
+<em>contributors</em>: Anees Shaikh, Josh George, Rob Shakir, Kristian Larsson<br>
+
+## Background and Motivation
+
+*For versioning the set of all OpenConfig models as a whole, see
+[releases.md](releases.md).*
+
+This document proposes to adopt [Semantic Versioning](http://semver.org/)
+(semver) for published OpenConfig YANG models in the same way that software
+projects use similar versioning schemes to indicate the maturity and
+compatibility of software as it evolves. Semver bases its versioning on an API
+contract with developers and users. The basic format of a semver number is
+XX.YY.ZZ-release where XX is the major version number, YY is the minor version
+number, and ZZ is a patch level. Additional information can be optionally
+provided with a suffix. Detailed specification on the semver versioning rules
+are available at the [link](http://semver.org/) above. Any non
+backward-compatible change to the API requires incrementing the major version
+number, while feature changes that do not break clients are indicated by
+incrementing the minor version number. Non-feature patches that are backward
+compatible are indicated with an increment to the patch number.
+
+Semantic versioning is proposed as an addition to YANG revision statements for a
+number of reasons:
+
+*   YANG language rules state that the API never changes in a
+    backward-incompatible way. From RFC 6020: “... changes are not allowed if
+    they have any potential to cause interoperability problems between a client
+    using an original specification and a server using an updated
+    specification.”
+
+This is simply not practical (and is largely motivated by SNMP MIB notions).
+YANG models are not mature (less than 5 models have been made IETF RFCs and
+these are not implemented by any major device platform). Server and client
+implementations are only now being developed and deployed and significantly more
+operational experience is needed before APIs can be frozen.
+
+*   YANG revision statements consist of a date and some informational text. As
+    such, they offer little information about what has changed from one revision
+    to the next. This is perhaps not surprising when considering the rigid rules
+    in YANG about guaranteed API compatibility.
+*   YANG revision statements are meant for human consumption -- they are not
+    very useful for any sort of programmatic dependency checking.
+
+Semantic versioning has its
+[own issues](https://gist.github.com/jashkenas/cbd2b088e20279ae2c8e) and it may
+be that in OpenConfig we will have to adapt the specification somewhat based on
+considerations for versioning YANG models. Also semver does not address the
+problem of how to version groups of interdependent modules (e.g., a device model
+composed of many constituent models).
+
+Note that we would continue to use revision statements, e.g., with a date set to
+the day a new semantic version is published. This allows consumers to continue
+to use current YANG constructs such as import by revision.
+
+## General guidelines for versioning OpenConfig YANG modules
+
+An immediate question that arises when considering how to version YANG modules
+is what criteria should be used to judge that a module is mature enough that an
+API contract should be established with a version number.
+
+According to the semver specification, software that is pre-release with major
+version 0 may break clients as long as the major version number remains < 1.
+That is, with major version 0, there should be no expectation of compatibility
+from one release to another, even if only the minor version number is changing.
+
+Based on these considerations, the following basic guidelines are proposed for
+versioning OpenConfig modules:
+
+*   All modules should start out with a 0 major number. The major number should
+    remain 0 as long as the model is being reviewed and revised based on
+    feedback from the OpenConfig operators and from vendors implementing the
+    model.
+*   Semver guidelines should be followed while the model is at major number 0,
+    i.e., API or feature changes should increment the minor number, while minor
+    fixes should increment the patch number.
+*   Once a vendor implementation for a model is in progress, the major number
+    should be changed to 1 to acknowledge that the API is being used by
+    implementors with correspondingly more disruption likely when the model
+    changes in incompatible ways. Deciding that vendor implementations are
+    sufficiently in-progress to justify moving to major version 1 may be
+    somewhat subjective and should be based on detailed discussions with
+    implementors to understand what stage they are in their implementations.
+
+## API changes in YANG modules
+
+For the purposes of semver, the API presented by a YANG model consists of its
+data nodes and corresponding paths. Other elements of the model may not,
+strictly speaking, be considered part of the API, but still could have
+significant impact on the use of the model by developers or clients. Such
+elements include default values, configurability of a node, and behavior of a
+given data node (as described by the description statement).
+
+Since the API of the YANG module is a combination of these explicit and implicit
+elements, the criteria for determining when a revision requires a major number
+increment is not always straightforward. Below we list some general rules for
+determining the API has changed, and consequently would increment the major
+version number.
+
+*   Any leaf, leaf-list, list, or container modifications that result in
+    changing an existing data node name, or the path to a data node (location in
+    the model)
+*   Changing the target of a leafref
+*   Removal of a data node (leaf, leaf-list, list, container)
+*   Changing the type of a leaf or leaf-list
+*   Changing a type definition such that data based on the existing typedef
+    would be invalid (e.g., removing a value from an enumeration, changing the
+    base type in a typedef, etc.)
+*   Changing the key of a list (i.e., using a different data node as the list
+    key)
+*   Changing a conditional statement, such as when or must, to be more
+    restrictive, or to be based on a different condition altogether