Need to provide some kind of guidance to COs about when/how ListSnapshots should be used

[bswartz]

The definition of ListSnapshots says that it can potentially return ALL the snapshots on a controller, including snapshots not created through CSI. It's relatively common for storage devices to automatically take snapshots on some kind of schedule (every hour, for example) which can cause the volume of data in this API to be huge.

We should strongly recommend the use of filters when calling the API, to avoid causing performance issues on the client or server or both. Also, we should recommend that COs not poll for snapshots regularly because the amount of data churn would be very large and we know that 99% of automatically created snapshots get destroyed without anyone ever caring they existed. The value of the automatically created snapshots is in that 1% case when they are needed, and the ListSnapshots API should be used in those specific cases.

It's up the the COs to figure out what those workflows look like, but we should recommend that they not naively try to track every snapshot that CSI knows about.

For reference:

• https://docs.google.com/document/d/1-oiNg5V_GtS_JBAEViVBhZ3BYVFlbSz70hreyaD7c5Y/edit#heading=h.5piyjba14wsr