doc: Alert that plugins run as separate processes, when turning on debug logs

[mateusoliveira43]

Thank you for contributing to Velero!

Please add a summary of your change

Add alert in docs about plugins behaviors. This already has come up mistakenly in an issue #5115

Also, this has been already explained in some issues:

• GCP plugin exits silently and leaves orphan snapshots behind #5641 (comment)
• Error message not logged from plugin #5205 (comment)

Does your change fix a particular issue?

No

Please indicate you've done the following:

• Accepted the DCO. Commits without the DCO will delay acceptance.
• Created a changelog file or added /kind changelog-not-required as a comment on this pull request.
• Updated the corresponding documentation in site/content/docs/main.

[mateusoliveira43]

/kind changelog-not-required

[codecov]

Codecov Report

Merging #6882 (42eb4fe) into main (563f1cc) will decrease coverage by 0.10%.
Report is 12 commits behind head on main.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #6882      +/-   ##
==========================================
- Coverage   60.76%   60.67%   -0.10%     
==========================================
  Files         247      249       +2     
  Lines       26306    26476     +170     
==========================================
+ Hits        15985    16064      +79     
- Misses       9186     9268      +82     
- Partials     1135     1144       +9     

see 18 files with indirect coverage changes

[draghuram]

Are we really sure that this error is benign? My understanding is that Velero server process starts each plugin as a separate process when it comes up and the plug-in processes keep running. If they exit, it must be due to some problem. But it is possible I am missing something here and it will be great if some one clarifies the exact lifecycle of plugin processes.

BTW, "plugins run as separate processes" is preferable to "run as binaries".

[mateusoliveira43]

I will make that change in the next commit, thanks

As an example, every minute Velero checks if Backup Storage Location is available (marking as valid, invalid or unknown). Each time it finishes the check, you are able to see this message (if log level is debug).

[draghuram]

Interesting. @sseago @shubham-pampattiwar Can you please shed some light on how Velero starts plug-ins? I think it uses https://github.com/hashicorp/go-plugin and I thought that the plug-in gRPC server will be up and running all the the time just like Velero server. But it doesn't seem to be the case?

[mateusoliveira43]

You are correct @draghuram, the exactly line the message is triggered https://github.com/hashicorp/go-plugin/blob/c57883964917bfb33721f5c5c5c8eeb98fe933e7/grpc_stdio.go#L142

[sseago]

@draghuram In my experience, when turning Debug on, we see these plugin process exit messages after a reconcile. So for BSL validation, it exits after checking once. For running a backup, all of the plugin invocations for a given BackupItemAction plugin are invoked on the same plugin process, but once backup processing completes, the plugin exits.

[draghuram]

Got it, thanks.