make debugging sytem storage users miconfiguration easier

[butonic]

While we were debugging an upgraded ocis kubernetes deployment it took a while to notice that the OCIS_SYSTEM_USER_ID variable, filled by the storage system secret config map user-id key had changed because it was accidentially deleted when changing the ocis image.

The storage system userid is persisted in the settings storage with the grant. This can be used to fix access by dumping the root node metadata, extracting the user id from the grant and then putting it back into the config map.

However, finding the cause is harder. The only thing in the logs is a error initializing metadata client with a not found: f1bdd61a-da7c-49fc-8203-0558109d1b4f!f1bdd61a-da7c-49fc-8203-0558109d1b4f/settings error in the settings service like this:

{
  "level": "error",
  "service": "ocis",
  "error": "error: not found: create container: error: not found: f1bdd61a-da7c-49fc-8203-0558109d1b4f!f1bdd61a-da7c-49fc-8203-0558109d1b4f/settings",
  "time": "2024-05-16T05:50:52Z",
  "line": "github.com/owncloud/ocis/v2/services/settings/pkg/store/metadata/store.go:70",
  "message": "error initializing metadata client",
}

The not found error is created in the decomposedfs when trying to create a directory:

	rp, err := fs.p.AssemblePermissions(ctx, n)
	switch {
	case err != nil:
		return err
	case !rp.CreateContainer:
		f, _ := storagespace.FormatReference(ref)
		if rp.Stat {
			return errtypes.PermissionDenied(f)
		}
		return errtypes.NotFound(f) 		// <- this is the origin of the error
	}

The line is triggered because the user id used by the settings service does not match the persisted grant on the metadata space.

The settings service tries to initialize the settings store like this:

// we need to lazy initialize the MetadataClient because metadata service might not be ready
func (s *Store) initMetadataClient(mdc MetadataClient) error {
	ctx := context.TODO()
	err := mdc.Init(ctx, settingsSpaceID)
	if err != nil {
		return err
	}

	for _, p := range []string{
		rootFolderLocation,
		accountsFolderLocation,
		bundleFolderLocation,
		valuesFolderLocation,
	} {
		err = mdc.MakeDirIfNotExist(ctx, p)
		if err != nil {
			return err
		}
	}
        // [...]

Hm, the mdc.Init() call makes a CreateStorageSpace request ... that returns ok ... then the first MakeDirIfNotExist call tries to create the settings subfolder, which returns this error.

But since the user id is different the user has no permission to create a folder or stat the parent , so we get a not found error. BTW, we hide the permission denied error in order to not expose the existence of a resource for security reasons.

1. shouldn't the CreateStorageSpace / Init request fail with an already exists error?
2. when it fails should we double check we have access permissions?
3. should the init probably ListStorageSpace with the right filter to see if it exists and has the correct permissions?
4. if permissions are off die? at least log an error? don't become ready? well ... the metadata client is initialized lazy so ... use the metadata client init check to determine ready stat on service start?
5. if the space does not exist create it