Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Unable to backup backing image through Longhorn UI #10023

Open
yangchiu opened this issue Dec 20, 2024 · 13 comments · May be fixed by longhorn/longhorn-manager#3392
Open

[BUG] Unable to backup backing image through Longhorn UI #10023

yangchiu opened this issue Dec 20, 2024 · 13 comments · May be fixed by longhorn/longhorn-manager#3392
Assignees
Labels
area/backing-image Backing image related area/ui UI related like UI or CLI area/volume-backup-restore Volume backup restore kind/bug priority/0 Must be implement or fixed in this release (managed by PO) reproduce/always 100% reproducible require/backport Require backport. Only used when the specific versions to backport have not been definied. severity/1 Function broken (a critical incident with very high impact (ex: data corruption, failed upgrade)
Milestone

Comments

@yangchiu
Copy link
Member

yangchiu commented Dec 20, 2024

Describe the bug

Trying to backup backing image through Longhorn UI on v1.8.x-head (longhorn-ui 4f6b1eb) or master-head (longhorn-ui cf5fcff), got 500 Internal Server Error:

backup-button

POST http://54.209.76.137:30000/v1/backingimages/test?action=backupBackingImageCreate
{"actions":{},"code":"Internal Server Error","detail":"","links":{"self":"http://54.209.76.137:30000/v1/backingimages/test"},"message":"EOF","status":500,"type":"error"}

Both v1 and v2 backing image have the same issue.

To Reproduce

Backup backing image through Longhorn UI

Expected behavior

Support bundle for troubleshooting

Environment

  • Longhorn version: master-head and v1.8.x-head
  • Impacted volume (PV):
  • Installation method (e.g. Rancher Catalog App/Helm/Kubectl):
  • Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version:
    • Number of control plane nodes in the cluster:
    • Number of worker nodes in the cluster:
  • Node config
    • OS type and version:
    • Kernel version:
    • CPU per node:
    • Memory per node:
    • Disk type (e.g. SSD/NVMe/HDD):
    • Network bandwidth between the nodes (Gbps):
  • Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal):
  • Number of Longhorn volumes in the cluster:

Additional context

Workaround and Mitigation

@yangchiu yangchiu added kind/bug area/ui UI related like UI or CLI severity/1 Function broken (a critical incident with very high impact (ex: data corruption, failed upgrade) reproduce/always 100% reproducible priority/0 Must be implement or fixed in this release (managed by PO) area/volume-backup-restore Volume backup restore area/backing-image Backing image related require/backport Require backport. Only used when the specific versions to backport have not been definied. labels Dec 20, 2024
@yangchiu yangchiu added this to the v1.8.0 milestone Dec 20, 2024
@github-project-automation github-project-automation bot moved this to New Issues in Longhorn Sprint Dec 20, 2024
@derekbit
Copy link
Member

cc @houhoucoop

@houhoucoop
Copy link

I can see 500 when attempting to back up the same image twice.
"message": "failed to create backup backing image 'parrot': backup backing image 'parrot' already exists"

@houhoucoop
Copy link

houhoucoop commented Dec 20, 2024

Since it's a 500 error, I think the root cause is likely on the backend side.

@yangchiu
Copy link
Member Author

I can see 500 when attempting to back up the same image twice. "message": "failed to create backup backing image 'parrot': backup backing image 'parrot' already exists"

It doesn't seem to be what I encountered. Also I observed inconsistent behavior when trying to back up from the following two locations. Could you try both of them?

backup-button

backup-menu

@houhoucoop
Copy link

houhoucoop commented Dec 20, 2024

These are all the cases I have tested, not sure which ones are expected and which are defects.

Case 1: One backup target: default

  • Backup single backing image
    • If the backup exists: 500 (backup already exists)
    • If the backup does not exist: 200 (success)

Case 2: Multiple backup targets: default, cifs

  • Backup the same backing image
    • Select backup target default: 200 (success)
    • Select backup target cifs 500 (backup already exists)

Case 3: Bulk backup action:

  • Attempt bulk backup for one or multiple backing images
    • Select one backing image 500 (EOF)
    • Select multiple backing images 500 (EOF)

@houhoucoop
Copy link

houhoucoop commented Dec 20, 2024

cc @a110605 @mantissahz ^

@houhoucoop houhoucoop moved this from New Issues to Analysis and Design in Longhorn Sprint Dec 23, 2024
@houhoucoop
Copy link

@mantissahz Can you confirm if Case 2 is a defect? frontend needs to make corresponding UI adjustments.

@mantissahz
Copy link
Contributor

mantissahz commented Dec 23, 2024

@mantissahz Can you confirm if Case 2 is a defect? frontend needs to make corresponding UI adjustments.

It is not allowed because UI creates the BackupBackingImage of a backing image using the same BackupBackingImage name (the backing name) for different backup targets.

  1. UI can allow users to fill in a custom name when (and passed it by the payload) creating a backup backing image.
    https://github.com/longhorn/longhorn-manager/blob/5187ffde3c6c2d009b5406db7193966463505b46/api/model.go#L180-L196.

  2. or the Backend can use a random name to create the BackupBackingImage, then UI displays the
    name of a BackupBackingImage by the backing image name in the response payload.

@houhoucoop
Copy link

@mantissahz Can you confirm if Case 2 is a defect? frontend needs to make corresponding UI adjustments.

It is not allowed because UI creates the BackupBackingImage of a backing image using the same BackupBackingImage name (the backing name) for different backup targets.

  1. UI can allow users to fill in a custom name when (and passed it by the payload) creating a backup backing image.
    https://github.com/longhorn/longhorn-manager/blob/5187ffde3c6c2d009b5406db7193966463505b46/api/model.go#L180-L196.
  2. or the Backend can use a random name to create the BackupBackingImage, then UI displays the
    name of a BackupBackingImage by the backing image name in the response payload.

I see, I prefer option 2.
After checking the frontend code, we don't generate UUIDs, so generating them on backend may ensure more consistency. What do you think?

@mantissahz
Copy link
Contributor

I see, I prefer option 2.
After checking the frontend code, we don't generate UUIDs, so generating them on backend may ensure more consistency. What do you think?

The backend can handle that if the name in the request payload is empty or equal to the backing image name, it will generate a random name to create a backup of the backing image.

The UI can still allow users to have a costume BackupBackingName name when creating a backup of the backing image if you want to.

@houhoucoop houhoucoop assigned a110605 and unassigned houhoucoop Dec 23, 2024
@ChanYiLin ChanYiLin self-assigned this Dec 23, 2024
@ChanYiLin
Copy link
Contributor

ChanYiLin commented Dec 23, 2024

I will fix the backend part tomorrow

@ChanYiLin ChanYiLin moved this from Analysis and Design to Review in Longhorn Sprint Dec 23, 2024
@longhorn-io-github-bot
Copy link

longhorn-io-github-bot commented Dec 23, 2024

Pre Ready-For-Testing Checklist

  • Where is the reproduce steps/test steps documented?
    The reproduce steps/test steps are at:

All operations can be done via UI

  1. Create a parrot BackingImage
  2. Setup two different BackupTarget: s3 and nfs
  3. Create two backups from the UI with two different BackupTargets
  4. Two BackupBackingImages are created successfully with parrot-XXXXX
  5. Delete the parrot BackingImage
  6. Restore the BackupBackingImage (Both should work)
  7. parrot is restored.

PR:

@derekbit
Copy link
Member

derekbit commented Dec 23, 2024

Recent issues are mostly related to the APIs and the integration between the backend and frontend.

@houhoucoop @a110605 In addition to implementing the UI part, remember to self-validate if the functionalities work before moving it to ready-for-testing. This will make you familiar with Longhorn more.

cc @a110605 @ChanYiLin @mantissahz @innobead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backing-image Backing image related area/ui UI related like UI or CLI area/volume-backup-restore Volume backup restore kind/bug priority/0 Must be implement or fixed in this release (managed by PO) reproduce/always 100% reproducible require/backport Require backport. Only used when the specific versions to backport have not been definied. severity/1 Function broken (a critical incident with very high impact (ex: data corruption, failed upgrade)
Projects
Status: Review
Development

Successfully merging a pull request may close this issue.

7 participants