Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: VFolder trash bin #835

Merged
merged 68 commits into from
Oct 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
65f1a8b
impl minimal vfolder trash bin
fregataa Oct 24, 2022
4c079e3
add news fragment
fregataa Oct 24, 2022
a7a9bd7
remove duplicate tasks in db migration
fregataa Oct 24, 2022
8ecb348
impl client part
fregataa Oct 24, 2022
ab61644
cleanup delete, purge and recover methods
fregataa Oct 24, 2022
092c2e1
fix migration
fregataa Oct 25, 2022
4fd6dd7
remove PURGE_COMPLETE status which means deleted vfolders
fregataa Oct 25, 2022
4b5352d
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Nov 15, 2022
e056248
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Nov 15, 2022
44e7f7d
add empty_trash_bin and list_trash_bin API
fregataa Nov 15, 2022
4a2495d
impl trash bin checker by GlobalTimer
fregataa Nov 15, 2022
5a96cc3
fix wrong interval second config
fregataa Nov 15, 2022
ed8e5c1
check max-vfolder-count in recover
fregataa Nov 22, 2022
83b1525
impl config and do purge on conditions
fregataa Nov 22, 2022
2f32589
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Nov 22, 2022
fc5a5da
impl move_to_trash method
fregataa Nov 22, 2022
86e63d0
delete vfolder by result status
fregataa Nov 23, 2022
871f762
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Nov 23, 2022
226c92e
refactor methods
fregataa Nov 23, 2022
d3760a3
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Nov 23, 2022
cd462a4
merge main and update migration
fregataa Nov 23, 2022
d493eff
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Nov 25, 2022
5f0fdd9
change default trash path to root mount path
fregataa Nov 25, 2022
6bd7edf
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Dec 4, 2022
b8afe69
skip when usage-trash-bin is false
fregataa Dec 4, 2022
367c329
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Dec 4, 2022
11165b2
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Jan 3, 2023
21dc76b
fix news fragment
fregataa Jan 3, 2023
36866df
migrate use-trash-bin from local toml to etcd
fregataa Jan 3, 2023
5b9f1ca
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Feb 2, 2023
65ac1b7
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Feb 28, 2023
221b984
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Mar 2, 2023
70f1c73
merge main and update only vfolder status when deleting
fregataa Mar 2, 2023
f4247d8
fix lint
fregataa Mar 2, 2023
084b6f7
update outdated codes and docs
fregataa Mar 2, 2023
941fde6
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Mar 5, 2023
8d64010
return if deleted vfolder's parent is not dir
fregataa Mar 5, 2023
01d0f6b
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Mar 14, 2023
364a972
revert storage APIs
fregataa Mar 14, 2023
cf94131
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Jun 12, 2023
ac18482
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Jun 12, 2023
ee76cf3
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Jul 6, 2023
f765214
refactor delete function
fregataa Jul 6, 2023
4338a30
update migration file
fregataa Jul 6, 2023
0e1ddff
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Aug 9, 2023
ef1d187
update migration and minor fix
fregataa Aug 9, 2023
dcc0fd1
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Aug 18, 2023
e21044e
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Aug 30, 2023
857dd04
update migration
fregataa Aug 30, 2023
125c414
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Sep 4, 2023
cbaed55
purge vfolder one by one
fregataa Sep 4, 2023
750217e
update alembic migration
fregataa Sep 4, 2023
dbfc7d9
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Sep 6, 2023
8c3bea4
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Sep 7, 2023
5d8397a
update alembic migration
fregataa Sep 7, 2023
7036160
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Sep 13, 2023
49b0baa
Merge branch 'main' into feature/vfolder-trash-bin
achimnol Sep 17, 2023
33a7869
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Sep 19, 2023
27868b0
Merge remote-tracking branch 'origin/feature/vfolder-trash-bin' into …
fregataa Sep 19, 2023
409dcfa
update alembic migration
fregataa Sep 19, 2023
fb1f803
revert initate_vfolder_purge to get list of deletion info
fregataa Sep 19, 2023
86111bb
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Sep 27, 2023
774216c
Merge branch 'main' into feature/vfolder-trash-bin
fregataa Sep 27, 2023
c6d2fae
Merge branch 'main' into feature/vfolder-trash-bin
kyujin-cho Oct 4, 2023
0136587
merge diverged migration branches
kyujin-cho Oct 4, 2023
7218ccd
fix typo
kyujin-cho Oct 4, 2023
9f47e5c
update comments, remove unused codes
fregataa Oct 4, 2023
bb0cb36
Merge branch 'main' into feature/vfolder-trash-bin
kyujin-cho Oct 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changes/835.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add vfolder purge API for permanent vfolder removal and change original vfolder delete API to update vfolder status only.
37 changes: 36 additions & 1 deletion src/ai/backend/client/cli/vfolder.py
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,9 @@ def create(name, host, group, host_path, usage_mode, permission, quota, cloneabl
@vfolder.command()
@click.argument("name", type=str)
def delete(name):
"""Delete the given virtual folder. This operation is irreversible!
"""Delete the given virtual folder.
This operation can be retracted by
calling `recover()`.

\b
NAME: Name of a virtual folder.
Expand All @@ -180,6 +182,38 @@ def delete(name):
sys.exit(ExitCode.FAILURE)


@vfolder.command()
@click.argument("name", type=str)
def purge(name):
"""Purge the given virtual folder. This operation is irreversible!

NAME: Name of a virtual folder.
"""
with Session() as session:
try:
session.VFolder(name).purge()
print_done("Purged.")
except Exception as e:
print_error(e)
sys.exit(ExitCode.FAILURE)


@vfolder.command()
@click.argument("name", type=str)
def recover(name):
"""Recover the given virtual folder from deleted status.

NAME: Name of a virtual folder.
"""
with Session() as session:
try:
session.VFolder(name).recover()
print_done("Recovered.")
except Exception as e:
print_error(e)
sys.exit(ExitCode.FAILURE)


@vfolder.command()
@click.argument("old_name", type=str)
@click.argument("new_name", type=str)
Expand Down Expand Up @@ -220,6 +254,7 @@ def info(name):
print("- Number of files: {0}".format(result["numFiles"]))
print("- Ownership Type: {0}".format(result["type"]))
print("- Permission:", result["permission"])
print("- Status:", result["status"])
print("- Usage Mode: {0}".format(result.get("usage_mode", "")))
print("- Group ID: {0}".format(result["group"]))
print("- User ID: {0}".format(result["user"]))
Expand Down
13 changes: 13 additions & 0 deletions src/ai/backend/client/func/vfolder.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
_default_list_fields = (
vfolder_fields["host"],
vfolder_fields["name"],
vfolder_fields["status"],
vfolder_fields["created_at"],
vfolder_fields["creator"],
vfolder_fields["group_id"],
Expand Down Expand Up @@ -238,6 +239,18 @@ async def delete(self):
async with rqst.fetch():
return {}

@api_function
async def purge(self):
rqst = Request("POST", "/folders/{0}/purge".format(self.name))
async with rqst.fetch():
return {}

@api_function
async def recover(self):
rqst = Request("POST", "/folders/{0}/recover".format(self.name))
async with rqst.fetch():
return {}

@api_function
async def rename(self, new_name):
rqst = Request("POST", "/folders/{0}/rename".format(self.name))
Expand Down
1 change: 1 addition & 0 deletions src/ai/backend/client/output/fields.py
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,7 @@
FieldSpec("status"),
FieldSpec("unmanaged_path"),
FieldSpec("usage_mode"),
FieldSpec("status"),
FieldSpec("permission"),
FieldSpec("ownership_type"),
FieldSpec("max_files"),
Expand Down
Loading