Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dpat 1749 #1212

Merged
merged 10 commits into from
Oct 19, 2023
Merged

Dpat 1749 #1212

merged 10 commits into from
Oct 19, 2023

Conversation

michaeljcollinsuk
Copy link
Contributor

@michaeljcollinsuk michaeljcollinsuk commented Oct 11, 2023

📝 Summary

This PR resolves ministryofjustice/analytical-platform#1749

This PR replaces S3Bucket deletion with a soft-delete action. This means that the database object is retained, but marked as deleted. This will allow us to capture details of who has deleted the bucket and when. Will also allow possibility of restoring S3 buckets in future.

In the immediate term, it means that work on ministryofjustice/analytical-platform#1887 can begin.

When the S3bucket is soft deleted:

  • It is marked as deleted, storing the user that deleted and the time of deletion
  • The actual S3 bucket in AWS is deleted in the same manner as before
  • All related S3 access objects also remain, since there is no cascading delete. However all access is revoked as before, which means that IAM roles are updated
  • A superuser can still access the deleted bucket in control panel. The template has been updated to indicate deletion details. In future, it would be possible to implement a "restore S3 bucket" button here.

As part of the changes, I also fixed some bugs within the template for all datasources - such as displaying the type of the bucket (warehouse or app).

🔍 What should the reviewer concentrate on?

  • Changes to the model
  • Changes to the deletion process.
  • Task to update IAM roles to revoke access
  • Changes to the template

🧑‍💻 How should the reviewer test these changes?

  • Run the server and the celery worker
  • Log in, create an S3 bucket (or use existing one) and grant yourself access. Click the "Warehouse data" tab - the bucket will be visible to you as before
  • Go to the manage datasource page, and click the "delete bucket" button - the object should not be displayed on the "Warehouse data" page.
  • Now go to "All datasources" page that is only accessible to superusers, scroll to the bottom and the bucket should be listed under a new section "Deleted datasources".
  • Click the manage datasource button - this is only accessible to superusers, and should display details of the deleted bucket
  • To check that your users IAM role has been updated to remove access to the bucket, by logging into your admin AWS account, finding your AWS role, and checking it is not listed in the S3 section

📚 Documentation status

  • No changes to the documentation are required
  • This PR includes all relevant documentation
  • Documentation will be added in the future because ... (see issue #...)

…to call it

Changes view to use form_valid rather than delete to handle deletion, as this
was a change introduced in django 4.0. See docs/release notes for full info.
- For admins, list deleted datasources
- Fix bugs in templates so that admin view displays datasource type
@michaeljcollinsuk michaeljcollinsuk marked this pull request as ready for review October 13, 2023 14:51
{% if request.user.is_superuser and deleted_datasources %}
<h3 class="govuk-heading-m">Deleted data sources</h3>
{{ datasource_list(deleted_datasources, datasource_type|default(""), request.user) }}
{% endif %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea to show this on admin page on a separated group

# TODO when soft delete is added, this should be updated to use the user that
# has deleted the parent S3bucket to ensure we store the user that has sent the
# task in the case of cascading deletes
tasks.S3BucketRevokeUserAccess(self, self.current_user).create_task()
revoked_by = revoked_by or self.current_user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above about revoked_by 😊 , happy to discuss it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH it was just so that I update my test to check that the task is called with the correct user... was difficult to test this by assigning it directly to the instance as current_user. Also felt it added a bit more flexibility, but maybe not necessary. And updating the code just for the test also isnt ideal.... i'll take another look and simplify

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, hmm, passing user info by this way (I meant as curent_user) is something I feel always is not very clean solution , but haven't got better approach 😊

Copy link
Contributor

@ymao2 ymao2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nice work to get the soft-delete feature added to S3 bucket which make the activity more trackable. 👍 👍

@michaeljcollinsuk michaeljcollinsuk merged commit 118b2e8 into main Oct 19, 2023
4 checks passed
@michaeljcollinsuk michaeljcollinsuk deleted the DPAT-1749 branch October 19, 2023 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

✨ Implement soft-delete for S3 datasources
2 participants