-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create User Dashboard #1721
Create User Dashboard #1721
Conversation
a8ba10b
to
9dd15f9
Compare
9dd15f9
to
aa624cf
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, I think it looks good. I would encourage us to consider changing the URL, as mentioned above. Also, let's remove the overridden User.__init__
method unless it's to be used.
cadasta/accounts/models.py
Outdated
@@ -75,6 +75,10 @@ class TutelaryMeta: | |||
{'error_message': | |||
_("You don't have permission to update user details")})] | |||
|
|||
def __init__(self, *args, **kwargs): | |||
super().__init__(*args, **kwargs) | |||
self.__original_avatar_url = self.avatar_url |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the purpose of caching the avatar URL like this? I'm guessing that we're planning on doing something if the user is saved and the avatar changes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@alukach you're right. I forgot to delete this after the Add feature to create user avatar thumbnail #1702 got blocked. The caching was useful if the thumbnail functionality was going to be merged -- in that way, this would have allowed us to check if the the avatar.url changed (and therefore, if we had to create a new thumbnail). I'm going to delete these lines, I just forgot to delete them in the past :)
cadasta/accounts/urls/default.py
Outdated
@@ -14,4 +14,6 @@ | |||
name="account_reset_password_from_key"), | |||
url(r'^password/reset/$', default.PasswordResetView.as_view(), | |||
name="account_reset_password"), | |||
url(r'^account-projects/$', default.AccountListProjects.as_view(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I question this URL name. It reflects that the view is only about showing a user's projects but is that possibly short-sighted? If this is to be a dashboard, then I'd expect that I could do other things (e.g. see the statuses of my background tasks).
Why not /account/dashboard/
?
@alukach in my last commit I addressed both your points and I added 2 simple property functions that do the same as |
07d76c6
to
444e28d
Compare
cadasta/accounts/models.py
Outdated
@@ -53,6 +53,8 @@ class User(auth_base.AbstractBaseUser, auth.PermissionsMixin): | |||
objects = UserManager() | |||
|
|||
history = HistoricalRecords() | |||
_dict_languages = dict(settings.LANGUAGES) | |||
_dict_measurements = dict(settings.MEASUREMENTS) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This feels like we're complicating things a bit. Casting a list of tuples to a dict isn't that expensive, we may as well do it in the <language/measurement>_verbose
properties. If performance is an issue, we would do better to create a LANGUAGE_DICT
property in the file itself so it would only be ran once per lifetime of app.
I'm fine with either way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll move all inside the respective property functions.
cadasta/accounts/views/default.py
Outdated
elif not org.archived: | ||
projects = list( | ||
AccountListProjects._get_unarchived_projects(org, user)) | ||
yield (org, is_admin, projects) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm concerned about all of the nested queries in this method. We lookup all the related orgs, then for every org we do a lookup for the user's org role, then either a) if the org role == 'admin', do a lookup for all of the org's projects; or b) if org role != admin we lookup all of the org's unarchived projects and then for every one of those projects we do a lookup for the user's project role.
This seems a bit inefficient. Admittedly, I don't know how likely it is for customers to be part of many organizations but I could see this causing loadtime issues for members of the Programs team. As a general rule of thumb, I find that if we're doing a query in a for
loop, there's probably room for improvement.
I wanted to sketch out the beginning of how we could improve this (and let you finish the rest), however I found it to be pretty complicated. I ended up coming up with something that I think works, however it's not pretty:
@staticmethod
def _get_orgs(user):
from itertools import groupby
# Let's find all the project roles for projects with an org where
# user as admin
all_projectroles_user_is_admin = user.projectrole_set.filter(
project__organization__organizationrole__admin=True,
project__organization__organizationrole__user=user
)
# Let's find all the project roles for unarchived projects with
# an unarchived org where user is not admin
unarchived_projectroles_user_is_not_admin = user.projectrole_set.filter(
project__organization__organizationrole__admin=False,
project__organization__organizationrole__user=user,
project__organization__archived=False,
project__archived=False,
)
# Combine into iterable
is_admin__qs = (
[True, all_projectroles_user_is_admin],
[False, unarchived_projectroles_user_is_not_admin]
)
for is_admin, qs in is_admin__qs:
qs = qs.prefetch_related('project', 'project__organization')
def get_org(pr): return pr.project.organization
for org, projectroles in groupby(qs, get_org):
yield (
org, is_admin,
[(pr.project, pr.role_verbose) for pr in projectroles]
)
# Finally, let's handle projects where there are no project roles
public_user_projects = Project.objects.filter(
organization__users=user
).exclude(projectrole__user=user).prefetch_related('organization')
def get_org(proj): return proj.organization
for org, projects in groupby(public_user_projects, get_org):
yield (
org, False, [(pr, _('Public User')) for pr in projects]
)
By doing a lookup on the ProjectRole
objects first (and running .prefetch_related('project', 'project__organization')
), we can get all of the user's project roles, projects, and organizations in a single shot. Unfortunately, because we need to know if the user is an admin
for each project role, I had to separate the queries for the admin and non-admin cases rather than combining them into a single query through the power of Django's Q
object (I couldn't figure out how to lookup the project role information after the cases were combined). Ultimately, this reduces the number of queries to a fixed amount of 3 queries, regardless of the user's numbers of organization or projects.
Take a look and let me know if you think this represents the same logic. Perhaps it can be made a bit prettier as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am glad you provided this solution because I was aware of the risks but I was not able to come up with a better solution myself. I'll integrate your solution in the next days once all the travel is done (tomorrow I fly to Europe!). Thanks for the useful review! 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, I just wanted to update on the fact that I'm working on this. Unfortunately, it looks like there's more work than I thought since your version does not cover all scenarios. I'm trying to make the changes and make it work as it should.
9777514
to
1cd4891
Compare
1cd4891
to
601b516
Compare
Hi @alukach, I have been working quite a bit on integrating your suggestion with the existing work for the user dashboard. I think I managed to make it work now, but I don't understand why a test keeps not passing. Could you help me to understand this? I want to point out some things I had to address while integrating your suggestion with my existing work:
I'd really appreciate if you could help me figure out what is causing the test to fail as soon as possible. Starting from Saturday 16th I'll be on holiday for weeks and won't be online for about a month. Cheers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work @laura-barluzzi , I think we're almost there. You've brought up a number of good points that I did not consider. However, I still think it is possible to avoid the lookups in a for-loop.
The first place we do a lookup in a for-loop is when dealing with the all_projects_where_user_has_projectrole
queryset:
yield (
org, is_admin,
[(proj, user.projectrole_set.get(
project=proj).role_verbose)
for proj in projects]
)
I would recommend refactoring the code so that we separate situations where we don't need to lookup Project Roles from situations where we do (i.e. don't handle them in the same for
loop). The logic to handle the Project Role lookup was included my earlier example. I think it could still apply. This is based on the assumption that the following is valid:
def project_based_lookup(user):
""" This is more or less what we currently have in the code, with lookups done in the for-loop """
qs = Project.objects.filter(
organization__organizationrole__user=user,
organization__organizationrole__admin=False,
organization__archived=False,
archived=False)
qs = qs.exclude(projectrole__role__isnull=True)
qs = qs.exclude(projectrole__role__exact='')
for proj in qs:
yield (proj, user.projectrole_set.get(project=proj).role_verbose)
def project_role_based_lookup(user):
""" Here, we are retrieving the same data but with a single query """
qs = user.projectrole_set.filter(
project__organization__organizationrole__admin=False,
project__organization__organizationrole__user=user,
project__organization__archived=False,
project__archived=False,
)
qs = qs.prefetch_related('project')
for pr in qs:
yield (pr.project, pr.role_verbose)
assert list(project_based_lookup(user)) == list(project_role_based_lookup(user))
The second place that we have a nested lookup is this final last-check for any organizations that we weren't able to find projects for:
if len(all_active_user_orgs) != 0:
for org in all_active_user_orgs:
yield (org, user.organizationrole_set.get(
organization=org).admin, [])
This is a pretty quick refactor:
if all_active_user_orgs: # this is a short-hand way to say len(all_active_user_orgs) != 0:
qs = user.organizationrole_set.filter(organization__in=all_active_user_orgs)
qs = qs.prefetch_related('organization')
for org_role in qs:
yield (org_rol.organization, org_rol.admin, [])
Although, I wonder if this is even necessary. Before this logic, we handle the following cases:
- all Projects where the User is an Org Admin
- all Projects where the User is not an Org Admin and there are no ProjectRoles
- all ProjectRoles for a User
What's left? I think it's just all Orgs where the user is a member and the Org has no Projects. Is this right? If so, we can just do:
qs = user.organizationrole_set.filter(projects=None)
qs = qs.prefetch_related('organization')
for org_role in qs:
yield (org_role.organization, org_role.admin, [])
However, if we're to follow the logic written in your code where we only display archived Orgs if the user is an Org Admin, we should implement the same logic here as well:
qs = user.organizationrole_set.filter(projects=None)
qs = qs.exclude(organization__archived=True, admin=False)
qs = qs.prefetch_related('organization')
for org_role in qs:
yield (org_role.organization, org_role.admin, [])
In the interest of time (knowing that you have holidays coming up and have already ended your internship), I have put together what I think incorporates all of your work and all of the above comments. A warning, I've been looking at this so closely that I may be missing the overall picture of what we're trying to do with _get_orgs()
, so please verify that I have a correct understanding of the rules around what should and should not be displayed.
@staticmethod
def _get_orgs(user):
# Handle Projects without in-db ProjectRoles
# - projects for which user is org admin (including archived projects
# and archived orgs)
org_admin_project = Project.objects.filter(
organization__organizationrole__user=user,
organization__organizationrole__admin=True
)
# - projects for which user is a project public user (excluding
# archived projects and archived orgs)
public_user_projects = Project.objects.exclude(
organization__organizationrole__admin=True,
organization__organizationrole__user=user
).filter(
organization__users=user,
organization__organizationrole__admin=False,
projectrole__role__isnull=True,
organization__archived=False,
archived=False,
)
is_admin__qs = (
[True, org_admin_project],
[False, public_user_projects]
)
for is_admin, qs in is_admin__qs:
qs = qs.prefetch_related('organization')
def get_org(proj): return proj.organization
for org, projects in groupby(qs, get_org):
role = _('Administrator') if is_admin else _('Public User')
yield (
org, is_admin, [(proj, role) for proj in projects]
)
# Handle Projects with in-db ProjectRoles
proj_roles = user.projectrole_set.filter(
project__organization__organizationrole__admin=False,
project__organization__organizationrole__user=user,
project__organization__archived=False,
project__archived=False,
)
proj_roles = proj_roles.prefetch_related('project')
proj_roles = proj_roles.prefetch_related('project__organization')
def get_org(pr): return pr.project.organization
for org, project_roles in groupby(proj_roles, get_org):
yield (
org, False,
[(role.project, role.role_verbose) for role in project_roles]
)
# Handle Orgs without Projects
org_roles = user.organizationrole_set.filter(projects=None)
org_roles = org_roles.exclude(organization__archived=True, admin=False)
org_roles = org_roles.prefetch_related('organization')
for org_role in org_roles:
yield (org_role.organization, org_role.admin, [])
This gets us down to a fixed number of 4 queries, regardless of the number of projects and organizations a user is part of.
cadasta/accounts/views/default.py
Outdated
projectrole__role__isnull=True, | ||
organization__archived=False, | ||
archived=False, | ||
).prefetch_related('organization') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm worried that this query logic may be wrong. I believe the first two lines of the query are saying "Project where a) user is member of organization; and b) where organization has no admin":
public_user_projects = Project.objects.filter(
organization__users=user,
organization__organizationrole__admin=False)
I think instead you want:
# Not Org Admin
public_user_projects = Project.objects.exclude(
organization__organizationrole__admin=True,
organization__organizationrole__user=user)
# Is Org public user (and org and project are active)
public_user_projects = public_user_projects.filter(
organization__users=user,
projectrole__role__isnull=True,
organization__archived=False,
archived=False,
).prefetch_related('organization')
cadasta/accounts/views/default.py
Outdated
all_projects_user_is_org_admin = Project.objects.filter( | ||
organization__organizationrole__user=user, | ||
organization__organizationrole__admin=True | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To double-check, we want to display archived Projects and Projects from archived Orgs if the user is an Org Admin?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, so far I have been doing it like this -- that archived projects and archived orgs are displayed only for org admin.
However, I took all these decisions alone so it's not a group-decision and I believe people may prefer changing this afterwards (e.g. adding a filter instead, or not displaying archived orgs).
However, this change can be done after this get merged as far as I'm concerned. I believe now we should focus on getting the existing functionalities working properly and merging everything. Once it'll be merged, people will developed a more informed opinion on these details and will open issues.
This said, now I'm showing archived orgs/projects to org admins.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 I think this is a reasonable approach.
Regarding failing tests, the actual output does not match the expect output. It's hard to see from the test output. For these kinds of things, I usually end up dropping a debugger statement into the test and examining what's going on. My favorite is Here is the actual and expected html output: https://gist.github.com/alukach/fd9c21e79a210f0cc1eca684d02707ce. I recommend downloading both files and opening them with your web browser. Ordering of organizations data is different and also the actual response does not group Project #0 and Project #1 under the same Organization |
@laura-barluzzi I know you're about to head off for vacation. How are you feeling about this? |
456a94e
to
8f1baf5
Compare
Hi @alukach, Today I worked on this and I believe everything is fine: 1. I fixed the problem with the Public User projects -- now they are always displayed even if other users are assigned project roles (such as 2. Now an organization gets only displayed once, even if the user has different roles for each project of that organization. Before, the test didn't pass (also) because when a user was member of Org1 and was assigned Org1.Proj1 3. Now the test reflects the existing order of Orgs and Projects displayed on the template. In my first version, I had total control on the order of organizations and projects being displayed due to the nested However, I noticed that the order of the data generated by the current approach is a bit random and sometime it changes... so sometimes the test doesn't pass :( Despite the randomized order of data displayed, all tests pass. I believe that is not possible to have less than 5 db calls. As you notice, I added a queryset in order to fix point 1 (properly querying Tomorrow I'll be starting my holiday so I'll be answering in about 2-3 weeks if anything needs to be changed. Either way, it'd be good if you could have a look on how to make sure 100% that the we can expect the order of the data passed to the context, so that we can make pass correctly 100% the test data into the test. |
8f1baf5
to
ab34b95
Compare
@laura-barluzzi Thanks so much for all the hard work you've put in! We still have some queries in a for loop, but those are easy to take out: # 2nd call: organizations where user is a member and not admin
user_orgs_not_admin = user.organizations.filter(
organizationrole__user=user,
organizationrole__admin=False,
archived=False,
).prefetch_related('projects').all()
# 3rd call: projects linked to the user
user_projects = user.project_set.all()
for org in user_orgs_not_admin:
for project in org.projects.all(): # <- These are new queries, despite the fact that we've prefetched projects!
if not project.archived and project not in user_projects:
public_user_projects.add(project) The logic to above is identical to the following: public_user_projects = Project.objects.filter(
organization__organizationrole__user=user,
organization__organizationrole__admin=False,
organization__archived=False,
archived=False,
).exclude(users=user) # This is basically saying "avoid projects that are in user.project_set.all()" To make tests a more consistent, we have two options:
I've gone ahead and made these changes and committed them to the branch. Enjoy your vacation! If you get a chance when you get back, feel free to double-check that the logic is sound. |
@seav - I think we should list all projects the user belongs too, even if only as a public user. Public users do get access to stats for locations, parties and resources that non-public users don't see. I can see someone using this to "watch" a project. |
Agree @seav but we are waiting on marketing to come up with messaging since we should do something for our users to notify them of the new page. I think this will be good to push for next sprint so we have time to coordinate. Update - I've made the change and I'll create some sort of messaging using inlinemanual (which won't require a code release). |
…ge and measurement which are translatable
Correcting misspellings Revert "Updated view name" This reverts commit 906cf57. Adjusted blocktrans tag Removed admin from orgs without projects Shouldn’t have to be an admin to see you are in an org without projects. Renamed AccountListProjects to UserDashboard Adjusting layout of starter text messages Removed add projects icon Added success url so returning users are directed to dashboard
bceb411
to
90d86eb
Compare
@clash99: Maybe there is something missing with our user roles and permissions, but any registered user who is not a project user or above is by default a public user of all public projects of all organizations. Shown below is a screenshot of a user who is a member of an org but is not assigned a project user role or above in any of the org's projects. The user is able to see basic statistics about the project: And shown below is a screenshot of a user who just has an account but is not a member of any org or project. As you can see, the user can still see statistics for a public project: Maybe if a user is a member of an org, we can list all of that org's projects in the user's dashboard, but we should probably indicate there that the user is not a project user (or above) for the project so that they won't get confused when they see the "Access to project data is restricted to current project members only" message. |
Not yet, as far as I know. IIRC, I think the plan is to eliminate this when we merge the registration-by-phone feature. |
While the I would expect multiple alert messages to be stacked vertically, like in the screenshot below: |
The permissions are working correctly. Public users see non-clickable stats, project members will see clickable green stats, and non-registered members will see the register/login option. I like your idea of adding more info/messaging for public users to avoid confusion on the project page. The user role for each project is displayed on their user dashboard but I think we should add that information to the project dashboard too. This way a user will see a "Your role/permission level is". I've created a new task for this ( #1825 ). This would pair perfectly with requesting access to the project ( #943 ) but would still be a value-add as a stand-alone. |
Good catch @seav. I don't think the message in the alert area should be used for longer messages with links so I've moved it to an alert-info box right next to the user account details. |
Let's ship this! And let's file bugs for the little things found after. :) |
I don't think we have any bugs, there is just more enhancements we would like to add :) |
@alukach thank you so much for taking ownership of this during my holiday :) I really appreciate! I learned quite a bit from you about db querying efficiency 👍 @seav and @clash99 thanks for spotting and improving the User Dashboard. I read everything and I agree with all points. I really like the new position of the alert message and I agree that this PR now is ready to be shipped so that the next bugs/improvements we'll be more evident to everybody just by using the User Dashboard. I just wanted to confirm @seav's answer regarding the following question:
I was told to remove the date from @valaparthvi because in her work (addition of a phone-authentication option) she removed the 24h limit to confirm the account. So we can keep it as it is now (without the check of 24h confirmation time) since we're expecting her work to be merged soon or later (if it's not already!). I am currently in Germany and on Wednesday 4th I fly and move to NYC -- so I'll be still on-and-off online this week. From next week I'll be online again :) |
Proposed changes in this pull request
This PR creates a User Dashboard (UD) as defined in the Epic issue User Dashboard #1491. Specifically it addresses these open issues: #1492, #1483, #1482, #1485, #610.
Added user functionalities:
user.verify_email_by
field){% url 'account_email' %}
)accounts/profile.html
)Added changes to implement the above functionalities:
accounts/urls/default.py
which brings to theAccountListProjects(ListView)
class where we refer to a new templateaccounts/user_dashboard.html
AccountListProjects(ListView)
I queried the db in order to access:user.organizations
and projects associated with each of those organizations;OrganizationRole.admin
andProjectRole.role
for each organization and project.Because of the nature of the information and the goal, in order to access data accordingly I structured the data as follows:
accounts/user_dashboard.html
I displayed a box titled PROJECTS with all the organizations and projects of the user. To do so, I simply made 2 iterations:accounts/user_dashboard.html
I displayed theuser.avatar
by applying the existing CSS used in the templateaccounts/profile.html
. In this way we mitigate the lack of an existingthumbnail
approach.verbose
value ofProjectRole.role
on the template, inorganization.models.ProjectRole
I added;_dict_role_choices = dict(ROLE_CHOICES)
shared by all instances (in this way the dictionary is created only once);@property
function that returns the verbose value given theProjectRole.role
key.organization/tests/test_models.py
andaccounts/tests/test_views_default.py
.accounts/user_dashboard.html
has the same base rules of both project and organization dashboards as defined incore/static/css/single.scss
Sample screenshot of Mario Rossi user dashboard
When should this PR be merged
Risks
Follow-up actions
Checklist (for reviewing)
General
migration
label if a new migration is added.Functionality
Code
Tests
Security
Documentation