Notification system for GraphSpace
GraphSpace is an user-friendly web-based platform that collaborating research groups can utilize for storage, interaction, and network sharing. A GraphSpace user can import graphs created in Cytoscape, upload them through a REST API, interact with them by customizing and saving layouts, share them within and between groups of collaborators, search for different graphs, and organize them using tags. These functions are enabled through GraphSpace’s comprehensive REST API, which allows users to communicate programmatically.
The goal of this project is to build a notification system for GraphSpace which will allow a user to track important activities on GraphSpace via notifications, the purpose being to keep users on par and current with their GraphSpace activity. In doing so, there will three type of notifications: Group, Owner and Watching.
Notifications are integral to all web applications as they are the means to tracking all important information. It is most optimal when a notification system is decoupled from the main application flow, because we can have a clear distinction between the utilities of the two applications. In this project, there are three primary types of notifications that need to be implemented: Owner, Group and Watching. A owner notification is defined by one where users automatically receive a notification when they create a resource (graph, layout or group) on GraphSpace. Group notifications, on the other hand, will automatically be received by users when they join a group or when members of group that they are part of shared resources. Lastly, Watching notifications will be automatically received by users when they are actively interacting with a graph or layout, but aren't necessarily the owner of it; users will receive this type of notification only if they have manually signed up for the Watching notification.
So, to implement this notification system, I will be using the asynchronous messaging service, Apache Kafka. While I considered RabbitMQ as an alternative service, its license Mozilla Public License (MPL) is incompatible with GraphSpace’s General Public License (GPL) v3. As this system should be decoupled from the main application flow, the main application will act like the producer, creating desired notifications and adding those to the messaging queue (see Figure 1). The producer will send different types of notification messages under different topics, and the consumer will add these notifications along with the status and type (from the topic of the message) to the same PostgreSQL database used by GraphSpace. Another component of the project will be implementing sockets in Django. This will be done using the django-socketio package. A socket is one endpoint of a two-way communication link between two programs running on the network. It allows real time, bidirectional communication between the web client and server. This allows for logged in users to see notifications in real time without refreshing the page. The notification will be pop down from the top right corner of the webpage the user is currently in.
A scheduled task, implemented using django-cron will email all unread notifications which have not been emailed before, to all users. Django-cron will allow us to write cron logic in Python to schedule tasks. The tasks will be scheduled to run whenever the GraphSpace server has less load. I will decide the criteria after discussion with my mentors and the GraphSpace team. The emails will be sent to only those users who have opted in for this service. The notification that have been emailed will be marked so in the database, so that they are not sent again.
In addition to adding a notification button or icon to the user’s dashboard displaying the number of unread notifications, a webpage to show all notifications will also be created. A user can then mark the notification as “read” or browse all notifications based on their read state, content, and type; this information can be accessed by clicking on the notification icon. The notification will be populated on this page by an API call to the notification application with request attributes. These request attributes will act like filters to get notification of a specific state, content and type.
To this end, I will be posting updates on my GSoC blog regarding the process and challenges faced throughout the working of the project, and will be in regular correspondence with project mentors regarding progress and improvements.
- Community Bonding Period
- Understand the application architecture for GraphSpace.
- Design schema for notification tables to implemented in PostgreSQL database.
- 1st week - 2nd week
- Research best practises and packages to implement notification system in Django.
- Study documentation and tutorials for Apache Kafka.
- Setup Apache Kafka and connect to Django.
- Architecture
- Write documentation for setting up and connecting Apache Kafka to Django
- 3rd week - 4th week
- Implement method for owner notification in consumer to read messages from queue and store messages to database.
- Implement methods in producer to create messages for owner notifications and add to queues.
- User will automatically receive these notifications when they create a resource (graph, layout or group) on GraphSpace. Here is a list of scenarios when a user will receive an owner notification:
- When a user uploads a graph.
- When a user creates a layout.
- When a user creates a group.
- When a user deletes a graph.
- When a user deletes a layout.
- When a user deletes a group.
- When a user updates a graph.
- When a user updates a layout.
- When a user updates a group.
- Test the creation and consumption of messages in queue.
- Document methods calls and execution flow of code for owner notifications.
- Phase 1 evaluation
- 5th week - 7th week
- Implement methods for group and watching notifications in consumer to read messages from queue and store messages to database.
- Implement methods in producer to create messages for these notifications and add to queues.
- User will automatically receive group notifications when they join a group. Here is a list of scenario when a user will receive a group notification:
- Someone shares a graph with the group.
- Someone shared a layout for a graph already shared with the group.
- Someone is added to / removed from the group.
- Someone un-shares a graph shared with a group.
- Someone un-shares a layout shared with a group.
- User will automatically receive watching notifications when they are actively watching a graph or layout, but aren't necessarily the owner of the graph or layout. The user will receive these type of notifications only if they have manually sign up for watching notification. Here is a list of scenario when a user will receive a watching notification:
- When a graph is updated.
- When a layout is updated.
- Test these methods.
- Document methods calls and execution flow of code for group and watching notifications.
- 8th week
- Create notification icon/button and webpage to display notification with mark as read and browsing functionality.
- Test webpage and all corresponding APIs.
- Document the APIs and view functions used to display the webpage, notifications and other functionality.
- Phase 2 evaluation
- 9th week
- Setup sockets to display notification to logged in users in real-time.
- Test the socket functionality.
- Document the adding socket functionality to any model.
- 10th week - 11th week
- Implement method to send groups of notification (if any) in an email to each user. These emails are sent to only those users who have opted for this service.
- Create email templates for different type of notifications.
- Implement cron job to run this method when the load on GraphSpace servers are low. This can decided in consultation with the GraphSpace team.
- Test email functionality and working of cron job.
- Document implementation of email functionality and timing for cron job.
- 12th week (Final Week)
- Code brush up and documentation
- Final Evaluation
Final report: https://gist.github.com/melvin15may/2ae50b6904844a8fa390030321394a3f