Harassment Manager is an Angular application with an Express server on the backend. The application also uses:
- The Twitter API to fetch Twitter data
- The Google Sheets API for exporting data to Sheets
- Perspective API to identify "toxic" comments
- Firebase for authentication and data storage
Before starting development, you'll need to follow the steps in this document to get these APIs and other dependencies set up. It should only be necessary to do this once, even if you're setting this up on behalf of an organization or team of developers.
The app makes use of several Twitter APIs, including:
- The Enterprise Full-Archive Search API or the v2 Full-Archive Search endpoint to fetch tweets directed at the logged in user
- The v2 blocks endpoint to block users on behalf of the authenticated user
- The v2 mutes endpoint to mute users on behalf of the authenticated user
- The v2 hide replies endpoint to hide replies on behalf of the authenticated user
To support all this functionality, you'll need to get access to the Twitter API.
NOTE: The full suite of Twitter APIs the app uses require additional access beyond the default Twitter API Essential access level. The Enterprise search API requires an enterprise account, while v2 Full-Archive Search requires academic access.
Once granted access, take note of the:
- Account name, app key, and app secret for your Twitter API developer account
- If using the Enterprise Full-Archive Search API, the Username and password for your enterprise account
- If using the v2 Full-Archive Search endpoint, the Twitter bearer token for your app
You'll need these credentials later on.
The tool is implemented in a way that either API can be used. While both APIs offer similar functionality, there are key differences in rate limits. We refer users to Twitter's comparison for more details. You may also see minute differences in:
- Which tweets are fetched. This is due to differences in granularity for timestamp format each API supports (YYYYMMDD for Enterprise and YYYY-MM-DDTHH:mm:ssZ for v2).
- The order the tweets are displayed when sorted by "Priority". This is due to small differences in how we parse out the tweet text, which causes some variation in the Perspective API scores for the text. See issue #19 for more details.
A GCP project is required to use several of the APIs the application depends on. This project forms the basis for creating, enabling, and using all Google Cloud services, including managing APIs, enabling billing, adding and removing collaborators, and managing permissions.
- Create a Google Cloud account if you do not already have one. Note you'll need to log in with a Google account, enter details about your account type, and add a payment method.
- Create a Google Cloud project. Developers who are setting the project up for an organization may want to review the quickstart for organizations.
- Register for Perspective API access.
- Once access is granted to Perspective API, search for and enable the
following APIs in the GCP API
Library.
- Perspective Comment Analyzer
- Google Sheets
- Visit the GCP Credentials page.
- Click Create Credentials and select OAuth client ID. Set "Application type" to Web application. Add all URLs you want to run the application with, including http://localhost:8080 (or your preferred port), under "Authorized JavaScript origins" and "Authorized redirect URIs". Consider renaming this ID to something you can easily identify.
-
Register for Perspective API
-
Enable the following APIs for your project:
- Google Sheets
- Perspective Comment Analyzer
-
Create credentials:
- Create an OAuth 2.0 client ID of type "Web application" to access the Sheets APIs. You will want to add all URLs for your application, including http://localhost:8080 (or your preferred port), under "Authorized JavaScript origins" and "Authorized redirect URIs".
- Log in to the Firebase console, then click Add project.
- Select your existing Google Cloud project from the dropdown menu, then click Continue.
- (Optional) Enable Google Analytics for your project, then follow the prompts to select or create a Google Analytics account. See below for more details.
- Click Add Firebase.
By default, the GoogleAnalyticsService emits logging events for several user actions across the app. To actually enable this logging and collect the data in Google Analytics,
- Create a Google Analytics account and property
- Find your ID (starting with "G-") to use with the gtag.js library. Take note of this ID for later on.
- In the Firebase console, click Authentication from the navigation panel.
- Go to the Sign-in Method tab
- Enable Google and Twitter authentication.
-
In the Firebase console, click Firestore Database from the navigation panel.
-
Select Create database and follow the prompts to enable Firestore.
-
Go to the Rules tab.
-
(Important) Change the existing rule to:
rules_version = '2'; service cloud.firestore { match /databases/{database}/documents { // Only allow users access to their own data match /users/{userId}/{documents=**} { allow read, write: if request.auth != null && request.auth.uid == userId } } }
This enables content-owner only access, so users can only read/write their own data.
-
Click Publish.
- In the center of the Firebase console's project overview page, click the Web icon to launch the setup workflow. If you've already added an app to your Firebase project, click Add app to display the platform options.
- Enter your app's nickname. This nickname is an internal, convenience identifier and is only visible to you in the Firebase console.
- Click Register app.
- Copy the
firebaseConfig
constant into a local file. You'll need this config to set up Firebase in your Angular project.
The app uses AngularFire to interact
with Firebase Authentication and Firestore. By default, AngularFire instructs
you to add the Firebase config you saved above to
/src/environments/environment.ts
. However, this exposes your API key in the
repository. This is problematic because we enabled billable APIs (Perspective
API, which is free, and Google Sheets) for this project. To help secure your API
key, we strongly recommend adding key
restrictions.
One such setup involves creating "client-side" and "server-side" specific keys. To do so,
-
Visit the GCP Credentials page.
-
Select the API key created by Firebase (i.e. the same key from the
firebaseConfig
you saved above). -
Under API Restrictions, add
- Identity Toolkit API
- Token Service API This restricts the key to only the APIs the client-side of the application sues.
-
Go back to the Credentials page and click Create Credentials and select API key. Consider renaming this key to "Client-side key" or something similar to easily identify it.
-
Under API Restrictions, add
- Cloud Firestore API
- Google Sheets API
- Perspective Comment Analyzer API
This restricts the key to only the APIs the server-side of the application sues. You'll use this key to configure the server later on. Consider renaming this key to "Server-side key" or something similar to easily identify it. Make sure to avoid checking in this key in any source code.
By default, the app uses the same Firebase project for both development and
production. This means the app writes to the same Firestore database. If you'd
like to set up different Firebase projects for development and
production,
you'll need to setup the API keys as described above and update
environment.prod.ts
with the client-side API key from your production project.
We highly encourage also adding referrer and IP restrictions to your production
API keys to restrict the client-side key to the domain you deploy the app to and
the server-side key to the IP address of your server.
You should now be ready to set up development.