Concerned citizens and impacted residents don't have a straightforward way of knowing what or how policies and regulations impact them or what they can do in response. A community leader could use this tool to help motivate their social followers.
Technology has the power to drive action. And right now, a call to action is needed to eradicate racism. Black lives matter.
We recognize technology alone cannot fix hundreds of years of racial injustice and inequality, but when we put it in the hands of the Black community and their supporters, technology can begin to bridge a gap. To start a dialogue. To identify areas where technology can help pave a road to progress.
This project is an effort to utilize technology to analyze, inform, and develop policy to reform the workplace, products, public safety, and legislation.
This is one of several open source projects underway as part of the Call for Code for Racial Justice led by contributors from IBM and Red Hat.
Concerned citizens and impacted residents don't have a straightforward way of knowing what or how policies and regulations impact them or what they can do in response. A community leader could use this tool to help motivate their social followers. Our target user will be referred to as "advocate".
-
Advocates are aware of policy that is being considered that is highly impactful to them, without needing to follow every vote.
-
Advocates are able to understand the specific impact of proposed policy on them without being a legal expert.
-
Advocates are able to share opinions so they can influence policy decisions before they are finalized.
-
Advocates can easily ascertain the voting record themes or trend of their elected officials and political candidates without prior knowledge of who they are.
-
Policy makers have visibility into how diverse citizenry will be impacted by multiple variations of a proposed policy.
Searching for legislation can be complicated by the fact that each government level has their own repository, their own numbering convention, and different formats. Technology can assist by allowing content to be centralized into a single database, in a consistent format, readily accessible to advocates that need simple-to-read information.
This is a web-based application written in Python programming language, using the Django framework and Bootstrap user interface styling. Its primary goal is to help advocates find legislation of interest based on an advocate's preferences for impact areas and geographical location.
The application is customizable, allowing application staff to specify the location hierarchy and impact categories.
The database is automatically updated weekly, combining the resources from Legiscan.com API with government repositories. Plain text is extracted from HTML and PDF formatted documents, processed through IBM Watson Natural Language Understanding (NLU) service to identify title, summary, location, and impact. The results can then be further curated to improve legibility.
By classifying legislation by impact and location, we hope to increase awareness of current and pending legislation and their ability to affect change through voting or other activism.
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
Prerequisites: Python, pip, pipenv
- Clone the Legit-Info Git repository
git clone https://github.com/Call-for-Code-for-Racial-Justice/Legit-Info.git
- Navigate to the Legit-Info directory
cd Legit-Info
- Install the dependencies using
pipenv.
pipenv install
- Use the following command if you already have installed the dependencies and want to activate the virtual environment only.
pipenv shell
- Use the following commands to create your local database. (If you are using Windows, there will be command variations for the next 3 steps)
./stage1 makemigrations
./stage1 migrate
Windows
stage1.bat makemigrations
stage1.bat migrate
- Run the following command to generate a default user with some pre-existing impacts and locations. This creates an admin user
cfcadminwith the passwordCall4Code.
./stage1 loaddata sources/cfc-seed.json
Windows
stage1.bat loaddata sources/cfc-seed.json
- Run the application with the following command.
./stage1 runserver localhost:3000
Windows
stage1.bat runserver localhost:3000
This project is designed for three deployment stages.
In stage 1, each developer has their own copy of application code and data, using SQLite3 that stores the entire database in a single file. Django provides a development webserver to allow local testing.
The original PDF or HTML version of the legislation, along with the extracted TEXT files, are stored on local file system and analyzed. The resulting title, summary, location and impact are stored in the database. This allows staff to review the AI/NLP-based classifications and make adjustments.
In stage 2, each developer has their own copy of application code, but a shared database, using Postgresql running in the IBM Cloud. The developer can choose to use the Django development webserver, or try out the production server called Gunicorn. The difference is that Django is designed for single-user, and Gunicorn for concurrent multiple users.
Optionally, the PDF, HTML and TEXT files of the legislation can be stored on a network file system or IBM Cloud Object Storage for shared access.
In stage 3, the application is running in the IBM Cloud in one pod, using the Postgresql running in the IBM Cloud from pre-production. Updates to the code are deployed using a Tekton pipeline.
The PDF, HTML and TEXT files of the legislation can be stored on the same web application container, a network file system, or IBM Cloud Object Storage.
Please read CONTRIBUTING.md for details on the code of conduct, and the process for submitting pull requests to the project. This project adheres to the PEP 8 Python Coding Style Guidelines, Django naming conventions, and other standards. See STYLE.md for details.
This starter kit can be taken in a variety of directions.
- Email:
Without an email SMTP server, we used instead Mailtrap.io
which allowed testing of the HTML and TEXT formats. Considering adding
an SMTP server to send out email.
- Permission Levels:
The Superuser can grant privileges to other users, and this could be implemented as a "upgrade user to Staff" button option with specific privilege levels pre-set.
- Share options:
The advocate can share results three ways: (a) print out the results on paper or PDF file; (b) email to self then forward that email to others from their regular email system, and (c) download CSV file to send out to others or perform other analyses. Additional options could be to send out tweets on Twitter, post to a Blog, or send out to other social media outlets like Facebook.
- Feedback options:
Add information to each law related to the proponents or opponents, these could be elected officials or activists that are actively involved with the legislation. Feedback options can be to send opinions directly to those people, or to allow those elected officials to easily see the collection of opinions directly from the website.
- Other locations and languages:
The solution is focused on United States and the English language. The MVP focused on just Arizona and Ohio. However, the location hierarchy does accomodate different governmental structures, and the Impact, Location, Title and Summary all support UTF-8 character set to allow for international characters. This solution could be modified to handle other locations, evaluating alternatives to Legiscan.com, and finding other public sources of information for the particular country or language.
The web application and cron job use the following IBM and open source technologies.
- Python - The programming language, along with pip and pipenv supporting tools
- Django - A framework for web applications written in Python
- Bootstrap - A popular User Interface toolkit, providing "responsive UI" to handle different screen sizes across smartphones, tablets and desktops.
- PDFminer - Extracts text from PDF
- SQlite - A simple, local database
- Postgresql - A robust, relational database
- Gunicorn - A WSGI HTTP webserver for Python
- NLTK - Natural Language Toolkit
- Watson Studio Natural Language Understanding
- Legiscan.com -- API for all 50 USA states
- AZLeg.gov -- Arizona State Legislature website
- Ohio.gov -- Ohio State website
The following technologies were used to deploy and demonstrate the MVP. Certainly, they are optional, as the above application can run on a traditional LAMP stack bare-metal server or virtual machine.
- Red Hat UBI - Red Hat Universal Base Image, based on Red Hat Enterprise Linux (RHEL) 8.2
- Docker - A container platform
- Kubernetes - An orchestration layer for Docker
- Red Hat OpenShift - Red Hat container platform based on Kubernetes
- Tekton - A Continuous Integration (CI) pipeline
This project is licensed under the Apache 2 License - see the LICENSE file for details Please also read the BSD Copyright Notice