Crowdsourcing Police Use-of-Force Reports: Is it possible?
Human Rights First is an advocacy group self described as: “an independent advocacy and action organization that challenges America to live up to its ideals. We believe American leadership is essential in the global struggle for human rights, so we press the U.S. government and private companies to respect human rights and the rule of law”. The Blue Witness program aims to leverage Machine Learning and technology to give the public access to a database of police use-of-force incident reports.
The data-science team (which I’m currently a part of) is trying to build a tool that can automatically sort through Twitter noise and decide what is and is not a police use-of-force report and include those that are in the database. Previous teams have already built an NLP (Natural Language Processing) model, BERT, that will process text and determine if it is use of force and what rank of force it is. While the model still needs more training, it works well enough to allow our current team to leverage it for building the TwitterBot. The TwitterBot is a program we are building to scrape Twitter for mentions of police, run the Tweets through the BERT model and reach out to the users of Twitter to gain more information on the incident before logging the report in the database.
The Challenge
The problem we are trying to solve is how to scrape the Twitter website for these reports, reach out for more information from users, and keep track of the attempts to reach out as well as the responses. The path we decided to take was to scrape the Twitter website for certain keywords (such as “police”, “cops”, etc.), using the Tweepy API search function, the received Tweets text would then be cleaned and run through the model, which is running on its own API. If the model predicts said tweet describes the use of police force, the information on the Tweet will be logged in a potential incidents database where the HRF admins can manually trigger a response to the users (this is a stepping stone to an automatic reply while the model and TwitterBot is fine-tuned).
One of the technical challenges we had was how to avoid duplicates of Tweets that are already in the main database, as well as Tweets that are in the Potential Incidents database waiting for responses or for actions from the admins. To solve this, we first query the database for the largest Twitter ID (since they are sequential), and we will start the search from those tweets onward:
State of Affairs
Currently, all modules of the TwitterBot (search, reply, response detection) can be called manually by running python scripts.
The scraper module can be run by executing the update_twitter_data() function in the scraper.py file. This will add the Tweets that the model determines contain use-of-force, and their relevant information (text, user, date created, etc.) to the potential incidents database.
Similarly the scripts for responding to a Tweet asking for more information (RFI Tweets), can be run manually from a python notebook, this will update the database with the information from the RFI Tweets.
There is one other scraper that checks Twitter for replies to the RFI Tweets that are still pending. This can also be called manually by running the update_mentions() function, this will check for replies and update the database accordingly, and later be presented to the HRF admin.
The next step is for all these modules to be brought together so that the scraper runs periodically searching for new Tweets, and incorporated with the Front End, so that the admins can trigger replies to Tweets and is notified when a user replies to his/her RFI Tweets. Eventually the goal is for this to happen automatically from scraping Twitter to sending the reply and then interpreting the reply, but for that to happen the model, as well as the application need to be a lot more mature to prevent the database being populated with wrongful info or unintentional spamming. After seeing the first iteration at work, and knowing the capabilities of the BERT NLP model, I’m thinking this might be possible after all.
