Documentation for a selection of our system’s common internal tools


A number of custom commands are available to developers via make. They offer options for quickly running tests, working with the docs and executing scrapers.

build-release        builds source and wheel package
check-release        check release for potential errors
coverage             check code coverage
format               automatically format Python code with black
help                 Show this help. Example: make help
lint                 run the linter
mypy                 run mypy type checks
run                  run a scraper. example: `make run scraper=IA`
serve-docs           start the documentation test server
tally-sources        update sources dashboard in the docs
test-docs            build the docs as html
test                 run all tests

For example, you can run this from the root of the project.

make run scraper=IA


The Cache class is used to save the raw HTML, PDFs and CSVs files our scrapers collect.


The utils module contains a variety of variables and functions used by our scrapers.



A 2013 summary by the Congressional Research Service


A 2003 audit by the Government Accountability Office