Adventures in the dark web of government data

1 minute read

Abstract

Government bureaucracy is your friend. The US federal government alone produces tens of thousands of different forms that collect information on everything from the owner and location of every oil well in the country, to the VIN number of every car that’s imported, the location and height of every cell phone tower, and much more. While most of this data is locked behind clunky 1990s-era search forms, or in exports of antiquated database formats, the enterprising researcher will find a treasure trove that exists outside the indexes of Google and LexisNexis.

I have written scrapers and parsers for 100s of these databases and will share with you what I’ve learned about coaxing OSINT out of some of the messiest and hard to find data out there.

The talk will specifically feature a deep dive into the data produced by the US Federal Communications Commission. The FCC has issued over 20 million licenses for transmitting on regulated parts of the electromagnetic spectrum. The data residue of this process can be used for everything from geo-locating electronic border surveillance infrastructure to discovering the location and transmission frequency of every McDonald’s drive-thru radio. In the second portion of the talk, I will discuss how various protocols for data transmission can be decoded and joined with other contextual public data. For instance, every cargo ship emits an ““Automated Identification System”” signal that can be joined with shipping records to understand what the ship is carrying.

By the end of the talk, I hope attendees will develop new intuitions and techniques for finding and working with government data, and specifically have the tools to run their own investigations using FCC data.

Speaker Profile

Marc DaCosta

Updated: