Big Apple’s Big Data Beats Washington’s

Municipal Government

Big Apple’s Big Data Beats Washington’s

Guest Post from John Turner

As Washington works to expose the Big Data of Big Government with the reintroduction of the DATA Act, New York City has already solved the Big Data problem associated with its own spending.

Checkbook NYC is the cutting edge of government spending transparency, providing access to disbursement-level details, including the date and recipient of each transaction, in real time. In contrast, disbursement-level transparency is absent from the federal government’s spending transparency website,, which summarizes each grant and contract but provides no details on transactions.

Updated daily, Checkbook NYC accounts for every dollar the city spends, from contract awards to vendor payments to payroll., meanwhile, includes grants and contracts but ignores all internal expenditures. A U.S. PIRG report analyzing city transparency websites found that it was easy to verify the accuracy and completeness of New York’s data by aggregating and comparing across the database (p34). The Sunlight Foundation has found that similar verifications cannot be performed on, since it provides only a limited subset of federal spending data.

Checkbook NYC is useful for average citizens and sophisticated coders alike. The data is accessible and intuitively searchable for casual users via the website, which includes plenty of graphs and charts for quick visualizations. Every dataset is also easily downloadable in machine-readable formats so that more sophisticated tech start-ups, watchdog groups, and coders can build their own analysis tools. does permit bulk downloads, but there are legal limits on how the data can be reused because a private company, D&B, owns the proprietary system of codes that are used to identify grantees and contractors.

New York launched Checkbook NYC after learning – the hard way – how opaque government spending is a breeding ground for waste, fraud, and abuse. CityTime, a massive contracting debacle, served as motivation for opening contract spending data to public oversight. In 1998, the city contracted to improve the payroll system at an expected cost of $63 million over a term of 5 years. Fraud permeated the contract for years as costs ballooned without investigation. More than ten years later and well over $600 million beyond budget, CityTime was still incomplete when the contract was finally terminated. On the federal level, executive-branch overspending motivated the House of Representatives to pass the DATA Act in 2012, but the White House opposed the proposal and it died in the Senate.

Checkbook NYC can’t get the wasted CityTime money back, but it can protect New Yorkers from similar corruption in the future. Not only is each and every payment on each contract publicly available, but Checkbook NYC also keeps a running list of master agreement and contract modifications. This means New Yorkers can pinpoint exactly when and by how much a contract goes over budget. As Checkbook NYC continues to develop, subcontractor data will be incorporated into the database and the text of each contract is expected to be digitally published as well.

Once fully operational, Checkbook NYC will place its background code in the public domain to encourage other state and municipal governments to use and build upon the platform. As NYC Comptroller John Liu put it, “We do not view sharing software as a selfless act; on the contrary, distributing code as ‘open source’ is a cost-effective way to identify and fix bugs and to leverage new features added by other developers” (p35).

New York City’s efforts to publish spending data provide both a roadmap and a yardstick for federal lawmakers. Full transparency for a government’s spending requires disbursement-level disclosure, inclusion of internal expenditures, and unfettered reusability. Checkbook NYC delivers all three. delivers none.

We hope the DATA Act, which awaits reintroduction in the 113th Congress, will help Washington measure up to New York.