California Data Demo Day: 5 New Lessons on Transparency Outside and Efficiency Inside


On October 19th in Sacramento, our California Data Demo Day brought together more than 100 supporters of opening up the Golden State’s data, representing dozens of government agencies and tech companies. Grant Thornton, Xcential, and OpenGov made our event possible through their sponsorship.

Our morning of speeches, panels, and live demonstrations celebrated the impacts of open data for California: transparency outside, efficiency inside.

By standardizing and publishing their information as open data, governments empower their constituents to understand the impact of policy – and to vote based on that understanding. But the “power users” of open data are leaders inside government, who gain access to a powerful foundation for decision-driving analytics.

As civil servants inside government get better access to standardized operational data, the impact is “potentially lifesaving,” according to Sen. Richard Pan, who delivered the keynote address.

Last week’s California Data Demo Day was our third annual celebration of open data in the California capital. But even though the event wasn’t new, its insights were.

We learned five interesting things about what happens when government information is transformed into open data and freely shared, outside and inside.

  1. Open data mandates needn’t be enacted to be effective.

Senator Richard Pan (D-Sacramento) delivers the keynote address with Tim Melton of OpenGov.

Dr. Pan, represents Sacramento in the California State Senate. Two years ago, our very first California Data Demo Day celebrated a 2015 Pan proposal, SB 573, which would have required the state to appoint a chief data officer.

Pan’s bill died, but his idea didn’t. Less than a year later, Governor Jerry Brown appointed the state’s first CDO, with an office within the state’s Government Operations Agency and a mandate to create data standards and push agencies for the state’s main open data portal – just as Pan originally intended.

“The endpoint is more important than whether something gets signed,” Sen. Pan told our Demo Day audience. “The [Brown] administration was convinced [of the need for a chief data officer]. They just didn’t want it in the legal code.”

Sen. Pan’s championship of SB 573 helped ensure a good result – even without being enacted.

  1. Even though open data is globally accessible, geography still matters.

California agencies and municipalities are publishing more data, leading to opportunities for tech companies anywhere in the world to republish, analyze, and automate it, for customers both outside and inside government.

But Sen. Pan sees these business opportunities as a particular boon for his own constituents in Sacramento. Californian open data is a “unique resource for our own economic development” in the state capital, he said. Even though open data is global, it’s still an advantage for companies working with California open data to be physically in Sacramento, because that’s where the custodians publishing, the experts in, and the users of those data sets are clustered.

Sen. Pan asked, specifically, for more examples of tech companies using open government data to create insights for private- and public-sector customers. Our participants at the Data Demo Day were more than happy to oblige. OpenGov, for instance, automatically plugs its customers’ data into comparisons against their peer governments who are customers – over 1,600 across the United States.

  1. Open data is a time-saver inside the legislative branch.

From left to right: Mendora Servin, Information Technology Manager, Legislative Counsel Bureau; Lance Christensen, Chief of Staff to Sen. John Moorlach; Robb Korinke, Principal, Grassroots Lab; and Ari Hershowitz, Director of Open Government, Xcential Legislative Technologies (moderator).

California’s state legislature is a world leader in the internal use of open data to manage legislative documents. Our Legislative Data Spotlight, which featured two policy professionals and a representative of the state’s Legislative Counsel, put this leadership in, well, the spotlight. 

The Office of the Legislative Counsel maintains bills, laws, and amendments as standardized XML data, rather than as PDF or paper documents, using a platform provided by Xcential. Instead of drafting their bills in Microsoft Word and converting into PDF and print, California legislative lawyers use a software tool that natively drafts bills as XML instances.

This means the Legislative Counsel can automatically show legislators how a proposed amendment would change a bill or underlying law. (Here’s an example of what this looks like.) There is no need to manually proofread a bill’s sectional structure, or its citations; that is automatic, too. And California state laws, once enacted, automatically flow into the state’s legal code, rather than through manual revisions.

Using open data to create and manage legislation “allows us to concentrate on … the mechanics of doing the drafting, so the [legislators and their] staff can concentrate on what’s really in the bill,” said Information Technology Manager Mendora Servin.

The U.S. Congress and other legislatures don’t use a consistent open data standard for legislation, like California does, and so they aren’t as efficient. Leaders like the Clerk of the House of Representatives are working to change this.

  1. In California, the future of open data – external and internal use of the exact same data set – is already happening.

Last year, our sister organization, the Data Foundation, reported that “the motivation for governments to express their information as open data is currently changing from external transparency to internal management.” The Foundation observed that as governments begin to treat their open data as a resource for their own managerial use, internal “decision-makers [begin to] rely on the same information, and sometimes even the same systems and software, as their external constituents do.”

At the California Data Demo Day, our Executive Panel confirmed that this is really happening.

Treasurer John Chiang, who previously served as the state’s controller, is responsible for a string of open data portals. As controller, Chiang established Publicpay.ca.gov, which offers open data on the compensation for “approximately 2 million positions at more than 5,000 public employers” in the state. Deputy Treasurer Jan Ross, representing Chiang, told our audience that the website proved to be as useful inside government as outside. Unions and human resources departments now pull the information they need for bargaining directly from Publicpay, rather than separately paying for compensation studies. The availability of this open data “eliminated an entire area of bureaucracy,” said Ross.

As controller, in addition to Publicpay, Chiang also set up By the Numbers, which provides open data on local governments’ revenues and expenditures. As treasurer, Chiang is responsible for DebtWatch, featuring open data on debt issued by state and municipal governments, and has more open data on the way.

Ross and her co-panelist, League of California Cities legislative representative Dane Hutchings, agreed that California state agencies and municipalities have a long way to go to fully realize the internal and external promises of open data.

First, unlike the U.S. federal government, which standardized its spending data into a single open data set as a result of the DATA Act of 2014, California’s state government cannot offer a single electronic picture of all of its spending because its finances are managed by four separate agencies. The state is in the midst of a multi-year, nearly-billion-dollar program, known as FI$Cal, to standardize spending systems and data across 160 agencies’ disparate financial systems.

Second, though the state’s local governments and special districts report their spending data to By the Numbers, their reports are not comparable from city to city. More standardization is needed to accomplish that goal. Cities “would love to work with the state of California to be able to have a unified hub” for standardized spending data, said Hutchings.

  1. Future victories: geospatial unity, regional cooperation, and better government.

California agencies’ and municipalities’ desire to transform their information into open data goes far beyond spending and financial transparency. Hutchings told our crowd the states’ cities are realizing that open data is “the best way that we can interact with our constituencies directly.” Los Angeles’ GeoHub, for instance, combines over 500 open data sets, maintained by 20 different city departments, on the same geospatial platform, so that external and internal users can see the confluence of construction, traffic, utilities, public safety, and much more on a single map.

Asked to identify the greatest impediments to better data, better transparency outside, and better efficiency inside, Hutchings blamed funding shortfalls. “Cities are often triaging” their resources to meet basic obligations, particularly due to employee pension obligations, but one possible solution is regional cooperation, said Hutchings.

Ross said the state’s greatest challenges are “the evil twins of vision and data.” Too often, agencies focus on their own missions without considering how their data might be combined with peers’ data to create unified insights. And the fragmentation of systems means data sets must be laboriously cobbled together to become useful.

But the combined energies of all the participants in the California Data Demo Day – representing agencies, municipalities, civic organizations, and tech companies – seemed to overmatch those challenges.

Thanks to the teamwork of these supporters, open data is already delivering transparency outside and efficiency inside for Californians and their government. And every new data set adds to those benefits.