Linking datasets without revealing personally identifiable information (PII)
Policy-relevant research often requires a combination of administrative datasets from different agencies,
especially when research questions concern more than one service or program. The preferred and most
accurate method is linking at the individual-level using PII. However, a major consideration in linking
individual-level administrative data is the need to protect individual privacy and comply with privacy
constraints. Statutes may also restrict the types of data that agencies can share.
The California Policy Lab (CPL) has developed a process to link individual-level data from multiple
agencies without receiving PII. Rather than sharing PII, such as full names and Social Security Numbers
(SSNs), agencies first encrypt the PII using a hashing technique before transferring any data. Once data is
transferred, CPL then links the data using the hashed identifiers.
To date, CPL has used this approach to help agencies address vexing policy issues, including:
●A project with the California Franchise Tax Board (FTB) and the California Department of Social
Services (CDSS) linking tax and social services data to understand whether Californians on food
stamps are claiming important anti-poverty tax credits.
●A project with CDSS, the California Student Aid Commission, the California Community
Colleges, and the University of California Office of the President, linking student, financial aid,
and social services data to understand the extent to which college students are eligible for food
stamps, but are missing out on receiving these benefits.
What is hashing and how is it applied to PII?
Hashing is a method of turning readable data into unique but unreadable data -- this can be applied to
PII to protect the values in a way that they can still be used to link datasets. When agencies hash data,
they apply a one-way encryption that turns the PII into indecipherable strings of letters and numbers.
The original datasets at each partner agency contain both PII (for example, names, addresses, and SSNs)
and other analytic data that does not include PII (for example, variables about the individuals such as
college enrollment details and financial aid received). Once the hashing process is complete, CPL
receives the hashed PII as well as the un-hashed analytic data.
Phase 1. The hashing process
A hash function performs a one-way transformation that can convert a readable piece of data of any
length into a different, unreadable piece of data of a fixed length. Hashing can turn PII (such as the name
“John” or an SSN) into an indecipherable but uniform string of characters. For example, using the SHA
256 hashing algorithm, “John” hashed becomes “a8cfcd74832004951b4408cdb0a5dbcd8c7e52d43f7fe-
244bf720582e05241da”. If even one character in the input string changes, the hash is different. For
example, even one digit in an SSN changes the hash completely:
