During my five years working on tech in the Federal government, the most frustrating rumor I heard was that, for Feds, “It’s illegal to do user research.”* Because of that rumor, very few services from the Federal government are built using the critical insights and feedback of the people who rely on those services.
If you’re a design professional, you may be squinting behind your cool glasses and saying, “wut.”
If you’ve tried to get help from the Federal government, you may be saying, “Figures!”
If you’re a Federal employee, you may be saying “Wait, that’s just a rumor?! I’ve been trying to do this work for years. Help!”
Wait, How Did We Get Here?
The Paperwork Reduction Act of 1995 (PRA) is the source of the rumor. The PRA is a law designed to “maximize the practical utility of and public benefit from information collected by or for the Federal Government,” specifically to:
1) Reduce burden on the public and prevent the Federal government from collecting redundant information or information that they don’t actually need to provide a service,
2) Make sure that the way the Federal government is collecting information to make policy is designed so that the resulting information can be used to produce meaningful, accurate statistics (think census), and,
3) Give the public a chance to weigh in on how the government is collecting information.
TL;DR PRA = The federal government can’t make you fill out bad forms all willy nilly, and you get to weigh in.
In other words, the PRA is actually awesome. It could make sure that when a student applies for Federal student aid, forms aren’t so hard to complete that they need help to finish.** It could help make sure the information from those forms is collected in a way that they’re actually useful in shaping policy to serve those students. It can make it possible for student advocates to fight back if invasive, unnecessary questions are added to the forms.
What does this have to do with user research?
There’s a common misunderstanding that user research— asking questions to understand needs and behaviors, or even observing people while they use services — falls under the jurisdiction of the PRA in the Federal space (Spoiler: it does not!).
In practice, this is typically what happens:
Jenny is working to improve the website for the U.S. Department of Flying Pigs — they’ve gotten feedback from their community relations office that the Flying Pig Farmers of America thinks the site is hard to use. Jenny decides that she wants to do usability testing, a type of user research where she would watch Joan the pig farmer actually try to use the current site to see where she is having the most trouble. Then, Jenny plans to have similar sessions with a few other farmers and see what she learns, and fix the biggest issues.
Jenny checks in with the legal team to see if it’s ok. The U.S. Department of Flying Pigs’ lawyer sees that Jenny wants to ask questions (during usability testing) of the public (Joan), and because the lawyers think it’s an “information collection,” they tell Jenny she will need PRA approval. PRA approval typically takes at least 9 months, includes two rounds of formal public comment through the Federal Register, requires sign off from — and I am not kidding — the Executive Office of the President at the White House, and requires her to submit every single question she will ask in advance.
Jenny is stunned. Setting aside the daunting prospect of nine months of wait time and White House sign off for one round of testing, knowing every single exact question she will ask in advance of a usability test is impossible — it’s a conversation. Jenny gives up. The website continues to drive the pig farmers crazy.
The Truth about User Research
Both the letter and spirit of the law and related guidance not only allow user research in the Federal government, they encourage it.
User Research*** is Not Subject to the PRA
Laws and regulations define key terms so that people know how to follow the rules. In the case of the PRA, the guidance explains that the law only applies when the Federal government is collecting a specific definition of “information” and even outlines a number of exceptions to that definition. Meaning, if what is being collected is not “information” according to the regulatory definition, that collection is not subject to the PRA.
There’s a handy-dandy chart that outlines a bunch of exceptions, but I want to highlight one that directly relates to what Jenny was trying to do.
Exceptions to the definition of “information” for the purposes of the PRA include: “Facts or opinions obtained through direct observation by an employee or agent of the sponsoring agency or through non-standardized oral communication in connection with such direct observations.” Read it for yourself in our old friend, the Code of Federal Regulations: 5 C.F.R. 1320.3(h)(3).
If you’ve never had the pleasure/anguish of watching a usability testing session, here’s a recording of one by the VA:
This video shows the first-ever rounds of usability testing on the Veterans health care application. The first time I saw this video, I cried. This form had been up for years before anyone actually understood what was happening to people who were trying to use this service. The people who conducted this research (and later helped fix the form), include Mary Ann Brody (whose voice you hear), Courtney Eimerman-Wallace, Anne Kainic, and Emily Tavoulareas. They fought to do this type of testing in the face of the false rumor that this research was not possible in the Federal government, and I will forever be grateful for their work.
In watching the video, you’ll note that they’re collecting facts and opinions through direct observation and through non-standard oral communication. Sound familiar?
The Office of Management and Budget Recommends User Research
The very first “play” in the Digital Services Playbook, put out by the Office of Management and Budget (the same part of the White House that oversees the PRA) recommends user research as the foundation for any service.
Over the years, the Office of Management and Budget has aspired to, been ordered to by Congress, or promised they would improve government customer service. User research is just one method they ask agencies to use to improve that customer service.
How Did it Get This Way, and How Can I help?
Jen Pahlka has this perfect explanation. When we worked together in the Office of the U.S. CTO, she explained to me that there are “red laws” and “blue laws” — red laws are the things that are written down in black and white, that you can look up and read and understand (the PRA). Blue laws are the things that people make up around the water cooler to try and simplify the rules (“You need *lots* of approval to ever collect information”).
The problem with this system is when a “blue law” is applied so broadly that it morphs and blocks critical work (“User research is illegal.”)
How to Help
The best way to help, is to
- Encourage government agencies to use the Digital Services Playbook
- Keep the handy-dandy chart handy in case they get the answer that Jenny received
- Insist that user research is conducted on government services. If you get pushback, show the above video of the Veterans health care application usability testing session to help explain the importance and urgency.
—
Erie Meyer is a recovering Fed, where she co-founded the U.S. Digital Service, conducted exhaustive user research, and has received such honors as emergency, generic, non-substantive, and plain ol’ PRA approval on work that actually required it. She is now writing Paperwork Reduction Act essays for fun, apparently.
A deep thank you to Kara DeFrias, Mary Ann Brody, Anne Kainic and Dana Chisnell for their edits to this piece, to Jen Pahlka for giving me language to understand what was going wrong and why it was actually the result of people trying to help, to the civil servants who fight to the bone every day to do the right thing, and most of all to the brilliant, tough women that I am so lucky to be surrounded by.
—
*For the purposes of this piece, I’m referring to all forms of qualitative user research. Quantitative user research is also not illegal, Uncle Sam, it just would need a different essay to explain.
**Could.
***If you’re saying “But what if someone calls it ‘user research’ but is making a long invasive form that is redundant and will be used to make statistics about American life that will then be used to make flawed policy!?!??!?!” First, you are a catastrophizing in a really weird way. Second, that’s not user research! If I steal your car and call it “throwing you a birthday party,” that does not mean that my creative naming supersedes the law.
