Assessment & Authorization Research
Veterans Administration User Research
District Court of Maryland Rent Escrow
Election Polling Place Judge Process
Maryland State Board of Elections
Vote by Mail Ballot Instructions & Inserts
Voter Registration Application
Request for Absentee Ballot
Candidate Information Sheet
Election Judge Instruction Book
Taxpayer First Vision
Internal revenue service
jan ‘20-feb ‘20
Taxpayer First Blueprint
The Taxpayer First Vision is a long-term strategic plan for transforming the taxpayer and IRS employee experience. The IRS team desired an eco-system blueprint that illustrated the long-term vision. We worked extensively with the team to define the most effective content. User stories were written to illustrate new journeys for individuals, partners, and businesses. Working with the strategic plan, we helped the IRS clarify their vision and the way to effectively communicate it.
A & A Process
Federal dept.
dec ‘19-Jan ‘20
This project focused on defining the assessment and authorization process for a division within the federal agency. Significant primary and secondary research was required both pre- and post-launch. The project team did extensive interviews and synthesis to help the agency refine and digitize the process along with refining role definitions for those involved in assessment and authorization.
User Research
VETERANS ADMINistration
MAR ‘20-MAY ‘20
The VA Support Services Division is implementing a new Salesforce application. Working with VA leadership, in-depth user interviews were conducted to define and streamline processes and activities. Given there was no uniform process from region to region, synthesis revealed strengths and weaknesses drawing from each regions process. Findings were presented including prioritization and personas. The synthesis helped guide the team in defining initial areas of focus and implementation.
Vote By Mail
the center for civic design
July ‘15-may ‘16
2nd Testing Session full mock-up
Two groups of voters have historically faced challenges in voting by mail: first, voters who are blind, have low vision, or low dexterity; and second, the many adult Americans who read at basic or below basic levels as defined by the National Adult Literacy Survey. Although for different reasons, both groups can find it challenging to navigate the complicated process of receiving, marking, and mailing a paper ballot privately, independently, and successfully. In 2008, the Election Assistance Commission reported between a 2-10% rejection rate by state for absentee voting. Reasons for the rejections included missing the deadline, lack of a valid signature, using an unsealed envelope, using an unofficial envelope and age-related impairments. This project was an effort to increase the accessibility and inclusivity of the vote by mail process using iterative usability testing and evaluation to revise paper instructions and envelopes in vote-by-mail packages—identifying and applying best practices of plain language and plain interaction in order to forestall common mistakes and increase trust.
Since ballot designs are usually constrained by law and inevitably vary across jurisdictions, the scope of the paper-based ballot research included only the design of the mailing package materials, the design of the voter’s certificate, and the package instructions. A key goal was to make the voting package more secure and private by eliminating signatures going through the open mail on the outside of the envelope; and separating the voter’s name and signature from the voter’s ballot to preserve anonymity of the ballot. While maintaining straightforwardness, ease of use, and trust in the process for low literacy voters
Key components of a vote-by-mail package:
-
The envelope containing all materials addressed to the voter
-
The ballot
-
Voter declaration of voting and secrecy
-
Secrecy for the ballot
-
A means to mail the ballot back to the board of elections
-
Information required about the person providing assistance, if assistance is needed
Three successive rounds of testing were performed. All participants in the testing sessions scored at a literacy level defined as low and a few had been diagnosed with mild cognitive impairment.
In the first testing round, participants were asked to choose two of six absentee packages currently being used in different states. Then to use the two to vote. This was to get a broad sense of process, layout and instructions that were working. The results produced four rejected ballots out of a total of 18. The results also demonstrated a need for the following practices.
Make the instructions complete.
-
Provide a simple instruction sheet that walks voters through the entire process.
-
Let voters know at the start what they will need in order to complete the process.
-
Include the final step of telling voters to mail or deliver their ballot.
Use visual design to make instructions and other text easy for voters to interpret.
-
Instructions for all voters should be larger and more prominently displayed than instructions for “special case” instances (e.g., on the front v. on the back). If this is not done, low-literacy voters are likely to follow all instructions, not just those that pertain to them.
-
Provide as many nonverbal cues as possible to help voters know what they need to do. These can include size, positioning, and sequence of text; size and positioning of objects (e.g., envelopes, instructions sheets); use of arrows, highlighted arrows, etc.; and illustrations.
-
Present instructions in a single-column format, with sequence clearly indicated through positioning and use of numbers.
-
Provide a checklist to help voters track their progress through multiple required actions.
-
Provide an accurate illustration for each action voters must take, whenever possible.
-
Provide white space between and around instructions to make them easier to follow and less intimidating.
-
Make text and line spacing large enough that voters can follow with their finger.
-
Make sure that voters can refer to instructions easily while completing the associated action.
Use clear language that is easy to understand.
-
Use simple, consistent terminology, without jargon.
-
Use imperative voice for instructions, make them as simple as possible, and do not over-explain.
Design the envelope to make it easy to use.
-
Make directive text on the envelope simple and large.
-
Avoid putting directive text in the postage area, as voters will often interpret this as meaning that postage is prepaid (regardless of the actual text).
-
Keep in mind that the size of envelopes will be interpreted by voters as a clue to what goes inside the envelope.
For the second round of testing, two prototypes were developed. Each was designed to increase voter privacy and security by ensuring voter signatures were not sent through the open mail and were also separated from the sealed ballot. The first used three separate envelopes, one each for ballot secrecy, signature and mailing with instructions. This version went through five versions during this round. The second used a single but more complicated envelope that incorporated a sealed secure pocket for the ballot and a removable flap for the voter’s signature with instructions. This prototype went through eight versions during the round of testing. The final three participants had no problems, indicating we could move forward with comparative testing.
During round three of testing, participants were observed using both the current absentee package for the state of Maryland and the single envelope solution used in round two. The single envelope solution was the focus of round three because it is likely to be more economical and it was more challenging for participants, so clarity and usability need to be ensured. The Maryland package uses one envelope with the voter’s signature on the outside of the return envelope. After using both packages and answering rating questions, the participants were asked for a preference. Twelve chose the one-envelope prototype, six chose the current Maryland solution and two expressed no preference.
Best practices for developing ballots are published by The Center for Civic Design, a partner in this project along with the Maryland Board of Elections. The Center for Civic Design is a non-profit whose mission is to make every interaction between government and citizens easy, effective, and pleasant. Field guides with design guidelines can used by election officials for decisions based on research and best practices given that each county or voting district throughout the country is responsible for creating their own ballots.
RENT ESCROW PROCEDURE
Maryland District Court
April ‘15-June ‘16
Between 2012 and 2015, an analysis of more than 300 rent escrow cases in the city of Baltimore revealed that neither judges nor plaintiffs in rent escrow cases understood or were correctly applying the relevant laws. Tenants didn’t understand the applicable laws or available remedies and were unable to fill out their rent escrow applications accurately. Judges knew that the documents provided by tenants were probably filled out incorrectly, so they routinely ignored these documents. Therefore, many rent escrow cases were decided based on established custom rather than on the facts of the case or relevant laws, and the results tended to favor landlords.
A year of iterative design, testing, and re-design resulted in a rent escrow form that could (1) help tenants understand their options and provide accurate information to the court, and (2) provide judges with accurate information while simultaneously reinforcing their understanding of the applicable laws. To reach a final design, 18 iterations were necessary.
ELECTION POLLING PLACE JUDGE PROCESS
Maryland state board of elections
April ‘15-June ‘16
The Maryland State Board of Elections requested an evaluation of all instructions and forms for Election-day Poll Workers.
This work necessitated a process audit of the required duties for election judges. Once a specific process was either determined usable or revamped, redesign was implemented. Processes spread across multiple forms often faltered when information was transferred. Discrepancies in the type of information required was a common problem.
For example: a tally of spoiled ballots is kept on one form as the day commences with no specific information required. When the number is transferred to a reporting sheet, the ballots need to be broken out by party and that information was not collected throughout the day. Further, when the end-to-end process was audited, the party affiliation was actually not required for the reporting sheet (this is for demonstration purposes only, specifics may have varied).
Some of the processes varied depending on the type of election, primary or general. There are a total of 19 forms and instruction sheets. One of the 19 was determined to be workable without changes. The remaining 18 required audits and iterative testing. Phases for each form ranged from 4-12 iterations. A nomenclature was implemented by adding letters at the beginning of each unique piece.
All redesigned forms had to be customizable. Each county needed access to edit for specific requirements or information once they received the new templates. This determined that word templates as opposed to pdfs had to be used. This complicated the matter as several of these files were more suited for a page layout tool. New photos were needed to more clearly represent needed actions and a photo shoot at the Howard County Board of Elections was done.
Best practices were followed for both simplified language and design using visual directives for process with clearly implied actions as much as possible. User testing was done with current and past election judges and chief judges.
Initial general observations were:
-
Processes were not accurately reflected in the forms, particularly when forms needed to work in concert with one another
-
Information hierarchy was not consistent or non-existent
-
Photos needed for demonstration were not helpful and often inaccurate
-
Steps were often mislabeled creating misdirection
Initial assessments for individual pieces with a final iteration count:
A_Ballot Opening & Closing Certificate_general only
required 12 iterations
-
Opening and Closing actions were not differentiated
-
Implied actions were unclear
-
Lists items were not parallel in action
-
Hierarchy was misleading
B_Ballot Opening Certificate_primary only
required 10 iterations
-
Numbered list implied steps as opposed to independent actions
-
Hierarchy was misleading
C_Ballot Closing Certificate_primary only
required 5 iterations
-
Broken process
-
Information needed at closing was not recorded
-
Totals required for summary forms were not available
-
Actions were not directed visually
-
Instructions were unclear
-
Hierarchy was misleading
D_BMD Integrity Report
required 5 iterations
-
Action indicators were unclear
-
Photo of equipment was out of date
-
Hierarchy was misleading
E_Closing Summary Report
required 6 iterations
-
This report is used at the end of a 12-hour day and visual strain is problematic
-
Numbers transferred could easily be incorrect
-
Categories had different names from form to form
-
Directives within the information are unclear
-
Hierarchy was misleading
-
The directions for the form were unusable and steps were not indicated in the correct order. The instruction sheet required each new form/process after process audits and design revisions were completed. After assessing the instructions, we decided to break the report and instructions into two separate pieces
E1_Closing Summary Report Instructions
required 5 iterations
-
Unusable
-
Steps were not indicated in the correct order
-
This report is used at the end of a 12-hour day and visual strain is common
-
The report is dependent on several other processes that needed auditing
-
Directives are used inconsistently
-
Heirarchy is misleading
G_EPB Integrity Report
-
No changes necessary
H_Provisional Ballot Certificate_general only
required 4 iterations
-
Broken process
-
Numbers needed at the end of the day were not produced
-
Hierarchy was misleading
I_Provisional Ballot Certificate_primary only
required 3 iterations
-
Broken process
-
Numbers needed at the end of the day were not produced
-
Hierarchy was misleading
J_Provisional Ballot Tally Sheet_general only
required 6 iterations
-
Broken process
-
Numbers needed at the end of the day were not produced
-
Hierarchy was misleading
K_Provisional Ballot Tally Sheet_primary only
required 7 iterations
-
Broken process
-
Information needed was not collected
-
Hierarchy was misleading
L_Provisional Opening and Closing Certificate
required 9 iterations
-
Broken process
-
Information needed was not collected
-
Insignificant information was collected
-
Hierarchy was misleading
M_Spoiled Ballot Tally Sheet
required 4 iterations
-
Broken process
-
Instructions were confusing and needed to be incorporated into the body partially
-
Information collection was inverted
-
Categories did not make sense
-
Hierarchy was misleading
N_Provisional Sign-in Sheet
required 4 iterations
-
Design change to incorporate directive instructions
O_Scanning Unit Opening Integrity Report
required 5 iterations
-
Relationship between information and demonstration photos doesn’t work
-
Photos are inconsistent
-
Photos do not demonstrate primary actions
-
Hierarchy needs clarification
P_Scanning Unit Closing Integrity Report
required 6 iterations
-
Similar problems to opening integrity report
-
Relationship between information and demonstration photos doesn’t work
-
Photos are inconsistent
-
Photos do not demonstrate primary actions
-
Hierarchy needs clarification
Q_Replacing Ballot Transfer Bin Report
required 5 iterations
-
Primary problem concerns demonstration photos
-
Hierarchy needs clarification
View audited and redesigned forms and instructions
View originals
OVERVIEW
Maryland state board of elections
July‘15-ongoing
Work with the Maryland State Board of Elections (MSBE) began in 2015. For each new project, we perform a process analysis, rewrite, redesign, and test.
To now we have worked on the following projects for the MSBE:
Vote by Mail Ballot Instructions & Inserts
Voter Registration Application
Request for Absentee Ballot
Instructions for Poll Workers
Candidate Information Sheet
Election Judge Instruction Book
Driving principles for this work are
-
simplified language, with technical terms stripped out (low literacy audience will skip steps if they become frustrated with words)
-
directive statements or possessive pronouns when possible
-
as few multiple requests per line as possible
-
text size of 14 points (although this is almost always impossible)
-
numbered steps
-
provide format and number of spaces when a number is required (phone, date)
-
when a signature is required, use a box if possible
-
when a signature is required, use an “x” to indicate where
-
overly emphasize visual hierarchy
All MSBE work is done in partnership with Dr. Kathryn Summers.
Sample Project:
Maryland state Request for Absentee Ballot
In addition to the customary challenges of MSBE work, there was an additional challenge presented by the complex instructions for deadlines to return the ballot. Four separate guidelines- two separate deadlines for the primary and the general elections depending on how the voter is returning the ballot. After the third iteration of testing, we created a third separate page with all county and city boards of election information. This enabled us to create a usable format for the instruction side of the form and also provided the MSBE with a sheet with location information that can be used anytime the county board information is needed.
The request for an absentee ballot required 16 iterations of testing.
Early testing notes included:
Micheline
-Did not check instruction side at all
-Did not see “Registered to vote” until second address section
-Did not consider how to return
-Had trouble deciphering deadline data
 Roderick: Realm/64
-Has voted absentee before
-Took time to read instructions (form was handed instruction side up)
-Looked back to instructions when he got to choosing email option
-Understood deadline June 22
-Supposed to email the form to elections.maryland.gov?
-He read the url in bottom right as address where he should email the form
-Noted that emailing the form would require him to have a printer and a -scanner at home, which most people might not have
-Asked why isn’t there a place you can go on the website, log in (to prove your identity) and complete the ballot?
-Not sure if he meant he wanted to vote online or just fill out this form online
-He thought this might already exist
-His assumption of how the voting by email works: they email you the ballot, you print it out, fill it in, scan it, and email it back
-Asked how secure is email? How do they verify your identity?
-His Ideas for voting fraud prevention:
-There should be a fine for tampering with ballots by email to discourage people
-There needs to be some sort of identity verification such as registering finger print or face to face verification using computer’s camera to video chat
Shaun: Realm/66
-Voted by absentee many years ago (1984?)
-Read through directions thoroughly
-Checked mail option, but still filled in email address
-Signed assistance box
-Took time to checked back over form when done
-Understood deadline dates
-Understood he could return by mail or hand deliver to Board of Elections
Zoe: Realm/65
-First time voting was this election (she’s 21 years old), never done absentee
-Confused about whether she should circle Jr (maybe she thought she was required to select one of the options?)
-Confused about where to write middle name
-Chose ballot by mail option but still filled in email address
-Understood deadlines
-Said she wanted the ballot by email (despite the fact that she checked mail on the form)
-To return form she thinks she has to take it in person to Board of Elections
FINAL ITERATION
