When a person must go to court to deal with a divorce, eviction, or credit card debt, or myriad other actions, the justice system can be difficult to navigate without professional help. It’s even more challenging for those who are not proficient in English. In recognition of California’s linguistic diversity — with over 200 languages spoken — the courts have made ‘language access’ a priority for innovation.
This policy practicum worked closely with California state courts to research, design, and test new strategies for making courts accessible to all people, regardless of language or cultural background. The student teams worked directly with court staff, users, and other stakeholders to create new tech-based and human-centered services that can reduce barriers to the justice system. They did fieldwork in local court self-help centers to test out various applications, technology tools, and paper guides to see what kinds of tools could best support people with various language backgrounds and capabilities when they sought legal help.
Students gained experience in design research, prototyping new technologies on-site at court, and building strategies for innovation in the government. As courts decide which technologies and programs best support access, this design research project served as a lab to test, refine, and advise the California Judicial Council on effective means to improve access to the courts for diverse linguistic populations.
Read our final report from the class
Our class authored a visual report for our partner, the Judicial Council of California, in February 2019 that summarizes our class and findings.
The Basics on the Class
In Autumn 2018, Stanford Law School and d.school will offer Design For Justice: Language Access. It will be taught by Margaret Hagan, Janet Martinez, Jonty Markby Olliff-Cooper, and Kursat Ozenc.
The class will be 9:30 AM – 11:30 AM Thursday mornings at Studio 2 at the d.school, and on site at Santa Clara County Court in San Jose. It will be a 2-credit class for a grade, with course code LAW 806M.
Final Deliverables: an evaluation of existing language access technologies based on user testing and on-site evaluations, a proposal of a fuller pilot, and policy recommendations for how courts can prioritize language and cultural inclusion in their innovation investments.
Pre-requisites: We strongly encourage applicants with language skills in Spanish, Russian, Vietnamese, Mandarin, Farsi, and beyond. Much of our work will involve working with people with limited English proficiency, and different language skills will be highly valued.
Elements used in grading: Attendance, Performance, Class Participation, Final Paper.
Outcomes of the Class
Learning Outcomes for the Students
For our 9-week class, students will be trained in the following methods, and will build portfolio and writing items in the following areas:
- New Policy-making Skills: Build new methods and mindsets for making policy, and improving public services, developing civic technology projects
- Human-Centered Design: Bring a strong focus on human-centered design as a problem-solving process, applying it to an existing policy proposal/initiative and learning how to take the design process from abstract descriptions to real application in a wicked, fast-moving project
- Applied Problem-Solving: Generate new policies and prototypes that are grounded in the contexts of real people’s problems in the situation — taking the policymaking from boardrooms and slide presentations, and demonstrate how it can be done more directly with stakeholders, on services’ front lines, and tied closely to solving specific problems.
- Experience with Design Evaluations: Conduct rounds of short, intentional tests that generate knowledge and test key hypotheses for design-driven policymaking — learning some of these evaluation instruments, and also refining them for the specific context
- Viable, Replicable Prototype/Policy-making: Produce a strong, replicable language access technology model in our specific partner court, that could be adjusted and replicated in other courts — along with the process, instruments, and evaluations that are documented for replication in future public interest technology policymaking.
Legal System Problems we will tackle
The prototypes and policy recommendations we create in the class will be aimed towards solving people’s problems while navigating the court — and the court’s challenges in making feasible, scalable technical innovation.
These are some of the guiding problems that we have identified in our early work with the court, to understand what’s going wrong on the ground.
The specific tasks are part of larger problems that court users are trying to solve — the problems around family, employment, debt, and housing that they have come to court to try to resolve. These are the larger design challenges that we aim to resolve, by focusing on particular touchpoints and tasks that people must deal with in court.
The Course Plan
| Date | Activities | Learning Outcomes |
| Week 1,
09/27 |
At the d.school: Design Bootcamp + Our Challenges
Introductions Design Bootcamp Team Challenges |
Essentials of Human-Centered Design
What is design? What is the state of Language Access and innovation in US state courts? How can we use design methods to create new technology plans + policies in government services? |
| To-do’s before Week 2 | Team Work
Each team member should complete CITI training on Human Subject research ethics: https://researchcompliance.stanford.edu/hs/new/resources/training/citi.html [Choose Group 2: IRB Nonmedical Research course] Team carpool to court plans |
Read /// The basics of Design for Dignity + the Legal System
Tim Brown, “Design Thinking,” Harvard Business Review, https://hbr.org/2008/06/design-thinking Margaret Hagan and Miso Kim, “Design for Dignity and Procedural Justice”, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2994354 Design Principles for Wayfinding: http://www.ai.mit.edu/projects/infoarch/publications/mfoltz-thesis/node8.html |
| Week 2,
10/04 |
At the court: Discovering the Status Quo
Tour the court and its Self Help spaces Teams observe their assigned spaces using design research methods Debrief with maps, themes, ideas |
Design Research: Observation + Empathy
How is a relatively well-resourced state court currently serving people with Limited English Proficiency? Where are key touchpoints, breakdowns, and opportunities to focus on? Who are the stakeholders involved, and what do we know about their needs, perspectives, and ways of working? |
| To-do’s before Week 3 | Team Work
Start shared Google Drive and Mural, collect notes, drawings, and photos Make list of top ideas to possibly propose as language access innovations Make list of top questions and ‘unknowns’ to be explored |
Read /// The Language Access Challenge
California Judicial Council: Strategic Plan for Language Access, http://www.courts.ca.gov/documents/CLASP_report_060514.pdf Wayfinding and Signage Strategies for Language Access in the California Courts: http://www.courts.ca.gov/documents/LAP-Wayfinding-and-Signage-Strategies-Language-Access-in-the-CA-Courts.pdf |
| Week 3, 10/11 | At the d.school: Understanding Stakeholders & Reframing the Problem
Interview stakeholders about their experiences, needs, and goals Map out key Personas and their journey map through the system Identify concept-prototypes to test next week at court |
Design Synthesis: Mapping, Personas, and Requirements
What are the key, human problems that new language access innovations would be solving? What is a map of people’s current journey through a court, and where are there opportunities for innovation? What are the requirements + constraints that should guide what prototypes + policies the courts deploy? |
| To-do’s | Team Work
Have Concept-Prototypes v1 prepared as drawings and descriptions, submit to teaching team by Wednesday 12 noon Run practice version of Usability Test with friends, to prepare for running with court users — Ensure all team members have gone through Ethics Training! |
Read /// Usability Testing + Ethics
Doing User Research in the Courts on the Future of Access to Justice: https://medium.com/legal-design-and-innovation/doing-user-research-in-the-courts-on-the-future-of-access-to-justice-cb7a75dc3a4b Ethical Design Engagement with your Community: http://www.legaltechdesign.com/2018/02/short-book-ethical/ This is Service Design Doing: https://www.thisisservicedesigndoing.com/methods (skim Ideation + Prototyping methods) |
| Week 4:
10/18 |
At the court: Usability Testing on site
Teams conduct Usability Tests with court users, ranking different concepts against each other Work with users to draft new or refined ideas for innovations Synthesize the feedback and define next steps for prototypes and policies |
Usability Testing Prototypes
How do people respond to the current first round of language-tech prototypes? What best engages them? What is easiest for them to use? Which helps them solve their key problems + serves their needs? What are other ideas that they would propose to refine the prototypes — or whole new prototypes that would be better? |
| To-do’s before Week 5 | Team Work
Gather test results and all feedback comments, put in shared folder Each team writes a short report (~750 words) on the insights and testing feedback from the on-site work, that will be published on Class Blog |
Read /// Policy + Design
Lucy Kimbell: Discovering the Policy Lab: https://researchingdesignforpolicy.files.wordpress.com/2015/10/kimbell_policylab_report.pdf Sabine Junginger: Design and Innovation in the Public Sector: https://www.researchgate.net/publication/271210543_Design_and_Innovation_in_the_Public_Sector_Matters_of_Design_in_Policy-Making_and_Policy_Implementation Mindlab: Methods: http://mind-lab.dk/en/methods (skim methods repository) |
| Week 5:
10/25 |
At the d.school: Prototyping after feedback
Review and revise current prototypes based on feedback Draft Language Access Innovation Principles Work on round 2 of prototypes and policies to test |
Prototyping + Policy by Design
How do you make use of findings from usability tests and observations? How can you make higher quality prototypes, that have more detail, fidelity, and context? How can you design for policy-making, to have innovations that can change not just the ‘interfaces’ of the system, but also the system itself? |
| To-do’s before Week 6 | Team Work
Prepare Prototype v2 (visual/digital + text description) and share with teaching team by Wednesday 12 noon List out more general ‘policy’ recommendations to accompany prototypes |
Read /// Dignity, Procedural Justice + Perceived Control, through Design
Social Design: http://eprints.brighton.ac.uk/13364/1/Social-Design-Report.pdf John Clarke and B. Borys, Usability is Free: Improving Efficiency by Making the Court More User-Friendly: https://ncsc.contentdm.oclc.org/digital/collection/ctadmin/id/1844/ Tom Tyler, Procedural Justice and the Courts: https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1254&context=ajacourtreview |
| Week 6:
11/1 |
At the court: Testing Round 2, of Prototypes and Policy Plan
Test the new round of prototypes for usability, and also on procedural justice, dignity, and perceived control metrics Gather more feedback and recommendations from court users Speak with staff for their thoughts on implementation and impact |
Testing for Dignity, Justice, and Control
How can you test a mid-level prototype in the field, before proposing higher-cost pilots and evaluations? How can you evaluate a new government innovation in regards to its effects on people’s sense of dignity, of procedural justice, and of perceived control? |
| To-do’s before Week 7 | Team Work
Each team writes a short report (~750 words) on the insights and testing feedback from the on-site work, that will be published on Class Blog Begin work on Final Prototypes, Policy Recommendations, and Presentation |
Read /// Managing innovation in government
Terrey: Managing by Design: case study of Australian Tax Office: http://www.canberra.edu.au/researchrepository/items/5baa04d2-8ea7-a461-dcef-0351978a6b9b/1/ (read introduction + skim rest) Delivering Public Service for the Future: https://lisboncouncil.net/publication/publication/117-delivering-public-service-for-the-future.html |
| Week 7:
11/8 |
At the d.school: Prototyping and planning for impact
Compare testing results team-to-team Make further refined plans for the prototype, how it could be piloted, and how it could be replicated Work on round 3 of prototypes and on final presentation for partners |
High-Fidelity Prototyping + Handoff Planning
How do you use visual and service design methods to make a detailed, almost-real version of your idea? How do you capture your proposal, surrounding details and needs, and testing/customization to run — so that other institutions (not this partner) could possibly adapt it to their unique setting? |
| To-do’s before Week 8 | Team Work
Prepare Prototype v3 (visual/digital + text description) and share with teaching team by Wednesday 12 noon |
Read /// Court Innovation Strategies
Quintanilla: Human-Centered Civil Justice Design, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2655818 Access to Justice: Meeting the Needs of Self Represented Litigants, https://www.kentlaw.iit.edu/sites/ck/files/public/component/access-to-justice-meeting-the-needs.pdf (skim proposals + presentation modes) |
| Week 8: 11/15 | At the court: Final On-Site Feedback
Teams conduct final set of tests with court users and staff Gather particular feedback about what could be feasible, sustainable, and scalable, to be communicated to partners Debrief with all teams, about presentations in final week |
Testing methods for new prototypes/policies, pre-pilot
How do you gather critical feedback from many stakeholders about an innovation strategy, before you propose it to leaders? What are the suite of tests you can run to gather preliminary evaluations of your prototype — before you invest in a full pilot and rigorous trial? |
| To-do’s | Team Work
Draft presentation and begin write-up of your team’s proposal for individual prototypes, overarching policies, and a pilot and evaluation plan |
No required reading: work on presentations! |
| Week 9:
11/29 |
At the d.school: Presentations and Review
Teams present to court partners about their final prototypes and policy recommendations, and how they could be piloted and replicated Stakeholders review the recommendations, ask further questions, and help refine the ultimate proposals |
Building a coalition for your proposal
How do you persuade government leaders and staff to take your prototype and policy plan forward? How do you package your proposal into a story and a resource, so that other courts could easily understand it and possibly adopt it? |
| To-do’s | Teams share their revised final presentations and write-up to teaching team
Teaching team will synthesize them into a final report for the partners |
The Class Reading List
Readings on language access in courts
California Judicial Council: Strategic Plan for Language Access, http://www.courts.ca.gov/documents/CLASP_report_060514.pdf
U.S. Department of Justice Civil Rights Division. 2016. “Language Access in State Courts.” https://www.justice.gov/crt/file/892036/download
National Center for State Courts: A National Call to Action Access to Justice for Limited English Proficient Litigants: Creating Solutions to Language Barriers in State Courts, https://www.ncsc.org/services-and-experts/areas-of-expertise/language-access/~/media/files/pdf/services%20and%20experts/areas%20of%20expertise/language%20access/call-to-action.ashx
Brennan Center: Language Access in State Courts, http://www.brennancenter.org/sites/default/files/legacy/publications/154142_LangWhite.pdf
Shepard: Access to Justice for People Who Do Not Speak English, https://mckinneylaw.iu.edu/ilr/pdf/vol40p643.pdf
Bell: The Case for Spanish-speaking courts, https://qz.com/1202841/the-case-for-spanish-speaking-courts-in-places-like-the-bronx-and-all-over-the-us/
Justice Department and Los Angeles County Superior Court Reach Agreement to Ensure Access to Justice for Limited English Proficient Individuals: https://www.justice.gov/opa/pr/justice-department-and-los-angeles-county-superior-court-reach-agreement-ensure-access
Wayfinding and Signage Strategies for Language Access in the California Courts: http://www.courts.ca.gov/documents/LAP-Wayfinding-and-Signage-Strategies-Language-Access-in-the-CA-Courts.pdf
National Center for State Courts/ State Justice Initiative: A National Call to Action Access to Justice for Limited English Proficient Litigants: Creating Solutions to Language Barriers in State Courts
Popular Democracy brief: Language Access https://populardemocracy.org/sites/default/files/languageaccess_11162013.pdf
Language Access Assessment and Planning Tool for federal programs, US DOJ Civil Rights Division, 2011, https://www.lep.gov/resources/2011_Language_Access_Assessment_and_Planning_Tool.pdf
National Center for State Courts: Language Access Services Section: https://www.ncsc.org/languageaccess
Readings on Legal Design for people without lawyers, and with language support
Hagan and Kim: Design for Dignity and Procedural Justice, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2994354
Hagan: A Human Centered Design Approach to Access to Justice, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3186101
Zorza: Self-Help Friendly Court, http://www.zorza.net/Res_ProSe_SelfHelpCtPub.pdf
Tyler, Tom: Procedural Justice and the Courts, Court Review, http://www.proceduralfairness.org/~/media/Microsites/Files/procedural-fairness/Tyler.ashx
Quintanilla: Human-Centered Civil Justice Design, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2655818
Access to Justice: Meeting the Needs of Self Represented Litigants, https://www.kentlaw.iit.edu/sites/ck/files/public/component/access-to-justice-meeting-the-needs.pdf
Crawford: Wired: Why Universities Need Public Interest Technology Courses, https://www.wired.com/story/universities-public-interest-technology-courses-programs
Readings on Design Methods
This is Service Design Doing: https://www.thisisservicedesigndoing.com/methods
Nesta: Innovation Policy Toolkit: https://media.nesta.org.uk/documents/tradecraftinnodiplomats_innovationpolicytoolkit.pdf
Mindlab: Methods: http://mind-lab.dk/en/methods
Ethical Design Engagement: http://www.legaltechdesign.com/2018/02/short-book-ethical/
Readings on Wayfinding and Visual Design Support in Government Services
Design Principles for Wayfinding: http://www.ai.mit.edu/projects/infoarch/publications/mfoltz-thesis/node8.html
Designing election systems for language access: https://civicdesign.org/wp-content/uploads/2017/07/Language-Access-V10-17-0710a.pdf
Wayfinding System Audit http://www.hpw.qld.gov.au/sitecollectiondocuments/wayfindingsystemaudit.pdf
99 Designs: Wayfinding in cities and buildings, https://99designs.com/blog/design-other/how-graphic-design-helps-us-navigate-built-environments/
Art of Smart Wayfinding, http://thoughtmatter.com/art-of-smart-wayfinding/
Latest Technologies in Wayfinding, https://www.travelwayfinding.com/latest-technologies/
Tech Options for Digital Wayfinding , https://www.hfmmagazine.com/articles/3325-tech-options-abound-for-digital-wayfinding
Southwest Design work at airports: https://www.continuuminnovation.com/en/what-we-do/case-studies/digital-wayfinding-design-and-prototype
Policy + Design
Social Design: http://eprints.brighton.ac.uk/13364/1/Social-Design-Report.pdf
Kimbell: Discovering the Policy Lab: https://researchingdesignforpolicy.files.wordpress.com/2015/10/kimbell_policylab_report.pdf
Delivering Public Service for the Future: https://lisboncouncil.net/publication/publication/117-delivering-public-service-for-the-future.html
Junginger: Design and Innovation in the Public Sector: https://www.researchgate.net/publication/271210543_Design_and_Innovation_in_the_Public_Sector_Matters_of_Design_in_Policy-Making_and_Policy_Implementation
Terrey: Managing by Design: case study of Australian Tax Office: http://www.canberra.edu.au/researchrepository/items/5baa04d2-8ea7-a461-dcef-0351978a6b9b/1/
What is the current vision of how we can bring innovation to court’s language access services?
