How can we best use technology to support people going through court without a lawyer, when they are not proficient in English?

When a person must go to court to deal with a divorce, eviction, or credit card debt, or myriad other actions, the justice system can be difficult to navigate without professional help. It’s even more challenging for those who are not proficient in English. In recognition of California’s linguistic diversity — with over 200 languages spoken — the courts have made ‘language access’ a priority for innovation.

This policy practicum works closely with California state courts to research, design, and test new strategies for making courts accessible to all people, regardless of language or cultural background. The student teams will work directly with court staff, users, and other stakeholders to create new tech-based and human-centered services that can reduce barriers to the justice system.

Students will gain experience in design research, prototyping new technologies on-site at court, and building strategies for innovation in the government. As courts decide which technologies and programs best support access, this design research project serves as a lab to test, refine, and advise the California Judicial Council on effective means to improve access to the courts for diverse linguistic populations.




In Autumn 2018, Stanford Law School and d.school will offer Design For Justice: Language Access. It will be taught by Margaret Hagan, Janet Martinez, Jonty Markby Olliff-Cooper, and Kursat Ozenc.

The class will be 9:30 AM – 11:30 AM Thursday mornings at Studio 2 at the d.school, and on site at Santa Clara County Court in San Jose. It will be a 2-credit class for a grade, with course code LAW 806M.

Final Deliverables: an evaluation of existing language access technologies based on user testing and on-site evaluations, a proposal of a fuller pilot, and policy recommendations for how courts can prioritize language and cultural inclusion in their innovation investments.

Pre-requisites: We strongly encourage applicants with language skills in Spanish, Russian, Vietnamese, Mandarin, Farsi, and beyond. Much of our work will involve working with people with limited English proficiency, and different language skills will be highly valued.

Elements used in grading: Attendance, Performance, Class Participation, Final Paper.



Learning Outcomes for the Students

For our 9-week class, students will be trained in the following methods, and will build portfolio and writing items in the following areas:

  1. New Policy-making Skills: Build new methods and mindsets for making policy, and improving public services, developing civic technology projects
  2. Human-Centered Design: Bring a strong focus on human-centered design as a problem-solving process, applying it to an existing policy proposal/initiative and learning how to take the design process from abstract descriptions to real application in a wicked, fast-moving project
  3. Applied Problem-Solving: Generate new policies and prototypes that are grounded in the contexts of real people’s problems in the situation — taking the policymaking from boardrooms and slide presentations, and demonstrate how it can be done more directly with stakeholders, on services’ front lines, and tied closely to solving specific problems.
  4. Experience with Design Evaluations: Conduct rounds of short, intentional tests that generate knowledge and test key hypotheses for design-driven policymaking — learning some of these evaluation instruments, and also refining them for the specific context
  5. Viable, Replicable Prototype/Policy-making: Produce a strong, replicable  language access technology model in our specific partner court, that could be adjusted and replicated in other courts — along with the process, instruments, and evaluations that are documented for replication in future public interest technology policymaking.

Legal System Problems we will tackle

The prototypes and policy recommendations we create in the class will be aimed towards solving people’s problems while navigating the court — and the court’s challenges in making feasible, scalable technical innovation.

These are some of the guiding problems that we have identified in our early work with the court, to understand what’s going wrong on the ground.

The specific tasks are part of larger problems that court users are trying to solve — the problems around family, employment, debt, and housing that they have come to court to try to resolve. These are the larger design challenges that we aim to resolve, by focusing on particular touchpoints and tasks that people must deal with in court.


The Course Plan

Date Activities Learning Outcomes
Week 1,

09/27

At the d.school: Design Bootcamp + Our Challenges

Introductions

Design Bootcamp

Team Challenges

Essentials of Human-Centered Design

What is design?

What is the state of Language Access and innovation in US state courts?

How can we use design methods to create new technology plans + policies in government services?

To-do’s before Week 2 Team Work

Each team member should complete CITI training on Human Subject research ethics: https://researchcompliance.stanford.edu/hs/new/resources/training/citi.html

[Choose Group 2: IRB Nonmedical Research course]

Team carpool to court plans

Read /// The basics of Design for Dignity + the Legal System

Tim Brown, “Design Thinking,” Harvard Business Review, https://hbr.org/2008/06/design-thinking

Margaret Hagan and Miso Kim, “Design for Dignity and Procedural Justice”, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2994354

Design Principles for Wayfinding: http://www.ai.mit.edu/projects/infoarch/publications/mfoltz-thesis/node8.html

Week 2,

10/04

At the court: Discovering the Status Quo

Tour the court and its Self Help spaces

Teams observe their assigned spaces using design research methods

Debrief with maps, themes, ideas

Design Research: Observation + Empathy

How is a relatively well-resourced state court currently serving people with Limited English Proficiency?

Where are key touchpoints, breakdowns, and opportunities to focus on?

Who are the stakeholders involved, and what do we know about their needs, perspectives, and ways of working?

To-do’s before Week 3 Team Work

Start shared Google Drive and Mural, collect notes, drawings, and photos

Make list of top ideas to possibly propose as language access innovations

Make list of top questions and ‘unknowns’ to be explored

Read /// The Language Access Challenge

California Judicial Council: Strategic Plan for Language Access, http://www.courts.ca.gov/documents/CLASP_report_060514.pdf

Wayfinding and Signage Strategies for Language Access in the California Courts: http://www.courts.ca.gov/documents/LAP-Wayfinding-and-Signage-Strategies-Language-Access-in-the-CA-Courts.pdf

Week 3, 10/11 At the d.school: Understanding Stakeholders & Reframing the Problem

Interview stakeholders about their experiences, needs, and goals

Map out key Personas and their journey map through the system

Identify concept-prototypes to test next week at court

Design Synthesis: Mapping, Personas, and Requirements

What are the key, human problems that new language access innovations would be solving?

What is a map of people’s current journey through a court, and where are there opportunities for innovation?

What are the requirements + constraints that should guide what prototypes + policies the courts deploy?

To-do’s Team Work

Have Concept-Prototypes v1 prepared as drawings and descriptions, submit to teaching team by Wednesday 12 noon

Run practice version of Usability Test with friends, to prepare for running with court users

— Ensure all team members have gone through Ethics Training!

Read /// Usability Testing + Ethics

Doing User Research in the Courts on the Future of Access to Justice: https://medium.com/legal-design-and-innovation/doing-user-research-in-the-courts-on-the-future-of-access-to-justice-cb7a75dc3a4b

Ethical Design Engagement with your Community: http://www.legaltechdesign.com/2018/02/short-book-ethical/

This is Service Design Doing: https://www.thisisservicedesigndoing.com/methods (skim Ideation + Prototyping methods)

Week 4:

10/18

At the court: Usability Testing on site

Teams conduct Usability Tests with court users, ranking different concepts against each other

Work with users to draft new or refined ideas for innovations

Synthesize the feedback and define next steps for prototypes and policies

Usability Testing Prototypes

How do people respond to the current first round of language-tech prototypes?

What best engages them?

What is easiest for them to use?

Which helps them solve their key problems + serves their needs?

What are other ideas that they would propose to refine the prototypes — or whole new prototypes that would be better?

To-do’s before Week 5 Team Work

Gather test results and all feedback comments, put in shared folder

Each team writes a short report (~750 words) on the insights and testing feedback from the on-site work, that will be published on Class Blog

Read /// Policy + Design

Lucy Kimbell: Discovering the Policy Lab: https://researchingdesignforpolicy.files.wordpress.com/2015/10/kimbell_policylab_report.pdf

Sabine Junginger: Design and Innovation in the Public Sector: https://www.researchgate.net/publication/271210543_Design_and_Innovation_in_the_Public_Sector_Matters_of_Design_in_Policy-Making_and_Policy_Implementation

Mindlab: Methods: http://mind-lab.dk/en/methods (skim methods repository)

Week 5:

10/25

At the d.school: Prototyping after feedback

Review and revise current prototypes based on feedback

Draft Language Access Innovation Principles

Work on round 2 of prototypes and policies to test

Prototyping + Policy by Design

How do you make use of findings from usability tests and observations?

How can you make higher quality prototypes, that have more detail, fidelity, and context?

How can you design for policy-making, to have innovations that can change not just the ‘interfaces’ of the system, but also the system itself?

To-do’s before Week 6 Team Work

Prepare Prototype v2 (visual/digital + text description) and share with teaching team by Wednesday 12 noon

List out more general ‘policy’ recommendations to accompany prototypes

Read /// Dignity, Procedural Justice + Perceived Control, through Design

Social Design: http://eprints.brighton.ac.uk/13364/1/Social-Design-Report.pdf

John Clarke and B. Borys, Usability is Free: Improving Efficiency by Making the Court More User-Friendly: https://ncsc.contentdm.oclc.org/digital/collection/ctadmin/id/1844/

Tom Tyler, Procedural Justice and the Courts: https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1254&context=ajacourtreview

Week 6:

11/1

At the court: Testing Round 2, of Prototypes and Policy Plan

Test the new round of prototypes for usability, and also on procedural justice, dignity, and perceived control metrics

Gather more feedback and recommendations from court users

Speak with staff for their thoughts on implementation and impact

Testing for Dignity, Justice, and Control

How can you test a mid-level prototype in the field, before proposing higher-cost pilots and evaluations?

How can you evaluate a new government innovation in regards to its effects on people’s sense of dignity, of procedural justice, and of perceived control?

To-do’s before Week 7 Team Work

Each team writes a short report (~750 words) on the insights and testing feedback from the on-site work, that will be published on Class Blog

Begin work on Final Prototypes, Policy Recommendations, and Presentation

Read ///  Managing innovation in government

Terrey: Managing by Design: case study of Australian Tax Office: http://www.canberra.edu.au/researchrepository/items/5baa04d2-8ea7-a461-dcef-0351978a6b9b/1/ (read introduction + skim rest)

Delivering Public Service for the Future: https://lisboncouncil.net/publication/publication/117-delivering-public-service-for-the-future.html

Week 7:

11/8

At the d.school: Prototyping and planning for impact

Compare testing results team-to-team

Make further refined plans for the prototype, how it could be piloted, and how it could be replicated

Work on round 3 of prototypes and on final presentation for partners

High-Fidelity Prototyping + Handoff Planning

How do you use visual and service design methods to make a detailed, almost-real version of your idea?

How do you capture your proposal, surrounding details and needs, and testing/customization to run — so that other institutions (not this partner) could possibly adapt it to their unique setting?

To-do’s before Week 8 Team Work

Prepare Prototype v3 (visual/digital + text description) and share with teaching team by Wednesday 12 noon

Read /// Court Innovation Strategies

Quintanilla: Human-Centered Civil Justice Design, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2655818

Access to Justice: Meeting the Needs of Self Represented Litigants, https://www.kentlaw.iit.edu/sites/ck/files/public/component/access-to-justice-meeting-the-needs.pdf (skim proposals + presentation modes)

Week 8: 11/15 At the court: Final On-Site Feedback

Teams conduct final set of tests with court users and staff

Gather particular feedback about what could be feasible, sustainable, and scalable, to be communicated to partners

Debrief with all teams, about presentations in final week

Testing methods for new prototypes/policies, pre-pilot

How do you gather critical feedback from many stakeholders about an innovation strategy, before you propose it to leaders?

What are the suite of tests you can run to gather preliminary evaluations of your prototype — before you invest in a full pilot and rigorous trial?

To-do’s Team Work

Draft presentation and begin write-up of your team’s proposal for individual prototypes, overarching policies, and a pilot and evaluation plan

No required reading: work on presentations!
Week 9:

11/29

At the d.school: Presentations and Review

Teams present to court partners about their final prototypes and policy recommendations, and how they could be piloted and replicated

Stakeholders review the recommendations, ask further questions, and help refine the ultimate proposals

Building a coalition for your proposal

How do you persuade government leaders and staff to take your prototype and policy plan forward?

How do you package your proposal into a story and a resource, so that other courts could easily understand it and possibly adopt it?

To-do’s Teams share their revised final presentations and write-up to teaching team

Teaching team will synthesize them into a final report for the partners

 

 


Readings on language access in courts

California Judicial Council: Strategic Plan for Language Access, http://www.courts.ca.gov/documents/CLASP_report_060514.pdf

U.S. Department of Justice Civil Rights Division. 2016. “Language Access in State Courts.”  https://www.justice.gov/crt/file/892036/download

National Center for State Courts: A National Call to Action Access to Justice for Limited English Proficient Litigants: Creating Solutions to Language Barriers in State Courts, https://www.ncsc.org/services-and-experts/areas-of-expertise/language-access/~/media/files/pdf/services%20and%20experts/areas%20of%20expertise/language%20access/call-to-action.ashx

Brennan Center: Language Access in State Courts, http://www.brennancenter.org/sites/default/files/legacy/publications/154142_LangWhite.pdf

Shepard: Access to Justice for People Who Do Not Speak English, https://mckinneylaw.iu.edu/ilr/pdf/vol40p643.pdf

Bell: The Case for Spanish-speaking courts, https://qz.com/1202841/the-case-for-spanish-speaking-courts-in-places-like-the-bronx-and-all-over-the-us/

Justice Department and Los Angeles County Superior Court Reach Agreement to Ensure Access to Justice for Limited English Proficient Individuals: https://www.justice.gov/opa/pr/justice-department-and-los-angeles-county-superior-court-reach-agreement-ensure-access

Wayfinding and Signage Strategies for Language Access in the California Courts: http://www.courts.ca.gov/documents/LAP-Wayfinding-and-Signage-Strategies-Language-Access-in-the-CA-Courts.pdf

National Center for State Courts/ State Justice Initiative: A National Call to Action Access to Justice for Limited English Proficient Litigants: Creating Solutions to Language Barriers in State Courts

Popular Democracy brief: Language Access https://populardemocracy.org/sites/default/files/languageaccess_11162013.pdf

Language Access Assessment and Planning Tool for federal programs, US DOJ Civil Rights Division, 2011, https://www.lep.gov/resources/2011_Language_Access_Assessment_and_Planning_Tool.pdf

National Center for State Courts: Language Access Services Section: https://www.ncsc.org/languageaccess


Readings on Legal Design  for people without lawyers, and with language support

Hagan and Kim: Design for Dignity and Procedural Justice, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2994354

Hagan: A Human Centered Design Approach to Access to Justice, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3186101

Zorza: Self-Help Friendly Court, http://www.zorza.net/Res_ProSe_SelfHelpCtPub.pdf

Tyler, Tom: Procedural Justice and the Courts, Court Review, http://www.proceduralfairness.org/~/media/Microsites/Files/procedural-fairness/Tyler.ashx

Quintanilla: Human-Centered Civil Justice Design, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2655818

Access to Justice: Meeting the Needs of Self Represented Litigants, https://www.kentlaw.iit.edu/sites/ck/files/public/component/access-to-justice-meeting-the-needs.pdf

Crawford: Wired: Why Universities Need Public Interest Technology Courses, https://www.wired.com/story/universities-public-interest-technology-courses-programs


Readings on Design Methods

This is Service Design Doing: https://www.thisisservicedesigndoing.com/methods

Nesta: Innovation Policy Toolkit: https://media.nesta.org.uk/documents/tradecraftinnodiplomats_innovationpolicytoolkit.pdf

Mindlab: Methods: http://mind-lab.dk/en/methods

Ethical Design Engagement: http://www.legaltechdesign.com/2018/02/short-book-ethical/


Readings on Wayfinding and Visual Design Support in Government Services

Design Principles for Wayfinding: http://www.ai.mit.edu/projects/infoarch/publications/mfoltz-thesis/node8.html

Designing election systems for language access: https://civicdesign.org/wp-content/uploads/2017/07/Language-Access-V10-17-0710a.pdf

Wayfinding System Audit  http://www.hpw.qld.gov.au/sitecollectiondocuments/wayfindingsystemaudit.pdf

99 Designs: Wayfinding in cities and buildings, https://99designs.com/blog/design-other/how-graphic-design-helps-us-navigate-built-environments/

Art of Smart Wayfinding, http://thoughtmatter.com/art-of-smart-wayfinding/

Latest Technologies in Wayfinding, https://www.travelwayfinding.com/latest-technologies/

Tech Options for Digital Wayfinding , https://www.hfmmagazine.com/articles/3325-tech-options-abound-for-digital-wayfinding

Southwest Design work at airports: https://www.continuuminnovation.com/en/what-we-do/case-studies/digital-wayfinding-design-and-prototype


Policy + Design

Social Design: http://eprints.brighton.ac.uk/13364/1/Social-Design-Report.pdf

Kimbell: Discovering the Policy Lab: https://researchingdesignforpolicy.files.wordpress.com/2015/10/kimbell_policylab_report.pdf

Delivering Public Service for the Future: https://lisboncouncil.net/publication/publication/117-delivering-public-service-for-the-future.html

Junginger: Design and Innovation in the Public Sector: https://www.researchgate.net/publication/271210543_Design_and_Innovation_in_the_Public_Sector_Matters_of_Design_in_Policy-Making_and_Policy_Implementation

Terrey: Managing by Design: case study of Australian Tax Office: http://www.canberra.edu.au/researchrepository/items/5baa04d2-8ea7-a461-dcef-0351978a6b9b/1/


What is the current vision of how we can bring innovation to court’s language access services?