Tag: language access

Categories
AI + Access to Justice Current Projects

AI, Machine Translation, and Access to Justice

Lessons from Cristina Llop’s Work on Language Access in the Legal System

Artificial intelligence (AI) and machine translation (MT) are often seen as tools with the potential to expand access to justice, especially for non-English speakers in the U.S. legal system. However, while AI-driven translation tools like Google Translate and AutoML offer impressive accuracy in general contexts, their effectiveness in legal settings remains questionable.

At the Stanford Legal Design Lab’s AI and Access to Justice research webinar on February 7, 2025, legal expert Cristina Llop shared her observations from reviewing live translations between legal providers’ staff and users. Her findings highlight both the potential and pitfalls of using AI for language access in legal settings. This article explores how AI performs in practice, where it can be useful, and why human oversight, national standards, and improved training datasets are critical.

How Machine Translation Performs in Legal Contexts

Many courts and legal service providers have turned to AI-powered Neural Machine Translation (NMT) models like Google Translate to help bridge language barriers. While AI is improving, Llop’s research suggests that accuracy in general language translation does not necessarily translate to legal language accuracy.

1. The Good: AI Can Be Useful in Certain Scenarios

Machine translation tools can provide immediate, cost-effective assistance in specific legal language tasks, such as:

  • Translating declarations and witness statements
  • Converting court forms and pleadings into different languages
  • Making legal guides and court websites more accessible
  • Supporting real-time interpretation in court help centers and clerk offices

This can be especially valuable in resource-strapped courts and legal aid groups that lack human interpreters for every case. However, Llop cautions that even when AI-generated translations sound fluent, they may not be legally precise or safe to rely on.

AI doesn’t pick up on legal context and mis-translates key information about trials, filing, court, and options.

2. The Bad: Accuracy Breaks Down in Legal Contexts

Llop identified systematic mistranslations that could have serious consequences:

Common legal terms are mistranslated due to a lack of specialized training data. For example, “warrant” is often translated as “court order,” which downplays the severity of a legal document.

Contextual misunderstandings lead to serious errors:

  • “Due date” was mistranslated as “date to give birth.”
  • “Trial” was often translated as “test.”
  • “Charged with a battery a case” turned into “loaded with a case of batteries.”

Pronoun confusion creates ambiguity:

  • Spanish’s use of “su” (your/his/her/their) is often mistranslated in legal documents, leading to uncertainty about property ownership, responsibility, or court filings.
  • In restraining order cases, it was unclear who was accusing whom, which could put victims at risk.

AI can introduce gender biases:

  • Words with no inherent gender (e.g., “politician”) are often translated as male.
  • The Spanish “Me Maltrata”, which could be translated either as She mistreats me or He mistreats me — without the gender being specified. The machine would default “me maltrata” as “He mistreats me,” potentially distorting evidence in domestic violence cases.

Without human review, these AI-driven errors can go unnoticed, leading to severe legal consequences.

The Dangers of Mistranslation in Legal Interactions

One of the most troubling findings from Llop’s work was the invisible breakdowns in communication between legal providers and non-English speakers.

1. Parallel Conversations Instead of Communication

In many cases, both parties believed they were exchanging information, but in reality:

  • Legal providers were missing key facts from litigants.
  • Users did not realize that their information was misunderstood or misrepresented.
  • Critical details — such as the nature of an abuse claim or financial disclosures — were being lost.

This failure to communicate accurately could result in:

  • People choosing the wrong legal recourse and misunderstanding what options are available to them.
  • Legal provider staff making decisions based on incomplete or distorted information, providing services and option menus based on misunderstandings about the person’s scenario or preferences.
  • Access to justice being compromised for vulnerable litigants.

2. Why a Glossary Isn’t Enough

Some legal institutions have tried to mitigate errors by adding legal glossaries to machine translation tools. However, Llop’s research found that glossary-based corrections do not always solve the problem:

  • Example 1: The word “address” was provided to the AI to ensure translation to “mailing address” (instead of “home address”) in one context — but then mistakenly applied when a clerk asked, “What issue do you want to address?”
  • Example 2: “Will” (as in a legal document) was mistranslated when applied to the auxiliary verb “will” in regular interactions (“I will send you this form”).
  • Example 3: A glossary fix for “due date” worked .
  • “Example 4: A glossary fix for “pleading” worked but failed to adjust grammatical structure or pronoun usage.”

These patchwork fixes are not enough. More comprehensive training, oversight, and quality control are needed.

Advancing Legal Language AI: AutoML and Human Review

One promising improvement is AutoML, which allows legal organizations to train machine translation models with their own specialized legal data.

AutoML: A Step Forward, But Still Flawed

Llop’s team worked on an AutoML project by:

  1. Collecting 8,000+ legal translation pairs from official legal sources that had been translated by experts.
  2. Correcting AI-generated translations manually.
  3. Feeding improved translations back into the model.
  4. Iterating until translations were more accurate.

Results showed that AutoML improved translation quality, but major issues remained:

  • AI struggled with conversational context. If a prior sentence referenced “my wife,” but the next message about the wife didn’t specify gender, AI might mistakenly switch the pronoun to “he”.
  • AI overfit to common legal phrases, inserting “petition” even when the correct translation should have been “form.”

These challenges highlight why human review is essential.

Real-Time Machine Translation

While text-based AI translation can be refined over time, real-time translation — such as voice-to-text systems in legal offices — presents even greater challenges.

Voice-to-Text Lacks Punctuation Awareness

People do not dictate punctuation, but pauses and commas can change legal meaning. For example:

  • “I’m guilty” vs. “I’m not guilty” (missing comma error).
  • Minor misspellings or poor grammar can dramatically change a translation.

AI Struggles with Speech Patterns

Legal system users come from diverse linguistic backgrounds, making real-time translation even more difficult. AI performs poorly when users:

  • Speak quickly or use filler words (“um,” “huh,” “oh”).
  • Have soft speech or heavy accents.
  • Use sentence structures influenced by indigenous or regional dialects.

These challenges make it clear that AI faces major challenges in performing accurately in high-stakes legal interactions.

The Need for National Standards and Training Datasets

Llop’s research underscores a critical gap: there are no national standards, training datasets, or quality benchmark datasets for legal translation AI.

A National Legal Translation Project

Llop saw an opportunity for improvement if there were to be:

  • A centralized effort to collect high-quality legal translation pairs.
  • State-specific localization of legal terms.
  • Guidelines for AI usage in courts, legal aid orgs, and other institutions.

Such a standardized dataset could train AI more effectively while ensuring legal accuracy.

Training for English-Only Speakers

English-speaking legal provider staff need training on how to structure their speech for better AI translation:

  • Using plain language and short sentences.
  • Avoiding vague pronouns (“his, her, their”).
  • Confirming meaning before finalizing translations.

AI, Human Oversight, and National Infrastructure in Legal Translation

Machine translation and AI can be useful, but they are far from perfect. Without human review, legal expertise, and national standards, AI-generated translations could compromise access to justice.

Llop’s work highlights the urgent need for:

  1. Human-in-the-loop AI translation.
  2. Better training data tailored for legal contexts.
  3. National standards for AI language access.

As AI continues to evolve, it must be designed with legal precision and human oversight — because in law, a mistranslation can change lives.

Get in touch with Cristina Llop to learn more about her work & vision for better language access: https://www.linkedin.com/in/cristina-llop-75749915/

Thanks to her for terrific, detailed presentation at the AI+A2J Research series. Sign up to come to future Zoom webinars in our series.Find out

more about the Stanford Legal Design Lab’s work on AI & Access to justice here.

Categories
Class Blog Project updates

Brainstorming new Language Access self help ideas

Brainstorming Potential Solutions in the Design for Justice Class: Language Access (Week 3)

By Sahil Chopra

Having experienced the court first hand, we returned to the classroom to revisit the tenets of Design Thinking and coalesce our thoughts, before engaging in a productive, rapid-brainstorming session.

Here’s a quick reminder of 5 “tenets” behind the design philosophies that drove our brainstorming:

  1. There is no “one perfect idea”. In fact, it is quite limiting to focus on “quality” ideas, i.e. those that seem practical or reasonable. In this initial phase of brainstorming, you should let your imagination roam free. You might be surprised by the ways you can turn an unreasonable idea into a truly impactful one.
  2. Don’t judge others. You can only be truly collaborative and helpful if you reserve judgement upon others’ ideas. Don’t analyze. Don’t constrain. Don’t judge.
  3. Be concise and specific. Yes, we all want to help provide language access to millions of Californians; but ideas won’t get us all the way there. In order to brainstorm effectively, you have to think “physical”, i.e. what could you make or build in an ideal world. Don’t think in abstractions but realities.
  4. Always respond to ideas with the phrase “yes and”. Saying “no” and “yes but” are conversation killers. Even if you don’t totally agree with an idea, embrace it and try to add your own spin to it, building upon it by saying “yes and”.
  5. Go for wild, ambitious, and impossible. Think big! We can always scale back later.

With these principles in mind, we drew upon our observations of the prior week to develop a list of current positives and negatives with language access at the court. We then brainstormed a list of potential successes and pitfalls, we might face while trying to improve language access.

Current Positives

  • Empathy: Sitting in on family court trials and observing the interactions between court staff and clients, it was apparent that those who work at the courthouse truly care. They are overwhelmed and understaffed, but they truly believe in the work and are trying their best to service the hundreds of clients that walk through the door each day.
  • Pathfinding: Signage was plentiful, though it could be improved by providing multilingual queues. The docket system, hosted on the large vertical flat screens, was especially useful in orienting oneself as they entered the courthouse.

Current Negatives

  • Form Accessibility: It’s often difficult to know what pieces of ancillary information are needed to fill out the form, which sections pertain to you personally, etc. There are workshops to help people fill out the forms, but they are understaffed; and the videos shown as part of the divorce workshop we observed weren’t entirely helpful, as they did not actually walk through the forms themselves.
  • Waiting: People line up in the self-help queue starting at 7:00 am, even though the service starts helping individuals at 9:00 am. The wait times are long.
  • Language: Many people who don’t speak English bring translators, but these must be 18+-year-olds; and involving someone else in the legal process implies that translator must also leave work, skip school, etc.

Future Positives (Ideas)

  • Real-Time Translation
  • Human-Oriented Experiences
  • Space Optimization (Court House)
  • Efficiency (Simplify Forms, Reduce Lines, etc.)

Future Negatives (Considerations)

  • Litigation
  • Budget Cuts / Restrictions
  • Buy-In: Unions, Staff, Judges, Clients

With these themes established, we brainstormed the following 10 ideas:

  1. Interactive Forms
    1. Concept: Make forms interactive on a website such that they become “choose-your-own-adventure.” Use simple questions written in the person’s native language to determine which portions of the form are necessary for the individual to actually fill out.
    2. Goal: This should make form-filling a more accessible and personalized experience. Hopefully, this makes the process for filing easier and less intimidating.
  2. Multilingual FAQs
    1. Concept: Update the court’s website with FAQs in various languages. Prospective users could read these FAQs for their specific problem before coming to the court itself, so that they have a better understanding of the court process for their issue before coming in. Similarly, these could be provided to those in the self-help line to read before they are served.
    2. Goal: This will improve understanding of the court processes in order to empower individuals with a sense of control.
  3. Multilingual Court Navigation Instructions
    1. Concept: Create an app or website, with top 5 languages spoken by LEP court users, that explains court layout, functions and services at each office, and language support services.
    2. Goal: The user can find answers to common questions on their phone and use it to navigate the courthouse and its services. This will save headaches about what they need to do to get from Point A to point B, both in terms of navigating the courthouse and its services and help customers more easily address their legal needs.
  4. Online Multilingual Workshop Videos
    1. Concept: Provide client with multilingual YouTube videos explaining the mechanics of different common problems (e.g. divorce) that people go to the court to address.
    2. Goal: Right now the videos are only in English and only viewable in the workshop. This poses double issues for accessibility of content. Multilingual YouTube videos may reduce the burden on the workshop staff and provide a better, informative experience to non-native speakers.
  5. Chunk Workshop Video Into Sections
    1. Concept: Split workshop videos into chunks rather than the current 45-minute video. Also, integrate the form-filling within the video watching experience. Rather than a presentation, the workshop videos should directly help the users fill out the necessary and related components of their paperwork.
    2. Goal: Currently, the videos are an information overload. Many definitions are not listed on the slides. Viewers cannot rewind the video in the workshop. And the video does not directly correspond to sections of the forms that the users have to fill out. Eliminate all these problems by providing information in nugget-sized-proportions and tightly coupling this video experience with the forms.
  6. NLT for Court Forms
    1. Concept: Integrate the web forms with Google Translate, or some other legal translation software.
    2. Goal: All forms must be submitted in English according to California State Law. Even if the forms are presented in Spanish, the user must respond in English — which poses a huge barrier without an interpreter. Instead, bring Natural Language Translation (NLT) systems to the user, so this form-filling process becomes much easier.
  7. Symbolic Signage at Court
    1. Concept: Replace English signs in help center with symbol-rich signs that are easier to understand and follow.
    2. Goal: Symbol-rich signs will be able to better direct court users to get the forms they need and access the services they require. This will improve the physical experience of navigating the courthouse.
  8. Brochure Placement
    1. Concept: Redesign help center brochures to be color coded according to languages and then placed in different sections of the room, according to language.
    2. Goal: By offering forms in both languages, court users can identify the right forms and will be able to understand them. They can then write their answers on the corresponding English language forms.
  9. Robotic Assistants
    1. Concept: Create mobile booths in different areas where people could lodge cases in their languages by speaking into a phone line which will then capture the information and translate it into English. The robotic booth will then print the documents which the user can scan and download through the mobile application.
    2. Goal: Reduce trauma and negative attitudes towards the court system by promoting privacy of individuals coming to court.
  10. Real Time Translation Services
    1. Concept: Have tablets and headphones available for rent upon court entrance that guide you in your respective language to where you need to go (with pictures) and act as real-time translators with court actors.
    2. Goal: Facilitate the processes of moving through the court and interacting with court personnel despite language barriers.

With these ideas in mind, we are going to spend next week whittling down these to five favorites, drawing out the ideas, and then interviewing individuals at the courthouse as to what they like and/or dislike about these potential solutions to language access problems.

Categories
Class Blog Project updates

Identifying A Single Prototype for language access improvement

By Sahil Chopra

(Part of a series of posts documenting the Design for Justice: Language Access class)

Entering home stretch of the Autumn quarter, we spent today’s class first synthesizing our findings and working on our final pitch to the California Judicial Council and then selecting one of our prototypes for further development.

To start the the synthesis process, we grabbed a whiteboard and divided it into two halves — with one side dedicated to answering “What we heard or saw?” and the other dedicated to answering “What do we do in response?” Starting with the former question, we started jotting down quotes and experiences we had catalogued over the past few weeks from our interviews and observations, before clustering them around common topics. This exercise yielded two incredibly salient themes that we hope to address with our revised prototype:

 

  • Time: People fear the courthouse, because it takes an inordinate amount of time and as a result deprives of economic and educational opportunity that they would be accumulating, had they not spent hours upon hours and days upon days within the courthouse. One woman we interviewed exclaimed that, “divorce right now is almost a full-time job”; while another lamented that the amount of time she had to spend in court affected her kids’ academics, as they had to accompany her so that she could have the proper assistance necessary to fill out the English-language forms.
  • The process of getting proper help seems to take too much time because the self-help desk is understaffed and because court users produce a large number of errors while filling out their paperwork. Many of the people have interviewed over the past several weeks have mentioned that they often spend several hours waiting to be helped, only to be told that they made a mistake in their documents and are then sent to back the of line to seek guidance.
  • As a result, we witness a vicious cycle. The self-help desk is constantly creating its own backlog of requests, ultimately increasing stress and time allotted per case — for both the clerks and the court users. As a result of this feedback, one of our primary goals is to reduce the amount of time that a user has to spend in order to fill and submit their proper paperwork. This will help users have a more pleasurable and accessible court experience, while reducing the stress upon the self-help clerks.

  • Language Barriers Are Multifaceted: One thing we did not realize until we began user testing was how multifaceted of a problem language barriers actually are. When presenting our “Redesigned-Form” prototype to non-native speakers last week, we established a situation where we asked our interviewees to file for divorce. On the second page of the prototype, we asked our court users to declare whether they wanted a “Divorce”, “Legal Separation”, or “Nullity” from a “Marriage” or “Domestic Partnership”. While it was clear to the user that they were on a page associated with divorce, they were unsure as to what the differences were between a “divorce”, “legal separation”, and a “nullity”. As a native English speaker these terms seem foreign, as they are rooted in precise legal terminology; so one apparent aspect of “language access” is to provide court users with simple language that unpacks these precise terms. But the problem with language access extends far behind legal terminology and words in different languages. There are often significant cultural barriers as well. When interviewing a technologically-savvy Uyghur woman, we saw her even tried opening Google Translate on her phone, writing the phrase, and having the service produce a Mandarin version of the text. The problem was, however, that the concept did not exist in her culture; so even though she had the translated phrase, the concept did not register. This highlights the fact that language access does not simply include English-barriers, but also cultural ones. We must overcome both in order to provide true access to court systems.

 

With this in mind, we shifted to the other half of the board, answering “What do we do in response?” Here are a few of the ideas from that brainstorm:

  1. Split the current forms into manageable chunks so that we do not overwhelm court users and narrow context of any page down to a singular topic so that it become easier for a non-native speaker to identify the goal of the page, even if they struggle to understand the bulk of it.
  2. Provide native-language instructions and definitions that unpack legal ease in laymen’s terms and pay attention to cultural differences, in their explanations of legal terms.
  3. Add legal advice forums like r/legal-advice into the court website; and provide a platform for non-native speakers to voice their experiences to others within their communities. We heard from many younger court users, that they looked online to blogs in order to understand the experience they were about to undertake, as a user of the court. These blogs reassured them and provided guidance, when they were most confused. It would be cool to provide this type of support on the court website and extend it to non-native speakers.

Moving forward, we are going to further pursue our “Redesigned-Form” prototype, diving deeper into the Divorce Experience to provide a more nuanced prototype experience.

Categories
Class Blog Project updates

Design for Justice: Language Access — an introduction in week 1

by Sahil Chopra

Language is the medium by which we interact with culture, express our ideas, and maintain our rights. Without “language access”, i.e. the ability to convey one’s thoughts effectively and understand others correctly, one is disempowered altogether. At a societal level this can lead to systemic inequality, whether intentional or not; and one of the places where this is most evident is the court system.

This Autumn, I’m one of the 25 students enrolled in Stanford’s Design for Language Access, a course initiated by the Stanford Legal Design Lab to investigate and advise how state courts may better serve Californians entering the legal system, who either do not speak or have limited proficiency with English.

As the Judicial Council of California’s Strategic Plan for Language Access in California Court details, 40% of Californians speak non-English languages at home, 200+ languages and dialects are spoken by Californians as a whole, and approximately ~20% of Californians have English language limitations. Going to court is always a stressful experience, as the impetus to seek court help is often a difficult circumstance itself. Coupling the weight of the incident with the inability to communicate and properly resolve your issue only magnifies the stress incurred by the individual. Moreover, it may be difficult to properly resolve one’s legal issue and receive the proper access to one’s legal rights if they are unable to effectively communicate with lawyers, clerks, and judges within the judicial branch. Thus, “language access”, as the Judicial Council of California titles it, is a critical issue that we must address in order to ensure and fair and equitable legal proceedings.

Personally, I have no prior background with judicial systems. I’m a computer scientist by training, completing my BS/MS with concentrations in Artificial Intelligence and Human Computer Interaction – focusing a bulk of my research in cognitive science and natural language processing. But that’s where the diverse experience of my classmates come in. We are lawyers, teachers, designers, business students, and computer scientists — all hoping to better understand this space and offer a different perspective.

Over the next nine weeks, we shall apply the fundamental principles of “Design Thinking” to first observe and interview individuals going through the court system and then hypothesize, prototype, and test potential strategies that may provide better language access to millions of Californians. Our class will culminate in a list of possible solutions and implementations which the California courts may consider as potential avenues by which the state can improve language access at scale. Additionally, we shall be evaluating a pilot program that California courts is running in San Jose, where tablets with Google Translate are being employed to help ease communication between non-English-speaking clients and English-speaking court staff.

Stay tuned to learn more week-by-week about our journey to help provide better language access to Californians!

Categories
Language Access Reading

Access to Justice for People Who Do Not Speak English

This article by Chief Justice Randall T. Shepard of Indiana describes what justice issues arise out of language access problems in state courts.