By Jonah Wu

Student research fellow at Legal Design Lab, 2018-2019

1. Can AI help improve access to civil courts?

Civil court leaders have a newly strong interest in how artificial intelligence can improve the quality and efficiency of legal services in the justice system, especially for problems that self-represented litigants face [1, 2, 3, 4, 5]. The promise is that artificial intelligence can address the fundamental crises in courts: that ordinary people are not able to use the system clearly or efficiently; that courts struggle to manage vast amounts of information; and that litigants and judicial officials often have to make complex decisions with little support.

If AI is able to gather and sift through vast troves of information, identify patterns, predict optimal strategies, detect anomalies, classify issues, and draft documents, the promise is that these capabilities could be harnessed for making the civil court system more accessible to people.

The question then, is how real these promises are, and how they are being implemented and evaluated. Now that early experimentation and agenda-setting have begun, the study of AI as a means for enhancing the quality of justice in the civil court system deserves greater definition. This paper surveys current applications of AI in the civil court context. It aims to lay a foundation for further case studies, observational studies, and shared documentation of AI for access to justice development research. It catalogs current projects, reflects on the constraints and infrastructure issues, and proposes an agenda for future development and research.

2. Background to the Rise of AI in the Legal System

When I use the term Artificial Intelligence, I distinguish it from general software applications that are used to input, track, and manage court information. Our basic criteria for AI-oriented projects is that the technology has capacity to perceive knowledge, make sense of data, generate predictions or decisions, translate information, or otherwise simulate intelligent behavior. AI does not include all court technology innovations. For example, I am not considering websites that broadcast information to the public; case or customer management systems that store information; or kiosks, apps, or mobile messages that communicate case information to litigants.

The discussion of AI in criminal courts is currently more robust than in civil courts. It has been proposed as a means to monitor and recognize defendants; support sentencing and bail decisions; and better assess evidence [3]. Because of the rapid rise of risk assessment AI in the setting of bail or sentencing, there has been more description and debate on AI [6]. There has been less focus on AI’s potential, or its concerns, in the civil justice system, including for family, housing, debt, employment, and consumer litigation. That said, there has been a robust discourse over the past 15 years of what technology applications and websites could be used by courts and legal aid groups to improve access to justice [7].

The current interest in AI for civil court improvements is in sync with a new abundance of data. As more courts have gathered data about administration, pleadings, litigant behavior, and decisions [1], it presents powerful opportunities for research and analytics in the courts, that can lead to greater efficiency and better design of services. Some groups have managed to use data to bring enormous new volumes of cases into the court system — like debt collection agencies, which have automated filings of cases against people for debt [8], often resulting in complaints that have missing or incorrect information and minimal, ineffective notice to defendants. If litigants like these can harness AI strategies to flood the court with cases, could the courts use its own AI strategies to manage and evaluate these cases and others — especially to better protect unwitting defendants against low-quality lawsuits?

The rise in interest in AI coincides with state courts experiencing economic pressure: budgets are cut, hours are reduced, and even some locations are closed [9]. Despite financial constraints, courts are expected to provide modern, digital, responsive services like in other consumer services. This presents a challenging expectation for the courts. How can they provide judicial services in sync with rapidly modernizing other service sectors — in finance, medicine, and other government bodies — within significant cost constraints? The promise of AI is that it can scale up quality services and improving efficiency, to improve performances and save costs [10].

A final background factor to consider is the growing concern over public perceptions of the judicial system. Yearly surveys indicate that communities find courts out of touch with the public, and with calls for greater empathy and engagement with “everyday people” [11]. Given that the mission of the court is to provide an avenue to lawful justice to constituents, if AI can help the court better achieve that mission without adding on averse risks, it would help the courts establish greater procedural and distributive justice for its litigants, and hopefully then bolster its legitimacy to the public and engagement with it.

3. What could be? Proposals in the Literature for AI for access to justice

What has the literature proposed on how AI techniques can address the access to justice crisis in civil courts? Over the past several decades, distinct use cases have been proposed for development. There is a mix of litigant-focused use cases (to help them understand the system and make stronger claims), and court-focused use cases (to help it improve its efficiency, consistency, transparency, and quality of services).

• Answer a litigant’s questions about how the law applies to them. Computational law experts have proposed automated legal reasoning as a way to understand if a given case is in accordance with the law or not [12]. Court leaders also envision AI to help litigants conduct effective, direct research into how the law would apply to them [4,5]. Questions of how the law would apply to a given case lay on a spectrum of complexity. Questions that are more straightforwardly algorithmic (e.g., if a person exceeded a speed limit, or if a quantity or date is in an acceptable range) can be automated with little technical challenge [13]. Questions that have more qualitative standards, like whether it was reasonable, unconscionable foreseeable, or done in good faith, are not as easily automated — but they might be with greater work in deep learning and neural networks. Many propose that expert systems, or AI-powered chatbots might help litigants know their rights and make claims [14].
• Analyze the quality of a legal claim and evidence. Several proposals are around making it easier to understand what has been submitted to court, and how a case has proceeded. Some exploratory work has pointed towards how AI could automatically classify a case docket, the chronological events in a case, in order that it could be understood computationally [15]. Machine learning could find patterns in claims and other legal filings, to indicate whether something has been argued well, whether the law supports it, and evaluate it versus competing claims [16].
• Provide coordinated guidance for a person without a lawyer. Many have proposed focus on developing a holistic AI-based system to guide people without lawyers through the choices and procedure of a civil court case. One vision is of an advisory system that would help a person understand available forms of relief, helping them understand if they can meet the requirements, informing them of procedural requirements; and helping them to draft court documents [17, 18].
• Predict and automate decisionmaking. Another proposal, discussed within the topic of online dispute resolution, is around how AI could either predict how a case will be decided (and thus give litigants a stronger understanding of their changes), or to actually generate a proposal of how a disputes should be settled [19, 20]. In this way, prediction of judicial decisions could be useful to access to justice. It could be integrated into online court platforms where people are exploring their legal options, or where they are entering and exchanging information in their case. The AI would help litigants to make better choices regarding how they file, and it would help courts expedite decision-making by either supporting or replacing human judges’ rulings.

4. What is happening so far? AI in action for access

With many proposals circulating about how AI might be applied for access to justice, where can we see these possibilities being developed and piloted with courts? Our initial survey identifies a handful of applications in action.

4.1. Predicting settlement arrangements, judicial decisions, and other outcomes of claims

One of the most robust areas of AI in access to justice work has been in developing applications to predict how a claim, case, or settlement will be resolved by a court. This area of predictive analytics has been demonstrated in many research projects, and in some cases have been integrated into court workflows.

In Australian Family Law courts, a team of artificial intelligence experts and lawyers have begun to develop Split-Up system, to use rules-based reasoning in concert with neural networks to predict outcomes for property disputes in divorce and other family law cases [21]. The Split Up system is used by judges to support their decision-making, by helping them to identify the assets of marriage that should be included in a settlement, and then establishing what percentage of the common pool each party should receive — which is a discretionary judicial choice based on factors including contributions, amount of resources, and future needs. The system incorporates 94 relevant factors to make its analysis, which uses neural network statistical techniques. The judge can then propose a final property order based on the system’s analysis. The system also seeks to make transparent explanations of its decision, so it uses Toulmin Argument structures to represent how it reached its predictions.

Researchers have created algorithms to predict Supreme Court and European Court of Human Rights decisions [22, 23, 24]. They use natural language processing and machine learning to construct models that predict the courts’ decision with strong accuracy. Their predictions draw from the formal facts submitted in the case to identify what a likely outcome, and potentially even individual justices’ votes will be. This judicial decision prediction research can possibly used to offer predictive analytic tools to litigants, so they can better assess the strength of their claim and understand what outcomes they might face. Legal technology companies like Ravel and LexMachina [25, 26], claim that they can predict judges’ decision and case behavior, or the outcomes of an opposing party. The applications are mainly aimed at corporate-level litigation, rather than access to justice.

4.2. Detecting abuse and fraud against people the court oversees

Courts’ role in overseeing guardians and conservators means that they should be reducing financial exploitation of vulnerable people by those appointed to protect them. With particular concern for financial abuse of elderly by their conservators or guardians, a team in Utah began building an AI tool to identify likely fraud in the reported financial transactions that conservators or guardians submit to the court. The system, developed in concert with a Minnesota court system in a hackathon, would detect anomalies and fraud-related patterns, and send flag notifications to courts to investigate further [28].

4.3. Preventative Diagnosis of legal issues, matching to services, and automating relief

A robust branch of applications has been around using AI techniques to spot people’s legal needs (that they potentially did not know they had), and then either match them to a service provider or to automate a service for them, to help resolve their need. This approach has begun with the expungement use case — in which states have policies to help people clear their criminal record, but without widespread uptake. With this problem in mind, groups have developed AI programs to automatically flag who has a criminal record to clear, and then to streamline the expungement. help automate the expungement process for their region. In Maryland, Matthew Stubenberg from Maryland Volunteer Lawyers Service (now in Harvard’s A2J Lab) built a suite of tools to spot their organization’s clients’ problems, including overdue bills and criminal records that could be expunged. This tool helped legal aid attorneys diagnose their clients’ problems. Stubenberg also made the criminal record application public-facing, as MDExpungement, for anyone to automatically find if they have a criminal record and to submit a request to clear it [29].

Code for America is working inside courts to develop another AI application for expungement. They are work with the internal databases of California courts to automatically identify expunge eligible records, eliminating the need for individuals to apply for [30].

The authors, in partnership with researchers at Suffolk LIT Lab, are working on an AI application to automatically detect legal issues in people’s descriptions of their life problems, that they share in online forums, social media, and search queries [31]. This project involves labeling datasets of people’s problem stories, taken from Reddit and online virtual legal clinics, to then train a classifier to be able to automatically recognize what specific legal issue a person might have based on their story. This classifier could be used to power referral bots (that send people messages with local resources and agencies that could help them), or to translate people’s problem stories into actionable legal triage and advisory systems, as had been envisioned in the literature.

4.4. Analyzing quality of claims and citations

Considering how to help courts be more efficient in their analysis of claims and evidence, there are some applications — like the product Clerk from the company Judicata — that can read, analyze, and score submissions that people and lawyers make to the court [32]. These applications can assess the quality of a legal brief, to give clerks, judges, or litigants the ability to identify the source of the arguments, cross check them against the original, and possibly also find other related cases. In addition to improving the efficiency of analysis, the tool could be used for better drafting of submissions to the court — with litigants checking the quality of their pleadings before submitting them.

4.5. Active, intelligent case management

The Hebei High Court in China has reported the development of a smart court management AI, termed Intelligent Trial 1.0 system [33]. It automatically scans in and digitizes filings; it classifies documents into electronic files; it matches the parties to existing case parties; it identifies relevant laws, cases, and legal documents to be considered; it automatically generates all necessary court procedural documents like notices and seals; and it distributes cases to judges for them to be put on the right track. The system coordinates various AI tasks together into a workstream that can reduce court staff and judges’ workloads.

4.6. Online dispute resolution platforms and automated decision-making

Online dispute resolution platforms have grown around the United States, some of them using AI techniques to sort claims and propose settlements. Many ODR platforms do not use AI, but rather act as a collaboration and streamlining platform for litigants’ tasks. ODR platforms like Rechtwijzer, MyLaw BC, and the British Columbia Civil Resolution Tribunal, use some AI techniques to sort which people can use the platform to tackle a problem, and to automate decision-making and settlement or outcome proposal [34].

We also see new pilots of online dispute platforms in Australia, in the state of Victoria with its VCAT pilot for small claims (that is now in hiatus, awaiting future funding) — and in Utah, for its small claims in one place outside Salt Lake City.

These pilots are using platforms like Modria (part of Tyler Technology), Modron, or Matterhorn from Court Innovations. How much AI is part of these systems is not clear — it seems they are mainly platform for logging details and preferences, communicating between parties, and drafting/signing settlements (without any algorithm or AI tool making a decision proposal or crafting a strategy for parties). If the pilots are successful and become ongoing projects, then we can expect future iterations to possibly involve more AI-powered recommendations or decision tools.

5. Agenda for Development and Infrastructure of AI in access to justice

If an ecosystem of access to justice AI is to be accelerated, what is the agenda to guide the growth of projects? There is work to be done on the infrastructure of sharing data, defining ethics standards, security standards, and privacy policies. In addition, there is organizational and coalition-building work, to allow for more open innovation and cross-organization initiatives to grow.

5.1.Opening and standardizing datasets

Currently, the field of AI for access to justice is harmed by the lack of open, labeled datasets. Courts do hold relatively small datasets, but there are not standard protocols to make them available to the public or to researchers, nor are there labeled datasets to be used in training AI tools [35]. There are a few examples of labelled court datasets, like from the Board of Veterans Appeals [36]. A newly-announced US initiative, the National Court Open Data Standards Project, will promote standardization of existing court data, so that there can be more seamless sharing and cross-jurisdiction projects [37].

5.2.Making Policies to Manage Risks

There should be multi-stakeholder design of the infrastructure, to define an evolving set of guidance for issues around the following large risks that court administrators have identified as worries around new AI in courts [4, 5].

• Bias of possible Training Data Sets. Can we better spot, rectify, and condition inherent biases that the data sets might have, that we are using to train the new AI?
• Lack of transparency of AI Tools. Can we create standard ways to communicate how an AI tool works, to ensure there is transparency to litigants, defendants, court staff, and others, so that there can be robust review of it?
• Privacy of court users. Can we have standard redaction and privacy policies that prevent individuals’ sensitive information from being exposed [38]? There are several redaction software applications that use natural language processing to scan documents and automatically redact sensitive terms [39, 40].
• New concerns for fairness. Will courts and the legal profession have to change how they define what ‘information versus advice’ is, as currently guide regulations about what types of technological help can be given to litigants? Also, if AI exposes patterns of arbitrary or biased decision-making in the courts, how will the courts respond to change personnel, organizational structures, or court procedures to better provide fairness?

For many of these policy questions, there are government-focused ethics initiatives that the justice system can learn from, as they define best practices and guiding principles for how to integrate AI responsibly into public, powerful institutions [42, 43, 44].

6. Conclusion

This paper’s survey of proposals and applications for AI’s use for access to justice demonstrates how technology might be operationalized for social impact.

If there is more infrastructure-oriented work now, that establishes how courts can share data responsibly, and set new standards for privacy, transparency, fairness, and due process in regards to AI applications, this nascent set of projects may blossom into many more pilots over the next several years.

In a decade, there may be a full ecosystem of AI-powered courts, in which a person who faces a problem with eviction, credit card debt, child custody, or employment discrimination could have clear, affordable, efficient ways to use use the public civil justice system to resolve their problem. Especially with AI offering more preventative, holistic support to litigants, it might have anti-poverty effects as well, ensuring that the legal system resolves people’s potential life crises, rather than exacerbating them.