Tag: access to justice

Categories
AI + Access to Justice Current Projects

User Research Workshop on AI & A2J

In December 2023, our lab hosted a half-day workshop on AI for Legal Help.

Our policy lab class of law students, master students, and undergraduates presented their user research findings from their September through December research.

Our guests, including those from technology companies, universities, state bars, legal aid groups, community-based organizations, and advocacy/think takes, all worked together in break-out sessions to tackle some of the big policy and legal opportunities around AI in the space.

We thank our main class partners, the Technology Initiative Grant team from the Legal Services Corporation, for assisting us with the direction and main feedback to our class user research work.

Categories
AI + Access to Justice Current Projects

Schedule for AI & A2J Jurix workshop

Our organizing committee was pleased to receive many excellent submissions for the AI & A2J Workshop at Jurix on December 18, 2023. We were able to select half of the submissions for acceptance, and we extended the half-day workshop to be a full-day workshop to accommodate the number of submissions.

We are pleased to announce our final schedule for the workshop:

Schedule for the AI & A2J Workshop

Morning Sessions

Welcome Kickoff, 09:00-09:15

Conference organizers welcome everyone, lead introductions, and review the day’s plan.

1: AI-A2J in Practice, 09:15-10:30 AM 

09:15-09:30: Juan David Gutierrez: AI technologies in the judiciary: Critical appraisal of LLMs in judicial decision making

09:30-09:45: Ransom Wydner, Sateesh Nori, Eliza Hong, Sam Flynn, and Ali Cook: AI in Access to Justice: Coalition-Building as Key to Practical and Sustainable Applications

09:45-10:00: Mariana Raquel Mendoza Benza: Insufficient transparency in the use of AI in the judiciary of Peru and Colombia: A challenge to digital transformation

10:00-10:15: Vanja Skoric, Giovanni Sileno, and Sennay Ghebreab: Leveraging public procurement for LLMs in the public sector: Enhancing access to justice responsibly

10:15-10:30: Soumya Kandukuri: Building the AI Flywheel in the American Judiciary

Break: 10:30-11:00 

2: AI for A2J Advice, Issue-Spotting, and Engagement Tasks, 11:00-12:30 

11:00: Opening remarks to the session

11:05-11:20: Sam Harden: Rating the Responses to Legal Questions by Generative AI Models

11:20-11:35: Margaret Hagan: Good AI Legal Help, Bad AI Legal Help: Establishing quality standards for responses to people’s legal problem stories

11:35-11:50: Nick Goodson and Rongfei Lui: Intention and Context Elicitation with Large Language Models in the Legal Aid Intake Process

11:50-12:05: Nina Toivonen, Marika Salo-Lahti, Mikko Ranta, and Helena Haapio, Beyond Debt: The Intersection of Justice, Financial Wellbeing and AI

12:05-12:15: Amit Haim: Large Language Models and Legal Advice12:15-12:30: General Discussions, Takeaways, and Next Steps on AI for Advice

Break: 12:30-13:30

Afternoon Sessions

3: AI for Forms, Contracts &  Dispute Resolution, 13:30-15:00 

13:30: Opening remarks to this session13:35-13:50: Quinten Steenhuis, David Colarusso, and Bryce Wiley: Weaving Pathways for Justice with GPT: LLM-driven automated drafting of interactive legal applications

13:50-14:05: Katie Atkinson, David Bareham, Trevor Bench-Capon, Jon Collenette, and Jack Mumford: Tackling the Backlog: Support for Completing and Validating Forms

14:05-14:20: Anne Ketola, Helena Haapio, and Robert de Rooy: Chattable Contracts: AI Driven Access to Justice

14:20-14:30: Nishat Hyder-Rahman and Marco Giacalone: The role of generative AI in increasing access to justice in family (patrimonial) law

14:30-15:00: General Discussions, Takeaways, and Next Steps on AI for Forms & Dispute Resolution

Break: 15:00-15:30

4:  AI-A2J Technical Developments, 15:30-16:30

15:30: welcome to session
15:35-15:50: Marco Billi, Alessandro Parenti, Giuseppe Pisano, and Marco Sanchi: A hybrid approach of accessible legal reasoning through large language models
15:50-16:05: Bartosz Krupa – Polish BERT legal language model
16:05-16:20: Jakub Dråpal – Understanding Criminal Courts
16:20-16:30: General Discussion on Technical Developments in AI & A2J

Closing Discussion: 16:30-17:00

What are the connections between the sessions? What next steps do participants think will be useful? What new research questions and efforts might emerge from today?

Categories
AI + Access to Justice Current Projects

Call for papers to the JURIX workshop on AI & Access to Justice

At the December 2023 JURIX conference on Legal Knowledge and Information Systems, there is an academic workshop on AI and Access to Justice.

There is an open call for submissions to the workshop. There is an extension to the deadline, which is now November 20, 2023. We encourage academics, practitioners, and others interested in the field to submit a paper for the workshop or consider attending.

The workshop will be on December 18, 2023 in Maastricht, Netherlands (with possible hybrid participation available).

See more about the conference at the main JURIX 23 website.

About the AI & A2J workshop

This workshop will bring together lawyers, computer scientists, and social science researchers to discuss their findings and proposals around how AI might be used to improve access to justice, as well as how to hold AI models accountable for the public good.

Why this workshop? As more of the public learns about AI, there is the potential that more people will use AI tools to understand their legal problems, seek assistance, and navigate the justice system. There is also more interest (and suspicion) by justice professionals about how large language models might affect services, efficiency, and outreach around legal help. The workshop will be an opportunity for an interdisciplinary group of researchers to shape a research agenda, establish partnerships, and share early findings about what opportunities and risks exist in the AI/Access to Justice domain — and how new efforts and research might contribute to improving the justice system through technology.

What is Access to Justice? Access to justice (A2J) goals center around making the civil justice system more equitable, accessible, empowering, and responsive for people who are struggling with issues around housing, family, workplace, money, and personal security. Specific A2J goals may include increasing people’s legal capability and understanding; their ability to navigate formal and informal justice processes; their ability to do legal tasks around paperwork, prediction, decision-making, and argumentation; and justice professionals’ ability to understand and reform the system to be more equitable, accessible, and responsive. How might AI contribute to these goals? And what are the risks when AI is more involved in the civil justice system?

At the JURIX AI & Access to Justice Workshop, we will explore new ideas, research efforts, frameworks, and proposals on these topics. By the end of the workshop, participants will be able to:

  • Identify the key challenges and opportunities for using AI to improve access to justice.
  • Identify the key challenges and opportunities of building new data sets, benchmarks, and research infrastructure for AI for access to justice.
  • Discuss the ethical and legal implications of using AI in the legal system, particularly for tasks related to people who cannot afford full legal representation.
  • Develop proposals for how to hold AI models accountable for the public good.

Format of the Workshop: The workshop will be conducted in a hybrid form and will consist of a mix of presentations, panel discussions, and breakout sessions. It will be a half-day session. Participants will have the opportunity to share their own work and learn from the expertise of others.

Organizers of the Workshop: Margaret Hagan (Stanford Legal Design Lab), Nora al-Haider (Stanford Legal Design Lab), Hannes Westermann (University of Montreal), Jaromir Savelka (Carnegie Mellon University), Quinten Steenhuis (Suffolk LIT Lab).

Are you generally interested in AI & Access to Justice? Sign up for our Stanford Legal Design Lab AI-A2J interest list to stay in touch.

Submit a paper to the AI & A2J Workshop

We welcome submissions of 4-12 pages (using the IOS formatting guidelines). A selection will be made on the basis of workshop-level reviewing focusing on overall quality, relevance, and diversity.

Workshop submissions may be about the topics described above, including:

  • findings of research about how AI is affecting access to justice,
  • evaluation of AI models and tools intended to benefit access to justice,
  • outcomes of new interventions intended to deploy AI for access to justice,
  • proposals of future work to use AI or hold AI initiatives accountable,
  • principles & frameworks to guide work in this area, or
  • other topics related to AI & access to justice

Deadline extended to November 20, 2023

Submission Link: Submit your 4-12 page paper here: https://easychair.org/my/conference?conf=jurixaia2j

Notification: November 28, 2023

Workshop: December 18, 2023 (with the possibility of hybrid participation) in Maastricht, Netherlands

More about the JURIX Conference

The Foundation for Legal Knowledge Based Systems (JURIX) is an organization of researchers in the field of Law and Computer Science in the Netherlands and Flanders. Since 1988, JURIX has held annual international conferences on Legal Knowledge and Information Systems.

This year, JURIX conference on Legal Knowledge and Information Systems will be hosted in Maastricht, the Netherlands. It will take place on December 18-20, 2023.

The proceedings of the conferences will be published in the Frontiers of Artificial Intelligence and Applications series of IOS Press. JURIX follows the Golden Standard and provides one of the best dissemination platforms in AI & law.


Categories
Current Projects

Paths Toward Access to Justice at Scale presentation

In October 2023, Margaret Hagan presented at the International Access to Justice Forum, on “Paths toward Access to Justice at Scale”. The presentation covered the preliminary results of stakeholder interviews she is conducting with justice professionals across the US about how best to scale one-off innovations and new ideas for improvements, to become more sustainable and impactful system changes.

The abstract

Pilots to increase access to justice are happening in local courts, legal aid groups, government agencies, and community groups around the globe. These innovative new local services, technologies, and policies aim to build people’s capability, reduce barriers to access, and improve the quality of justice people receive. They are often built with an initial short-term investment, to design the pilot and run it for a period. Most of them lack a clear plan to scale up to a more robust iteration, or spread to other jurisdictions, or sustain the program past the initial investment. This presentation presents a framework of theories of change for the justice system, and stakeholders’ feedback on how to use them for impact.

The research on Access to Justice long-term strategies

The presentation covered the results of the qualitative, in-depth interviews with 11 legal aid lawyers, court staff members, legal technologists, funders, and statewide justice advocates about their work, impact, and long-term change.

The research interviews asked these professionals about their long-term, systematic theories of change — and to rate other theories of change that others have mentioned. They were asked about past projects they’ve run, how they have made an impact (or not), and what they have learned from their colleagues about what makes a particular initiative more impactful, sustainable, and successful.

The goal of the research interviews was to gather the informal knowledge that various professionals have gathered over years of work in reforming the justice system and improving people’s outcomes when they experience legal problems.

This knowledge often circulates casually at meetings, dinners, and over email, but is not often laid out explicitly or systematically. It was also to encourage reflection among practitioners, to move from a focus just on day-to-day work to long-term impact.

Stay tuned for more publications about this research, as the interviews & synthesis continue.

Categories
Background

Simple at the front, Smart at the back: design for access to justice innovation

A colleague working on improving the legal system in New Zealand from a user-centered design perspective mentioned this phrase to me in a recent email: Simple at the Front, Smart at the Back. Now it’s my constant refrain.

What does it mean? That when we build tools, guides, explainers, or anything else for laypeople to deal with the legal system, they should be simple, intuitive & clear for the person. But this interface should be covering up a very intelligent and robust complex system. We aren’t actually ‘simplfiying’ so much as improving the user experience of the system through tools that make people feel that navigating the system is simple. Using the power of coordinated, interactive, smart technology (and planning) we can make complex systems seem simple.


Next week I am convening a working group on how we can make the internet a better place for legal help.
This is one of the three main themes of work this year at my Legal Design Lab at Stanford. The main question is how we can make it incredibly easy for a lay person who is using Google or the starting point to find legal info, to get them from their Google search to essential information about their situation, tools and information to help them make smart decisions, and then actually follow through on them.

My goal is to promote more coordination among the many organizations that currently (and could possibly) provide these online services right now.

These are not a standard group of people. Some of them are non-legal — search engine providers like Google, Microsoft, Yahoo, or referral services like 211 & United Way. Others are legal help site maintainers, or courts, or the attorney general office, or consumer law companies, or startups, or legal aid groups, or non-profits, or legal publishers. So many different entities, all potentially with value to offer a person on their journey from “Do I Have a Legal Problem?” to “I Got My Problem Resolved.”

The future I see is one of coordination of all these various service-providers so that a layperson in search of help can journey between their services seamlessly — with all their personal information in tow, knowing how to get from one provider to the next, not getting confused or lost after one service’s offering ends, and having the different providers help her along to resolution.

To get to this seamless user journey & enjoyable online user experience, it might be tempting to call for a Mega Portal. This would be the one amazing, well-designed, user-friendly, comprehensive website that can get a person from problem-query to answers, appointments, and tools.

This is a pie-in-the-sky dream, and also not that close to what different people actually want. There will not be the one almighty legal site that everyone in the country will go to. The closest we will get on that front is Google. There is simply no strong enough legal brand or comprehensive-enough resource that can cover all the many touchpoints a person must go through in a typical legal process.

Rather than bank on this one almighty Mega Portal, I see a future where we can build a whole lot of different ways to access this coordinated legal system. It would be full of different apps, websites, in-person stations, court and self-help kiosks, text messaging systems — and whatever new technology is coming our way in the next decades. We don’t need one perfect portal, we need a rich ecosystem of new & cutting-edge tools that can get different kinds of people to the same legal help.

Better internet for legal help coordinated system w many doors

And to achieve this vision, what’s the key thing — the first step? It sure doesn’t sound sexy, but I am increasingly obsessed with it: Data Standards. APIs. Getting all these different kinds of Internet-based legal service providers to make sure that their systems can talk to each other, that they accessibly present their databases of information about what legal help exists, who can access what, what rules apply to whom, what documents to submit, where to submit them, and on & on….

We can’t afford to live in the current siloed & proprietary world, where vendors and agencies hoard data & don’t talk to other service-providers. Where groups build a legal services tool and then don’t let this tool talk to other tools or pass a user over to next steps on her journey. We need to set requirements that providers must coordinate with each other, for the sake of the legal user. The layperson’s attempts to find help are going to be disjointed, confusing, and frustrating to the point of giving up if all these different providers don’t talk to each other, and help the person go from an Internet search to a trustworthy, supportive path of legal help.

Now’s the time to be investing in the infrastructure of the Internet for Legal Help. It will make Google search results better — surfacing jurisdiction-correct and public help resources, instead of spammy and irrelevant hits. It will allow for a new generation of legal tools to be built by young entrepreneurs and lawyers, who have ideas for better ways to present legal resources, or who just want to experiment in this space. It will foster innovation in access to justice, by giving a solid backbone of content and resources for innovators to draw from and a network of other tools to link to.

That’s how we’re going to get to “Simple in the Front and Smart in the Back” — with investment in coordinating the underlying system of how service providers present their info & share it with other providers, and by opening up the consumer law/access to justice space for more experimentation and creation.

Categories
Background

Everyone is starting to get it: innovating access to justice

Here is a small sketch I made while listening to talks at the Legal Service Corporation’s 40th Anniversary celebration in downtown San Francisco last month. It was from Justice Jonathan Lippman, the Chief Judge of NY’s Court of Appeals. The conversation was about the growing momentum from courts and lawyers to invest in new ways of providing services, getting legal information to laypeople, and giving support mechanisms to help people through the complicated court system. Chief Judge Lippman has been a consistent & vocal advocate for more user-friendly & innovative court systems, and he said that more legal professionals are coming around to this point.

Categories
Background

My (sketched) vision of the future of accessible legal services

How can we help people on-ramp into the legal system in much easier & accessible ways? This is the solution that’s been growing in my mind (but still obviously a little rough) over the last few months.

Accessible Legal Services

We need to invest in several layers at once:

1) The especially hard one: Building a central repository (or a smart, organized network) of all the information about legal services that people need when they are figuring out what legal help they need & how they can get it. This includes: basic legal info about options and procedures, eligibility info about what they qualify for, service options and hand-offs to people/orgs who can help them, know your rights materials that help them spot legal issues and respond to them quickly.

2) The more designerly one: Building better tools, interfaces, and user-facing services that help people access information from this central repository in a clean, just-in-time, quick way. These are the apps, websites, kiosks, in-person meetings, SMS channels that let a person find the right help for them right when they need it.

3) The flashier one: Building a single, memorable brand that lets a person know that the legal help they’re accessing is trustworthy, up-to-date, and right for them, and that is sticky enough that they can remember it if they’re in an emergency or just far away from lawyers.

A lot to do, I know — but an exciting one, and something that is not impossible. I have been experimenting in this second camp over the past few years — with websites & SMS channels that would let people find and access content in user-friendly ways. But I keep coming back to camp #1 & #3. We need all three in order to build a truly accessible & 21st century system of legal services.

I’m excited to be working with Open Referral & others on starting to think about how #1 is achievable, at least in a small Bay Area pilot.

Are you interested in any of these 3 undertakings? Let me know!

Categories
Ideabook Triage and Diagnosis

Can we improve how we deliver legal help via the Internet?

This week I have been finishing up my research paper on what user-centered standards for better online legal help sites would be. I had surveyed lay adults about how they’ve used the Internet in the past to respond to legal issues, and then also had them do some searches for legal help & reviews of certain legal help websites.

I’ve been playing around with small graphics to sum up some of the comments that the users have reported back. Here is one such visual:

Internet for Legal Help user voices

In addition, at Legal Design Lab we have started a working group around this topic specifically. You can read about our process here, and our outcomes, standards, and work here.

Categories
Background

An Agenda for Next Generation Legal Services

Next Generation Legal Services for access to justice an agenda

In the world of access to justice, consumer law, and even big law services, we need to think more clearly about what kinds of new products and services we should be developing. Rather than being reactive or tech-driven, we should begin with what lay people want & need to do (these are the functions we should be providing them), and what preferences they have for learning information and taking action (these are the interfaces we should be delivering the functions through).

I’ve been running a host of workshops, design sprints, and hackathons to generate these new concepts and then to test them — which kind of legal products and services have enough promise to pursue?

Margaret Hagan - legal design exploratory events

From these exploratory events, and from user-testing for projects that I’ve been developing, I’ve seen patterns of what lay people want from legal services & what kind of interfaces they want to use.

There are three types of legal products/services that we (in the world of legal innovation) need to be focusing our development efforts:

  1. Better Portals for Legal Services
  2. Process Navigator Tools
  3. Decision-Making Tools

There are plenty of other categories of tools for development later — from quality checks, to intake of client data. Other categories have lots of work going into them right now — how to assemble documents together, how to match people with lawyers.  But these three categories I’ve listed above — these are the families of products and services that users are showing high need for, and that we’re not currently working (enough) on.

Margaret Hagan - Next Generation of Legal Services - Open Law Lab - Slide058

 

Next Generation Legal Services - better legal help portals

Better Portals are online & offline entry points for legal help. This could be (one of my pet projects) a Google Search intervention, that catches legal-ish queries that a user enters into the search box, and then directs the user to good, quality, if not public & jurisdiction-specific legal resources. Or it could be real-world, situational entry-points — places in people’s everyday lives (in libraries, schools, hospitals, main streets) that allow them to get legal help in situations when they need it.

We as the legal community need to build a new set of on-ramps for people with problems to realize there can be legal relief for their ‘life problems’. And to be quality, engaging on-ramps, we need to find those touchpoints where people are open to seeking out legal help, and their preferred modes of doing so.

Next Generation Legal Services - legal Process Navigators

Process Navigator Tools are products or services that can guide a lay person (or even a novice lawyer) step-by-step through a legal matter. As the category title implies, it’s about taking a process-based view of how legal tasks can get done. We must break the procedures down into a concrete sequence of steps, and then for each step we give granular, plain English, visual guidance for how to get it done.

This kind of development work means exploding the usual ways we as lawyers convey legal guidance. No more static PDFs, no more hour long power point webinars, no more overly short & generic appetizer article about do’s & don’ts. Rather, we need a comprehensive & staged, thorough & interactive process navigator, that will lead a person through every nitty-gritty detail of getting a legal process done but do it in a responsive, smart, companion-like way.

It should be written in Plain language, it should allow a user to check tasks off, set reminders for others, save her progress, share her info & have it saved into the navigator, complete her forms and tasks on the platform, and generally be her all-in-one guide to getting this task done.

It should be like an expert paralegal, or court navigator, plus personal assistant to help a person complete all the steps of a legal procedure without missing paperwork, deadlines, or crucial small details. It should also help the person form an accurate timeline & workload expectation from the outset of a legal process, so she has a more transparent view of what’s coming & what she should be doing.

Next Generation Legal Services - Decision Making TOols

Decision Making Tools are interactive, customizable ways for a person to figure out how taking a certain action might play out. They allow a person to enter in her personal data (or an imagined version of it) and her preferences, and then it shows her potential outcomes that may result from different legal paths.

The value of these tools is in helping people to think through many scenarios, weigh their options, and see more long-term outcomes. One of the main barriers lay people express as they consider whether & how to engage legal help is ‘not knowing what I don’t know’ and ‘not being able to think through all the possible options’. People routinely express that they don’t want to pursue a legal course of action, because they’re not sure if it is comparatively the best fit for their situation & their goals.

Can we in the legal community build better tools, interventions, services that help people envision what consequences (short & long term, legal and otherwise) might result from different legal paths they take? Even if these envisionings aren’t perfectly accurate, if they help a person get a better overview of ‘what they don’t know they don’t know’ and start to play around with their preferences & resulting scenarios — they can go a long way in encouraging smarter decision-making.

 

These three camps of new products & services — better portals, process-based navigators, and decision-making tools — should be at the top of the agenda when we in the legal innovation community talk about providing better access to legal services. There is a need for each of these functions, if we are going to get people to realize that there are legal remedies for their life problems, and get them empowered to comprehend how they can get legal relief and how to choose the right path to pursue.

In later posts, I will show concept proposals, as well as examples and borrowed patterns, for each of these three product families. For now, the goal is merely to set these product families as an agenda for innovation. We need a more focused plan for how we are going to build better access to justice & legal user experience — these are three very needed & high-value targets for us to focus our innovative energy.

 

Categories
Background

What would you spend $10 million on for Access to Justice?

Access to Justice - what would you spend 10 million dollars on

I made another visual based on a short questionnaire I ran back in November-December last year, on people’s thoughts on Access to Justice. Earlier visuals of the questionnaire responses are here (Is there a coherent Access to Justice Movement?) and here (What’s going wrong with the Access to Justice Movement?)

I asked respondents where they would spend a hypothetical $10 million on Access, to see what kind of ideas & priorities would emerge. Here are the responses.

Access to Justice - where would you spend 10 million

You can see some clear priorities emerging:

  1. Experimentation in new legal services
  2. Scaling up tech-based legal platforms to cover more jurisdictions, reach more users
  3. Providing more one-to-one assistance to people going through legal processes
  4. Public legal education
  5. Establishing a ‘Brand’ for legal services

If you have other projects that you would fund with $10 million, leave your thoughts in the comments.  There must be other agenda points, add them to this first collection of funding priorities.