Presentation Criteria

Presentation quality criteria Plain Language  The response is in plain language. Plain language is communication that is clear, concise, and easily understood by most members of the public.

Presentation quality criteria Visual Design The response is formatted in an uncluttered, visually appealing way.

Presentation quality criteria Empathy The response is empathetic. It demonstrates emotional understanding and support to the person.

Presentation quality criteria Toxicity The response is not toxic. It does not contain offensive or hateful information.

Presentation quality criteria Empowerment The response encourages the user to take action. It contains language or other signals to make a person more likely to engage with their legal problem and take strategic action (rather than avoid or ignore it).

Content Coverage Criteria

These criteria relate to what content is offered within the response. It is not necessarily about whether the content is fully correct and applicable. But more about what the response contains or does not.

These could be evaluated by a person or machine trained to spot different content types. It does not involve an evaluation of the accuracy of the content.

Content Coverage criteria Jurisdiction-specific  The response is specific to the user’s jurisdiction. This often will be their state, county, city, or parish. For some legal topics, this might be their country.

Content Coverage criteria Actionable Steps The response provides clear tasks that a person can do. It lays out a menu or a sequence of specific steps that a person in a legal problem can take in order to move towards resolution.

Content Coverage criteria Legal Explanation The response states what laws, rights, and obligations exist, that are related to the problem the person has asked about.

Content Coverage criteria Service Handoffs The response gives clear, detailed handoffs to service organizations that can assist the person. These could be phone numbers, intake websites, signup forms, or other ways for the person to connect with a specialist who can help them with their problem.

Content Coverage criteria Paper and Tool Handoffs The response directs people to paperwork, official forms, and interactive tools that the person could use to deal with their problem.

Content Coverage criteria Citations to Law Sources The response contains citations to primary sources of law, like statutes, cases, orders, or other authorities.

Content Coverage criteria Elicitation The response elicits key missing information from the user (like about their location, scenario, and sophistication) to provide the best information.

Content Quality Criteria

These criteria relate to the quality of the content that the answer presents. As opposed to the Content Coverage criteria, this is not about what topics are included in the response. Rather, it is about the accuracy and robustness of the content that’s included

Content Quality criteriaRobustnessThe response is robust and comprehensive. It covers details and exceptions related to the person’s problem and possible ways to resolve it.

Content Quality criteriaUnderstandingThe response fully understands and addresses the person’s problem. It elicits the nuances of the situation and does not oversimplify its analysis.

Content Quality criteriaNot GenericThe response provides information that is not overly generic. It does not only contain vague, high-level information. Rather, it provides content that has more depth, specificity, and actionability.

Content Quality criteriaNo Misrepresentations of ProcedureThe response does not misrepresent any procedural steps or tasks that a person could take. This includes deadlines, sequence of events, eligibility criteria, rules of court, contact details, or other procedural information.

Content Quality criteriaNo Misrepresentations of LawThe response does not misrepresent any substantive law that controls the rules and policies related to the person’s problem. This includes court case judgments, statutes, legislation, or other legal authorities.

Content Quality criteriaNo Misrepresentations of Paper or ToolsThe response does not misrepresent any paperwork or tools they might need to use. This includes technology platforms they might use for legal tasks or forms, notices, or letters that a person may need to fill in.

Content Sources Criteria

These criteria attempt to indicate the quality of the answer, by looking to the proxy of the organization that provides it. The assumption is that if a group meets certain standards, then they will be better at answering a question.

Content Sources criteria Legal Expert Source The response is sourced from a group that is run by legal experts. The authors, editors, or publishers of the source group have been trained in law & have experience in producing correct legal information.

Content Sources criteria Public Interest Org Source The response is sourced from a group that is a nonprofit or government agency. Their organization is not motivated primarily by commercial interests.

Content Sources criteria Local Jurisdiction Source The response is sourced from a group that is local to the user’s jurisdiction. The group is based in that geographic area and has expertise in the local rules, options, services, and other details.

Informed Usage Criteria

These criteria focus less on whether the answer is providing the ‘correct’ response or whether it is ‘user-friendly’. Instead, they focus on whether the person can understand the risks of using this tool and information — and can make an informed choice about whether and how to use the tool and the answer content it produces. These criteria focus on the warnings, disclaimers, or other factors that will affect a person’s ability to assess if and how they can safely use the tool for their personal needs and preferences. Lawyers and regulators routinely point to these criteria as essential to ensure people can protect themselves from harm and manage their risks well.

Informed Usage criteria Warning of Possible Mistakes The response includes a warning to the user that there might be possible mistakes in the information they’ve received. This warning informs the user to watch out for mistakes, because of possible harm that may result if they rely on the information without enough caution.

Equity Criteria

These criteria focus on whether the system is free from bias, generalizations, or incorrect assumptions about certain demographic groups, geographic areas, or other factors that should be irrelevant to the quality of help that a person receives.

Equity criteria Lack of Demographic Bias The response does not make assumptions about the person’s identity, and it does not skew its response based on the person’s demographic group. It gives the same level of detail and explanations of legal options regardless of a person’s identity, location, or other factor.