Your Law Firm Has an AI Policy. How Do You Know Anyone Is Following It?
Why effective AI governance requires more than written rules, and how law firms can translate professional obligations into supervision, verification, enforcement, and institutional accountability.
Analysis by
Founder, Lex Wire Journal • Technology, Governance & Sovereignty Strategist
Consider a midsize law firm that has recently adopted a comprehensive artificial intelligence policy.
The policy identifies approved AI tools, prohibits uploading confidential client information into unauthorized platforms, requires attorney review of AI-generated legal research, and establishes general expectations for responsible use.
The policy is distributed throughout the firm. Attorneys and staff acknowledge that they have received it. The management committee considers the initiative complete.
Several months later, a partner discovers that an associate has been using a personal AI account to assist with drafting motions. A paralegal has uploaded discovery documents to a different application. Another attorney has incorporated AI-generated citations into a memorandum without reviewing the underlying authorities.
The firm’s written policy prohibited all three activities.
Does the firm have effective AI governance?
The question is more difficult than it first appears.
The policy may be professionally sound. It may accurately reflect the firm’s understanding of its obligations. It may even prescribe reasonable safeguards.
But the firm has not established whether those safeguards are actually operating within its legal workflows.
The distinction is between adopting a policy and implementing a functioning system of accountability.
In my continuing legal education program, Ethics in the Age of AI, I emphasize the importance of clear procedures, role-specific training, independent verification, and ongoing supervision of AI-assisted legal work.
Those measures address a problem that becomes more significant as AI moves from occasional use into routine legal operations.
A firm’s obligations cannot be fulfilled simply by assuming that every lawyer and employee will independently interpret and follow an AI policy.
The institution must consider whether it has established reasonable measures to support compliance and identify problems when they arise.
The central issue is therefore not whether a law firm has a policy.
It is whether the firm has a reasonable basis for believing that its people, systems, and workflows are operating consistently with that policy.
The Bottom Line
Effective AI governance requires more than a written policy. Law firms must establish reasonable organizational measures that translate professional obligations into actual practices. ABA Model Rules 5.1 and 5.3 provide an important foundation for managerial and supervisory responsibilities.
The Lex Wire Operational AI Accountability Framework proposes five connected functions: Define, Implement, Verify, Enforce, and Evolve.
These functions distinguish the existence of governance rules from the firm’s ability to determine whether they are being followed, address deviations, and improve its practices over time.
The Policy Illusion: Why Written AI Rules Are Only the Beginning
Written policies play an important role in the management of legal practice.
They can establish expectations, allocate responsibilities, define permitted conduct, and create consistent standards across attorneys and staff.
An AI policy may specify which applications are approved, what information may be entered, when attorney verification is required, and who must authorize higher-risk uses.
Those provisions are useful. But they describe intended behavior, not necessarily actual behavior.
A law firm can adopt an excellent policy while continuing to operate through inconsistent practices.
Employees may misunderstand a restriction, lack access to the approved technology, or rely on an informal workaround because the intended process is impractical.
A supervising attorney may believe someone else has reviewed an AI-generated document. A vendor’s settings may change after a system was originally approved.
None of these possibilities establishes that a particular law firm is noncompliant.
They demonstrate why a policy alone cannot establish the effectiveness of governance.
| Governance Stage | What Is Established | What Remains to Be Evaluated |
|---|---|---|
| Policy Adopted | Written expectations | Whether employees understand them |
| Policy Communicated | Guidance and training | Whether workflows reflect the guidance |
| Policy Implemented | Procedures and controls | Whether the controls operate as intended |
| Governance Verified | Reasonable evidence of implementation | Whether controls remain effective over time |
The distinction matters because a firm may mistake administrative completion for operational success.
Distributing a policy and collecting acknowledgments are measurable activities. They do not necessarily demonstrate whether legal work is being performed in accordance with the policy.
In my view, governance should therefore be understood as an ongoing organizational practice rather than a one-time policy development project.
ABA Rules 5.1 and 5.3: The Professional Responsibility Foundation
The professional responsibility foundation for operational governance is already established within the ABA Model Rules.
ABA Model Rule 5.1 addresses the responsibilities of partners, managers, and supervisory lawyers.
Rule 5.1(a) requires lawyers with specified managerial authority to make reasonable efforts to ensure that the firm has measures providing reasonable assurance that lawyers comply with the Rules of Professional Conduct.
Rule 5.1(b) addresses lawyers who directly supervise other lawyers. They must make reasonable efforts to ensure that those lawyers conform to professional obligations.
Rule 5.1(c) separately identifies circumstances in which a lawyer may be responsible for another lawyer’s professional misconduct.
That provision is important because managerial responsibility should not be confused with automatic disciplinary liability for every subordinate violation.
The duties to establish reasonable firm measures, supervise others, and respond to known misconduct have different legal elements.
The Rule 5.1 comments also recognize that the appropriate measures may depend on the firm’s size, structure, and nature of practice.
A small firm of experienced attorneys may be able to operate through relatively informal supervision and periodic review.
A larger organization or one handling especially complex ethical issues may require more elaborate procedures.
ABA Model Rule 5.3 establishes related obligations concerning nonlawyer assistance.
These duties can become significant when paralegals, administrative personnel, and other nonlawyer staff incorporate AI into their work.
ABA Formal Opinion 512 , issued July 29, 2024, applies existing professional duties to generative artificial intelligence.
Its supervision discussion addresses clear policies governing permissible generative AI use, appropriate oversight, and training on relevant technologies and risks.
The opinion does not create a universal requirement that every law firm adopt identical software, staffing, or audit procedures.
Instead, it reinforces the continuing responsibilities of lawyers who manage and supervise AI-assisted legal work.
“Effective AI supervision means establishing clear protocols for each tool, because they all work a bit differently and offer different styles of output.”
Jeff Howell, Esq. | Ethics in the Age of AI, Original CLE Instructional Materials
An important limitation should be recognized.
AI systems are not human associates or employees. Rule 5.3 directly governs responsibilities regarding nonlawyer assistance.
Nevertheless, when personnel use AI in performing legal work, their professional supervisors must address the risks associated with that work.
The operational question becomes whether the institution has established reasonable measures to support those obligations.
Five Places Where Law Firm AI Governance Can Break Down
A policy’s effectiveness can be undermined at several points between drafting and implementation.
Drawing from the practical concerns examined in my CLE, five areas deserve attention.
1. Unclear Authorization
Personnel may not understand which AI applications are permitted or how approval differs across particular tasks.
A tool may be appropriate for reviewing public information but not approved for processing privileged client communications.
A meaningful policy should communicate these distinctions in terms employees can apply.
2. Inadequate Training
Employees may know that a policy exists without understanding how to follow it.
A requirement to verify AI output is incomplete if the relevant attorney does not understand how to evaluate source integrity, factual accuracy, and legal reasoning.
3. Missing Workflow Controls
A firm might require attorney approval without identifying when approval occurs or what material the attorney must review.
The resulting procedure may rely on assumptions rather than a clearly assigned responsibility.
4. Undefined Supervision
A firm may distribute responsibility among practice leaders, information technology personnel, and individual attorneys without specifying who makes the final professional decision.
Technology personnel can support implementation, but technical administration does not replace legal professional judgment.
5. No Feedback or Correction Mechanism
Even carefully designed controls can become outdated or function differently than intended.
Without an appropriate way to identify deviations, investigate incidents, and revise procedures, the firm may continue relying on ineffective practices.
These are illustrative operational failure points, not an empirical ranking of the most common causes of AI policy failures.
Their common feature is a disconnect between a written expectation and an observable practice.
Shadow AI: When Employees Use Tools the Firm Has Not Approved
One challenge for organizational governance is the use of AI applications outside established approval and oversight procedures.
This activity is often described as shadow AI.
In practical terms, it can include personnel using personal AI accounts, subscribing to unapproved services, or connecting unauthorized applications to firm information.
But the problem extends beyond the use of unapproved products.
An approved application can also be used for an unauthorized purpose.
Consider a firm that authorizes an AI assistant for general legal brainstorming using public information.
An attorney may incorrectly assume that approval also permits uploading sensitive client documents into the same service.
The product is approved. The particular use may not be.
Another concern arises when software settings or access permissions differ from those evaluated during the firm’s approval process.
These examples demonstrate why an approved-tool list should not be treated as complete governance.
Reasonable organizational measures may include clear restrictions, approved alternatives, relevant access controls, and straightforward procedures for requesting authorization of new tools.
Firms should also examine why unauthorized use occurs.
An employee may disregard a clear requirement. Alternatively, the approved workflow may be inaccessible, confusing, or unsuitable for a legitimate task.
Those circumstances require different responses.
Appropriate accountability must address individual conduct, but governance also requires evaluating whether organizational controls function as intended.
This does not necessarily require universal monitoring of employees’ digital activity.
Monitoring practices themselves should respect confidentiality, legitimate privacy considerations, and applicable law.
AI Risk Classification: Different Tasks Require Different Controls
An important theme in my original CLE is the need to tailor verification and confidentiality measures to the activity involved.
A general-purpose AI assistant used to brainstorm a public legal topic presents different professional concerns from a system processing privileged discovery documents.
The same technology can also present different risks depending on the task.
A reasonable governance program should therefore classify uses and information, not merely products.
“Create verification protocols that are specific to each type of AI use. Citation checking for research, privilege review for document analysis, fact verification for content creation.”
Jeff Howell, Esq. | Ethics in the Age of AI, Original CLE Instructional Materials
| AI Activity | Principal Concern | Illustrative Control |
|---|---|---|
| Public-Information Research | Incorrect or incomplete legal analysis | Source verification before reliance |
| Client-Specific Drafting | Confidentiality and substantive accuracy | Approved environment and attorney review |
| Court Filing Preparation | False authorities and tribunal obligations | Appropriate prefiling verification |
| Discovery Review | Incorrect relevance or privilege classifications | Validated workflow and professional oversight |
| Client Intake Assistance | Sensitive information and misleading advice | Defined scope and escalation to lawyers |
| Litigation Predictions | Unsupported assumptions and overreliance | Independent professional evaluation |
This table provides illustrative control categories. It does not establish a legally mandated risk-ranking system.
For additional organizational guidance, the National Institute of Standards and Technology AI Risk Management Framework offers a voluntary structure organized around four functions: Govern, Map, Measure, and Manage.
Those functions provide a useful general reference for understanding AI risks, evaluating organizational controls, and adapting practices over time.
NIST’s framework is not a substitute for a lawyer’s professional duties, and its adoption does not automatically establish compliance with applicable ethics rules.
Its value in this context is helping firms think systematically about the relationship between AI risks and organizational responses.
The Lex Wire Operational AI Accountability Framework
The distinction between governance policy and governance performance can be translated into a practical operating model.
I propose the Operational AI Accountability Framework , organized around five functions: Define, Implement, Verify, Enforce, and Evolve.
The framework is intended to help law firms convert professional obligations and internal policies into functioning organizational procedures.
It is my proposed professional methodology, not a separate ABA standard or judicially adopted test.
From organizational expectations to verifiable and adaptable practice.
Establish approved uses, restrictions, risk categories, decision-making authority, and relevant professional responsibilities.
Key question: Can personnel identify what the firm requires for their particular task?
Translate policies into approved tools, workflows, permissions, training, and task-specific review procedures.
Key question: Do the firm’s operating systems support the required behavior?
Assess whether the controls are functioning through appropriate supervision, testing, sampling, or review.
Key question: What reasonable evidence exists that the procedures are being followed?
Address unauthorized uses, material deviations, supervision failures, and corrective actions.
Key question: What happens when actual conduct departs from the firm’s requirements?
Reassess policies and procedures as tools, vendor arrangements, professional guidance, and operating conditions change.
Key question: How does the firm know when existing safeguards need to be reconsidered?
Define and Implement: Expectations Must Become Workflows
Defining acceptable conduct is an essential first step, but implementation is where a written policy begins influencing actual legal work.
A rule prohibiting confidential information in unauthorized AI applications should be supported by clear approved alternatives and appropriate guidance concerning information classification.
A requirement for attorney review should identify the tasks requiring review and the nature of the professional evaluation.
These measures reduce the distance between a general rule and a decision an employee must make in practice.
Verify: Determine Whether Controls Function
Verification concerns whether the prescribed practices are occurring with reasonable consistency.
It may involve sampling completed work, reviewing permissions, examining exceptions, or assessing employee understanding.
The required intensity should be proportionate to the work, risks, and firm structure.
Enforce: Respond to Deviations
Verification has limited value if the firm consistently identifies material deviations but takes no appropriate action.
Responses may include corrective instruction, access restrictions, workflow changes, additional supervision, or other measures justified by the circumstances.
Enforcement should not be equated solely with employee discipline.
A recurring deviation may also reveal an impractical workflow or unclear policy.
Evolve: Keep Governance Current
AI products and their operating arrangements can change.
New features, revised contracts, different information flows, or legal developments may require the firm to reconsider an earlier approval.
Governance should therefore include appropriate triggers for renewed evaluation.
“A policy defines what an organization intends to do. Operational accountability requires the organization to determine what is actually happening and respond when the two diverge.”
Jeff Howell, Esq. | Lex Wire Journal, Professional Analysis
These five functions are connected.
A firm may move through them repeatedly as its use of AI becomes more sophisticated.
The objective is not bureaucratic perfection.
It is the establishment of reasonable, functioning practices that support the firm’s professional responsibilities.
How Can a Law Firm Verify That Its AI Policy Is Being Followed?
The question at the center of operational governance is how a firm can develop a reasonable basis for assessing compliance with its own requirements.
No single monitoring method is appropriate for every practice.
But several approaches can provide useful information.
Review Selected AI-Assisted Work
A firm might examine a sample of completed assignments to determine whether the prescribed verification steps occurred.
For legal research, that may involve checking whether material authorities were reviewed and accurately represented.
For document analysis, it may involve assessing how significant exceptions were identified and resolved.
Review Tool Permissions and Approved Uses
Authorized applications should be evaluated in connection with their actual configurations and permitted purposes.
A firm may periodically review whether staff access remains appropriate and whether a provider’s material terms have changed.
Assess Training Effectiveness
Completing a training module does not necessarily establish that an employee understands how to apply its requirements.
Practical examples, scenario-based questions, or supervisor discussions may provide more useful information about understanding.
Evaluate Incidents and Exceptions
A firm should establish an appropriate process for reporting and evaluating material AI-related concerns.
The point is to determine whether an incident reflects isolated conduct, inadequate training, a defective workflow, or some combination.
These methods should be proportionate.
A small firm may accomplish much through periodic attorney review and direct discussions.
A large organization may require designated governance personnel and more formal reporting.
Neither model guarantees perfect compliance. Both can be evaluated against the applicable reasonable-efforts obligations.
Measuring AI Governance: What Actually Demonstrates Effectiveness?
Law firms can easily measure administrative activity.
They can count completed training sessions, approved software products, signed policy acknowledgments, and committee meetings.
These measures may be useful, but they do not necessarily establish that employees are making better professional decisions.
The more difficult question is whether governance measures are functioning as intended.
| Possible Indicator | What It Can Help Evaluate |
|---|---|
| Training Completion | Whether relevant personnel received instruction |
| Practical Assessments | Whether personnel understand key requirements |
| Review Exceptions | Where workflow errors or deficiencies occur |
| Incident Resolution | Whether identified problems receive timely attention |
| Repeated Deviations | Whether controls, guidance, or workflows need improvement |
| Vendor Reassessment | Whether material changes are being evaluated |
These measures are indicators, not conclusive proof of professional compliance.
For example, a low number of reported incidents may indicate fewer problems.
It could also mean that employees do not recognize potential problems or do not report them.
Likewise, an increase in identified exceptions could reflect deteriorating practices or improved detection.
Measures must therefore be interpreted in context.
The objective is not to generate a perfect dashboard or claim that every ethical risk can be reduced to a numerical score.
It is to give firm leadership sufficiently useful information to exercise informed supervisory judgment.
AI Training and a Culture of Professional Accountability
Effective governance depends on the people who carry out the firm’s work.
Policies and technology controls are important, but attorneys and staff still make decisions about what information to submit, when to rely on an output, and when to seek additional professional review.
My original CLE emphasizes that AI training should be ongoing and tailored to the responsibilities of different personnel.
Partners and managers should understand their supervisory roles and the firm’s principal operational risks.
Practicing attorneys need task-specific guidance concerning source verification, professional judgment, confidentiality, and client communication.
Paralegals and support staff need clear guidance concerning approved applications, information restrictions, authorized tasks, and escalation procedures.
Training should help personnel recognize when a matter exceeds their authority or requires attorney judgment.
An effective reporting process is equally important.
Personnel should understand how to raise uncertainty or identify a problem before it becomes more serious.
Appropriate accountability does not mean that every mistake receives the same response.
A misunderstanding may require clarification or training.
Deliberate disregard of a material restriction may justify a different response.
The governance system should allow the firm to distinguish those circumstances and act reasonably.
Three Practical AI Governance Scenarios
The following hypothetical examples illustrate how the Operational AI Accountability Framework can guide organizational responses to different risks.
The Unapproved AI Research Assistant
An associate uses a personal AI account to help draft a litigation brief.
The firm has an approved-tool policy, but the associate finds the unauthorized application easier to use.
The supervising lawyer discovers the practice during review.
The immediate response should evaluate what information was entered, whether the work was independently verified, and whether any applicable obligations have been implicated.
The institutional response should also consider whether the restriction was understood, whether approved tools were reasonably available, and whether existing supervisory controls functioned appropriately.
Under the framework, the firm examines the authorization rule, its implementation, detection of the issue, corrective action, and possible improvements.
The response addresses both the particular incident and the system that allowed the issue to arise.
AI-Assisted Discovery Without Adequate Review Procedures
A litigation team adopts an approved AI document-review system.
The software identifies potentially privileged communications, but the firm has not established sufficiently clear procedures for handling uncertain classifications.
A review identifies inconsistencies in how different team members handle potentially protected information.
This example demonstrates the distinction between approving a product and approving a reliable process.
The firm may need to clarify review standards, establish escalation procedures, provide additional training, or modify quality-control practices.
The relevant professional obligations depend on the circumstances, including the nature of the information and any applicable discovery orders.
The governance lesson is that product approval is not a substitute for task-specific professional oversight.
The Approved Vendor Changes Its Terms
A firm approves an AI platform after examining its information-handling terms and available safeguards.
The provider later materially revises its retention or processing arrangements.
The firm must determine whether the revised arrangements remain appropriate for the authorized uses.
A previous approval should not necessarily be treated as permanent approval regardless of changes.
The firm may need to review the new terms, consult appropriate technical personnel, revise permissions, or modify the approved workflow.
This is the Evolve function in practice.
A governance system should recognize meaningful changes and respond when earlier assumptions are no longer valid.
Institutional Intelligence: Governance Beyond Individual AI Tools
As AI becomes integrated into legal operations, the governance question expands beyond individual applications.
Firms may increasingly connect AI systems to document repositories, internal precedents, research collections, practice workflows, and institutional knowledge.
These connections may create valuable capabilities that depend on particular vendors or technical arrangements.
The resulting issues include information access, portability, configuration control, retention, and the firm’s ability to change service providers.
These are not automatically independent ethics violations.
They are broader governance and institutional control considerations that may intersect with professional duties.
I examine those questions in Who Controls Your Law Firm’s AI Intelligence Layer? , which considers the relationship between technology platforms, accumulated institutional capabilities, and digital sovereignty.
The connection is important.
A firm that does not understand its operational AI environment may struggle to evaluate what it controls, what it depends upon, and how its practices will be affected by future changes.
Governance therefore has both an immediate professional responsibility dimension and a longer-term institutional dimension.
The Bottom Line: Governance Is an Institutional Practice
A law firm can adopt a comprehensive AI policy without knowing whether its attorneys and staff are consistently following that policy.
That gap is the central problem of operational AI governance.
Written requirements are important, but they are only the beginning of organizational accountability.
Effective governance requires reasonable measures that connect policy to actual practice.
The Operational AI Accountability Framework provides five connected functions: Define, Implement, Verify, Enforce, and Evolve.
Together, these functions support a process through which firms can clarify expectations, establish workflows, assess performance, respond to problems, and adapt to changing conditions.
The objective is not perfect control over every employee or technological interaction.
Nor is it to replace professional judgment with increasingly rigid procedures.
It is to establish reasonable institutional systems that reinforce professional judgment and accountability.
“The purpose of AI governance is not to eliminate professional judgment through increasingly rigid procedures. It is to create institutional systems that support, verify, and reinforce the exercise of that judgment.”
Jeff Howell, Esq. | Lex Wire Journal, Professional Analysis
Rules 5.1 and 5.3 provide the professional responsibility foundation for reasonable managerial and supervisory measures.
Formal Opinion 512 places those obligations within the context of generative AI.
The practical challenge is translating those principles into organizational conduct.
A firm’s AI governance should ultimately be evaluated not only by the policies it has adopted, but by the reasonable measures it has established to ensure those policies are reflected in the work its people perform.
Five Questions for Law Firm Leadership
- Can our attorneys and staff identify which AI uses are permitted for their work?
- Do our workflows provide the tools, instructions, and supervision necessary to follow those requirements?
- Do we have a reasonable way to evaluate whether the procedures are being followed?
- Is there a defined process for handling material deviations, errors, and emerging concerns?
- Who is responsible for reassessing our AI governance as technology, contracts, and legal obligations change?
Related Lex Wire Journal Analysis
- Building an AI Governance Framework for Law Firms: Six Pillars of Operational Accountability examines foundational governance responsibilities and organizational controls.
- The Ethics of Artificial Intelligence in Legal Practice: Professional Judgment in the Age of Automation addresses the attorney’s independent professional judgment when using AI.
- Beyond Hallucinations: Evidence, Verification, and Defensible AI Use in Legal Practice develops the Defensible AI Work Product Framework.
- AI and the Attorney-Client Privilege: Who Can Access Your Client’s Confidential Information? examines third-party processing, confidentiality, privilege, and information control.
- Client Communication in the Age of AI: Transparency, Informed Consent, and the Attorney’s Duty to Explain considers how AI use intersects with informed client participation.
- Building Your Law Firm’s AI Playbook addresses practical policies, procedures, and implementation.
Legal Authorities and Source Materials
Professional Responsibility
- American Bar Association, Model Rule 5.1: Responsibilities of Partners, Managers, and Supervisory Lawyers .
- American Bar Association, Comments to Model Rule 5.1 , addressing organizational measures, firm structure, supervision, and professional responsibility.
- American Bar Association, Model Rule 5.3: Responsibilities Regarding Nonlawyer Assistance .
- American Bar Association, Formal Opinion 512: Generative Artificial Intelligence Tools (July 29, 2024).
- American Bar Association, Model Rules of Professional Conduct , including Rules 1.1, 1.4, 1.6, 5.1, and 5.3.
AI Risk Management and Professional Guidance
- National Institute of Standards and Technology, AI Risk Management Framework , including Govern, Map, Measure, and Manage.
- American Bar Association, A Sample of Artificial Intelligence Risk Management Policies for Law Firms .
Original CLE Instructional Materials
- Jeff Howell, Esq., Ethics in the Age of AI: Presentation Outline and Script , continuing legal education instructional materials, including the sections on professional supervision, verification systems, information security, employee training, and practical AI governance.