Hoag Law.ai

Tag: generative AI

  • Everything you need to know about California’s SB 574: The first state law to regulate AI in legal practice

    Everything you need to know about California’s SB 574: The first state law to regulate AI in legal practice

    This is educational material and does not constitute legal advice nor is any attorney/client relationship created with this article, hence you should contact and engage an attorney if you have any legal questions.

    If you’re a California attorney using generative AI — and at this point, who isn’t — you need to know about SB 574. Also, it’s inevitable that it will soon be ethical malpractice not to use AI in the practice of law, so what follows is essential knowledge.

    Introduced by Senate Judiciary Chair Tom Umberg (D-Santa Ana), SB 574 would create the first U.S. statute specifically governing how lawyers and arbitrators use generative AI.

    What SB 574 requires

    The bill creates three distinct regulatory tracks: one for attorneys, one for court filings, and one for arbitrators.

    Attorneys’ duties when using AI

    SB 574 would impose four statutory duties on any attorney using generative AI to practice law:

    Protect confidential data. Don’t feed confidential, personally identifying, or other nonpublic information into a public generative AI system. Note the qualifier — “public.” This is the bill drawing a line between consumer-facing tools (think free-tier ChatGPT) and enterprise-grade deployments with contractual data protections. The distinction matters; see below.

    Prevent discrimination. Ensure your use of AI doesn’t unlawfully discriminate against individuals or protected groups. This goes beyond what most attorneys probably think about when using AI for drafting or research, but consider: if you’re using AI to screen documents, evaluate claims, or prioritize case intake, bias in the model becomes your bias.

    Verify and correct. Take reasonable steps to verify the accuracy of any AI-generated material, correct hallucinated or misleading content, and remove biased or harmful output. “Reasonable steps” will inevitably be the subject of future litigation, but the intent is clear — you can’t blindly copy-paste.

    Consider disclosure. Consider whether your use of AI should be disclosed when producing content provided to the public. Note: this is a “consider” requirement, not a mandate. The bill leaves disclosure to professional judgment rather than imposing a blanket rule.

    Court filings and citation verification

    The bill would amend CCP § 128.7 — the sanctions statute — to make it explicitly unlawful for any court filing to contain a citation the attorney hasn’t personally read and verified, including citations provided by generative AI. This is therefore a direct response to the hallucination problem; SB 574 converts the formerly implicit “don’t do this” into a statutory prohibition with teeth: sanctions under the existing § 128.7 framework.

    For practitioners, this means building verification checkpoints into your workflow. If you’re using AI for legal research — and again, you probably are — the days of trusting the output without reading every cited case yourself are formally numbered.

    Arbitrator limits on AI

    This is the most forward-looking piece of the bill, and it arrives at an interesting moment.

    SB 574 would impose three constraints on arbitrators:

    No AI delegation. An arbitrator cannot delegate any part of their decision-making process to any generative AI tool.

    Record-only reliance. Arbitrators can’t rely on AI-generated information from outside the record without disclosing that use to all parties.

    Human responsibility. The arbitrator must assume full responsibility for the award regardless of any AI assistance.

    As a practical matter, if SB 574 passes in its current form, California-based arbitrations would effectively be unable to use the AAA-ICDR’s AI arbitrator product, at least without significant structural changes to ensure the human arbitrator retains genuine decision-making authority rather than rubber-stamping AI output.

    What SB 574 does NOT do

    A few important negatives worth flagging:

    It does not ban attorneys from using AI. This is regulation, not prohibition.

    It does not prescribe specific tools or require AI registration, audits, or approved-vendor lists.

    It does not create new criminal penalties. Enforcement is through existing mechanisms: court sanctions under § 128.7 and State Bar disciplinary proceedings. The bill doesn’t currently specify any enforcement mechanism for arbitrators, which is a gap worth watching.

    The bigger picture

    California has been on a regulatory tear: SB 53 (the Transparency in Frontier Artificial Intelligence Act, signed September 2025) established the first U.S. safety framework for frontier AI models. SB 243 regulates AI companion chatbots. AB 853 expands AI content disclosure requirements.

    SB 574 is the profession-specific layer on top of that broader regulatory stack. And if California’s pattern holds — see: CCPA becoming the de facto national privacy standard, vehicle emission rules setting the industry baseline — expect other states to follow with similar attorney AI regulations.

    These are exactly the kinds of issues I work on every day at Hoag Law.ai and as Chair of the Beverly Hills Bar Association’s AI and the Law Section, alongside Vice-Chair Amanda Harris. If you’re building AI products, navigating AI compliance, or just trying to figure out how to use these tools responsibly in your practice, let’s talk.

  • Why AI breaks traditional indemnification (and what founders need to know)

    Why AI breaks traditional indemnification (and what founders need to know)

    This is educational material and does not constitute legal advice nor is any attorney/client relationship created with this article, hence you should contact and engage an attorney if you have any legal questions.

    The entire legal architecture of indemnification was built for a world where you could control your product’s outputs. But AI breaks this assumption.When a traditional SaaS company indemnifies a customer against, say, patent infringement claims, that SaaS company can actually prevent infringement by controlling exactly what their software does. The code is deterministic. If version 2.3.1 doesn’t infringe, it won’t suddenly start infringing tomorrow.

    But an AI system? It might generate infringing content on Monday that it didn’t generate on Sunday, using the exact same model weights. You can’t “fix” this with a patch. The non-deterministic nature of generative AI means you’re being asked to indemnify against outcomes you cannot fully control or predict. This is a fundamentally different risk profile, and most MSAs haven’t caught up.

    The Three Pillars: Defend, Indemnify, Hold Harmless

    Before diving deeper into AI-specific issues, let’s clarify what these terms actually mean, because even sophisticated founders conflate them:

    Defend means you’re paying for the lawyers and managing the litigation. This is an active, immediate obligation. The moment a claim lands, you’re on the hook for legal fees, win or lose. This is often the most expensive component.

    Indemnify means you reimburse for losses and damages after the fact. This kicks in once a judgment is rendered or a settlement is reached. You’re covering the bill, but you weren’t necessarily driving the defense.

    Hold Harmless is the broadest obligation, and means you’re preventing the indemnified party from suffering any loss whatsoever. This can include things like reputational harm, business disruption, and other consequential damages that might not be covered under pure indemnification.

    The distinction matters because many contracts use these terms together (“shall defend, indemnify, and hold harmless…”) when the parties haven’t actually thought through what each entails. When something goes wrong, that ambiguity becomes very expensive very quickly.

    Why Symmetry Is Usually Right (But Not Always)

    In a typical SaaS MSA, indemnification should generally be mutual and symmetrical. You indemnify the customer for claims arising from your product; they indemnify you for claims arising from their data or misuse of your service. This makes intuitive sense because each party is best positioned to control and insure against their own risks.

    But here’s where it gets nuanced: certain asymmetries are not only acceptable but necessary.

    Carve-outs that make sense:

    • Material unlawful acts: You shouldn’t indemnify someone for their own illegal conduct. If your customer uploads stolen data to your AI tool and gets sued, that’s on them.
    • Gross negligence or willful misconduct: Standard carve-out. If one party acts recklessly or intentionally causes harm, they shouldn’t be shielded by the other party’s indemnification.
    • Modifications or combinations: If the customer modifies your software or combines it with something that creates the infringement, that’s outside your control and shouldn’t be your liability.
    • Continued use after notice: If you notify the customer of a potential infringement issue and they keep using the product anyway, the indemnification should cease.

    Asymmetries to watch for:

    Enterprise customers often hand startups contracts with wildly one-sided indemnification. Red flags include:

    • Indemnification obligations carved out of your liability cap (meaning your exposure is unlimited)
    • “Arising from” or “relating to” language that’s so broad it captures things you can’t control
    • No reciprocal indemnification from them for their data or misuse
    • Defense obligations triggered by mere allegations rather than actual claims

    The AI-Specific Problem: What Exactly Are You Warranting?

    This is where it gets genuinely difficult for companies leveraging generative AI services.

    Traditional IP indemnification works because the human is fully in the loop: You can warrant that your software doesn’t infringe third-party IP. You wrote the code. You know what’s in it. You can control it.

    With generative AI, the calculus changes because the human is, by definition, no longer in the loop end-to-end. Consider:

    Output IP risk: Your AI might generate text that infringes a third party’s copyright. Not because your model was trained on that text (though that’s a separate issue), but because the model probabilistically produced something substantially similar to protected work. You didn’t intend this. You couldn’t predict it. But under a broad indemnification clause, you might be on the hook.

    Hallucinated defamation: Your AI generates false statements about a real person or company. The customer’s end-user sees this output, relies on it, shares it. Someone sues for defamation. Who’s liable? The answer depends heavily on how your indemnification clause is drafted.

    Data privacy exposure: Your AI processes customer data in ways that arguably violate GDPR or other privacy laws—perhaps through unexpected data retention in model weights, or outputs that reveal PII from training data. The customer gets hit with a regulatory fine. Are you indemnifying that?

    The honest answer is that nobody knows how courts will ultimately allocate these risks (not to mention insurance providers). The case law is embryonic. Which means your MSA is doing heavy lifting that it probably wasn’t designed to bear.

    But here’s the uncomfortable part: none of these risks are actually unknown anymore, three years into this post-ChatGPT world.

    Output IP risk, hallucinations, data privacy leakage aren’t speculative edge cases. They’re thoroughly documented, widely discussed, and entirely foreseeable. Every AI company knows this. Every sophisticated customer knows this. The courts will certainly know this.

    Which changes the legal calculus considerably. You can’t later argue you didn’t appreciate the risk you were assuming. Foreseeability cuts against you. If you signed a broad indemnification clause covering AI outputs, you did so with eyes open. That’s assumption of risk, not surprise.

    What is genuinely unsettled is how courts will ultimately allocate liability, whether Section 230 protections apply to AI outputs, how copyright’s substantial similarity analysis works for probabilistic generation, whether GDPR fines can be contractually shifted. The minefield is clearly mapped. We just don’t know which mines are live.

    This is why precise contract drafting matters so much right now. You’re not protecting yourself against the unforeseeable. You’re negotiating the allocation of known risks before the case law catches up.

    Practical Implications for Founders

    If you’re an AI startup founder, here’s what this means for your contract negotiations:

    1. Read the indemnification clause. Actually read it. I know this sounds obvious, but most founders skim contracts looking for the price and payment terms. The indemnification section is often where the real risk lives. If you’re signing enterprise paper (their contract, not yours), this is where they’ve hidden the landmines.

    2. Push back on “arising from” or “relating to” language. These phrases are dangerously broad. “Arising from” can capture claims that have only a tangential connection to your product. “Relating to” is even worse. Negotiate for “directly resulting from” or “caused by” language that creates a tighter causal nexus.

    3. Insist on liability caps that apply to indemnification. Many enterprise contracts carve indemnification out of the overall liability cap. This means your maximum exposure isn’t 12 months of fees — it’s potentially unlimited. For an AI company facing unpredictable output liability, this is existential risk.

    4. Add AI-specific carve-outs. Consider language that limits your indemnification obligation for claims arising from: (a) outputs generated by the AI that you couldn’t reasonably prevent or predict, (b) customer’s failure to implement recommended guardrails or content filters, or (c) customer’s use of outputs without human review in contexts where review was warranted.

    5. Understand your insurance coverage. Most E&O and cyber policies weren’t written with AI output liability in mind. Before you sign a broad indemnification clause, confirm with your broker that you actually have coverage for the risks you’re assuming. You might be surprised.

    6. Consider warranty disclaimers carefully. Traditional SaaS companies warrant that their software will perform as documented. Can you warrant that about AI outputs? Probably not. Consider express disclaimers that AI outputs are probabilistic, may contain errors, and should be verified by humans before reliance.

    Don’t Indemnify Against the Unknown

    When you sign a broad indemnification clause for AI outputs, you’re agreeing to cover risks that neither party fully understands. The legal landscape is evolving. Courts haven’t weighed in on most of the novel issues AI presents. Regulatory frameworks like the EU AI Act are still being implemented.

    So you need to be clear-eyed about what you’re agreeing to, negotiate where you can, and price the risk appropriately where you can’t.

    The founders who get this right will be the ones who treat indemnification not as boilerplate to skim past, but as a core business risk that deserves the same attention as product-market fit or unit economics.

    For further questions on AI contract negotiation, MSAs, or indemnification structuring, feel free to reach out.