NYC MyCity Chatbot Told Businesses to Break the Law

The City and The Markup’s 2024 investigation found NYC’s “MyCity” business chatbot routinely answered legal questions incorrectly — it said landlords could turn away Section 8 tenants, that businesses could refuse cash payments, and that employers could legally keep workers’ tips or fire someone for disclosing a pregnancy. All of these contradict city law.

The city’s response was a disclaimer that the bot “may occasionally produce incorrect information,” not a fix. It stayed live for weeks after the errors went public. The chatbot was eventually shut down by a later administration, which called it “functionally unusable.”

Pattern match: the bot had no mapping between its answers and the actual regulatory text (No Policy Mapping), and it produced outputs that violated real compliance boundaries with no guardrail catching it before the answer reached a business owner (Compliance Boundary Violation).

The fix: ground answers in the authoritative policy source at generation time, not after the fact — and treat “the model might be wrong” disclaimers as a mitigation of last resort, not a substitute for review on legally consequential answers.