It took a subpoena, a threat of court enforcement, and nearly three weeks of defiance — but on Friday, October 16 at 10:00 (UTC-4), Elon Musk's SpaceXAI finally sat down (virtually) before the New York City Council. What happened next was a masterclass in why "self-regulation" is not a safety strategy.
The subpoena that broke precedent
This wasn't a routine hearing. On September 28, Council Speaker Julie Menin issued the New York City Council's first-ever subpoena against a private actor, compelling SpaceXAI to testify after the company skipped an October 5 hearing where Anthropic, Google, Meta, and OpenAI all appeared under oath. Menin then threatened to ask a state judge to enforce it. Two days later, SpaceXAI agreed to show up.
The company sent Gabriela Castillo Madrid, Senior Manager of Global Government Affairs — not Dario Amodei-level leadership, but a government relations staffer. That choice itself told the room how seriously SpaceXAI was taking the proceeding.
Grok failed the live test
Castillo Madrid opened by insisting Grok has "strict controls in place to prevent the sexualization of children or real people."
Menin didn't argue. She pulled out her phone.
"That morning, my team put my name into regular, free Grok — not spicy mode — and a sexualized image of me came up in less than 30 seconds," Menin said, according to the Council's official account. "It is available for all to see. For my daughter to see. For anyone to see."
Castillo Madrid could not explain how the image bypassed the controls she had just described. She offered to have her safety team "look into that specific instance."
This wasn't a hypothetical risk or a red-team finding buried in a system card. It was a sitting elected official holding up a real, generated sexualized image of herself while the company's representative sat under oath and could not account for it.
The whistleblower: 3 safety people out of 200
The hearing also featured Devin Kim, a former SpaceXAI employee who says he was fired after raising safety concerns about Grok. Kim testified that during his tenure, only three of the company's 200 technical employees worked on AI safety training — 1.5% of the technical workforce. He also described an internal content standard that allowed what he called "tasteful nudity."
Asked under oath whether she was aware of Kim's termination or his allegations, Castillo Madrid said she was not.
Kim's claims are unproven and reflect his personal tenure, not necessarily current staffing. But the contrast is stark: Anthropic just embedded six external METR evaluators with employee-level access (reported October 15, UTC), and OpenAI maintains a dedicated Preparedness Framework team. Three people for a product used by millions is a number that deserves scrutiny.
A British MP is already suing
Menin wasn't the only elected official with a Grok problem. The Council heard from British Member of Parliament Jess Asato, who is currently in a legal battle with SpaceXAI after Grok generated nonconsensual explicit photos and videos using her likeness. Asato's testimony connected the New York hearing to an emerging global pattern: public figures, particularly women, are becoming the first high-profile victims of unregulated AI image generation.
What the Council is actually doing
The hearing wasn't just theater. Two concrete pieces of legislation are moving:
- Intro 504, sponsored by Deputy Speaker Dr. Nantasha Williams, would ban the unauthorized AI generation of photos, videos, or audio depicting elected officials.
- Intro 2605, sponsored by Menin, is a first-in-the-nation law that would let whistleblowers collect a portion of fines recovered from AI companies that break the law — essentially a False Claims Act model for AI safety.
The Council is also preparing follow-up letters to the companies and has signaled additional legislation is coming.
Why this matters
The easy take is "SpaceX got dragged to court and Grok embarrassed itself." The deeper story is about who gets to set AI safety rules, and the answer is shifting fast.
Three weeks ago, the dominant narrative was that AI labs would regulate themselves — Anthropic was pitching its own federal bill (reported October 16, UTC), OpenAI had its Preparedness Framework, and companies were racing to demonstrate voluntary responsibility. Friday's hearing showed the limit of that model: when a company simply refuses to show up, voluntary frameworks collapse. The subpoena was the blunt instrument that forced the issue, and Menin made clear it won't be the last.
The 3-out-of-200 figure, if accurate, is the more disturbing data point. AI safety isn't a feature you bolt on after launch; it's a staffing commitment. For context, Anthropic's safety team is estimated in the hundreds, and even that has been criticized as insufficient. If a frontier lab is allocating 1.5% of technical headcount to safety while shipping image generation to millions of free users, the math doesn't work — and the Grok demo proved it in real time.
There's also a political dimension worth watching. New York City isn't Congress, but it's the largest city in the country and a legislative first-mover. Intro 2605's whistleblower bounty model is the kind of mechanism that could spread to state legislatures and eventually Washington. If AI companies face financial liability driven by their own former employees, the calculus around cutting safety corners changes.
The critical caveat: city-level regulation has real limits. Grok is a global product; a New York City ban on AI-generated official imagery won't stop someone in another jurisdiction from generating it. And SpaceXAI has deep pockets for legal challenges. The hearing's lasting impact may be less in the bills themselves and more in the precedent it set — that governments will compel testimony, and that "self-regulation" won't be accepted as an answer when the product is demonstrably failing.
What to watch next
- Whether SpaceXAI's safety team actually investigates Menin's image and publishes findings, or whether "we'll look into it" becomes the end of it.
- Whether Intro 2605's whistleblower bounty model gets picked up by California or the EU — that's the provision with real teeth.
- Whether other cities issue their own subpoenas. Once one council does it, the cost of doing it drops.
- Whether Kim's allegations prompt more former SpaceXAI employees to come forward. Whistleblower cases rarely stay at one person.
The era of AI companies choosing whether to answer to governments may be ending. New York just proved you can force the issue. The question now is whether anyone else will.
No comments yet