This is a linkpost for https://amistrongeryet.substack.com/p/increasing-visibility

As many readers of this blog know all too well, there has been ferocious debate around California SB 1047, a bill which would enact regulations on AI. Even the “Godfathers of AI” – Yann LeCun, Yoshua Bengio, and Geoffrey Hinton – are divided. LeCun seems to hate the bill; last month he tweeted^[1]:

Excellent argument by @AndrewYNg against the ignominious California regulation SB1047, which would essentially kill open source AI and significantly slow down or stop AI innovation.

Meanwhile, Bengio and Hinton signed a letter in which they “express our strong support”:

…we are deeply concerned about the severe risks posed by the next generation of AI if it is developed without sufficient care and oversight. SB 1047 outlines the bare minimum for effective regulation of this technology.

It is tempting to think that people are just “talking their book” – supporting or opposing the bill according to how it affects their professional or financial interests. Opponents of the bill are often associated with AI labs (LeCun is Chief AI Scientist at Meta), while many proponents work on AI safety. But I think there’s more to the story.

It’s Not All About Vested Interests

OpenAI opposes SB 1047, but competitor Anthropic has more or less endorsed it. What’s going on here?

To shine some light on what drives views of the bill, let me talk about two people I know personally – Dean Ball of the Mercatus Center and Nathan Labenz of the Cognitive Revolution podcast. Dean and I recently appeared on Nathan’s podcast^[2] to discuss the bill, and they participated in an offline panel discussion on AI regulation that I recently organized. They are both thoughtful, honest brokers, well versed in current developments. And they are on opposite sides of the debate. Dean has serious concerns with the bill:

Maybe it’s a worthwhile tradeoff. ... Maybe AI capabilities will become sufficiently dangerous that releasing them without extensive, government-mandated testing is wildly irresponsible. Maybe they’ll become so dangerous that it really is too risky to release them as open source, since currently anyone can subvert the safety protections of an open-source model. And maybe after that happens, Meta or another well-resourced company, with its shareholders and its public reputation on the line, will choose to disregard a

...

Posts

Wiki Contributions

Comments