The Download: AI’s refusal problem and weight-loss drug side effects

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

We’re putting too much faith in AI’s ability to say no

Today’s AI models are trained to refuse a vast number of prompts. If you ask your chatbot how to poison a colleague or how to tie a noose, chances are it’ll turn you down. But refusal fails, sometimes horrifically.

Some people are attempting to use AI to hone biological pathogens and build autonomous drone swarms. Sooner or later, failed refusals may result in global calamity. What’s more, relying on refusal means drawing a line between what a model should obey and what it must disobey.

Governments will also get to draw their own lines, potentially restricting free speech. At the same time, the capabilities that make AI useful could also make it dangerous. …