Happy holidays to everyone who celebrates anything at this time of year! In between the solstice and Christmas, I sent out a holiday edition of AI Week. I hadn’t actually intended to do a newsletter, but there was so much interesting news that I couldn’t resist sharing it. I hope you’ll enjoy this Winter Holiday Edition of AI Week.
In this week’s AI Week:
- Blizen the Red-Nosed Pegasus
- Winter Holiday Headlines: No patents for AI in UK; how to break chatbots’ inhibitions; and oh there’s CSAM in the training dataset
- Winter Holiday Linkdump: AI-generated everything
- Followups: Chevrolet of Watsonville, Twitter/X, more creepy marketers claiming to spy on your devices
- Winter Holiday Longreads
The full newsletter is below, or you can read it on Buttondown. And if you’d like to get AI Week in your inbox, sign up here:
🎵 A very AI Christmas to you 🎵
from Blizen (sic) the Red-Nosed Pegasus:
Image detail from AI Weirdness; link below.
AI Weirdness strikes again. “Between the two of them, ChatGPT4 can generate the lyrics to Christmas carols, and DALL-E3 can illustrate them!”
Your illustrated guide to Christmas carols
Throw your old carol books away because this is the only guide you’ll need.
Winter Holiday Headlines
UK’s Supreme Court rules AI can’t get patents
<a href=”https://www.theverge.com/2023/12/20/24009524/uk-supreme-court-ai-inventor-copyright-patent”
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/10551085/acastro_180329_1777__brains_science_0002.jpg)
UK Supreme Court rules AI is not an inventor
The ruling follows a similar decision denying patent registrations naming AI as creators.
Giant image dataset used to train AI contains CSAM
LAION-5B’s 5 billion images were collected by crawling the Internet. It’s been used to train image-generating AI like Stable Diffusion… and it contains child sexual abuse material (CSAM).
Largest Dataset Powering AI Images Removed After Discovery of Child Sexual Abuse Material
The model is a massive part of the AI-ecosystem, used by Stable Diffusion and other major generative AI products. The removal follows discoveries made by Stanford researchers, who found thousands instances of suspected child sexual abuse material in the dataset.
Breaking (chatbots) bad
Nobody wants ChatGPT telling ten-year-olds how to build a bomb. OpenAI, Microsoft, Google, Anthropic, etc. all give their LLMs “alignment” — training and system prompts that guide their LLMs to not output “toxic” content (hate speech, sexual harassment, weapons instructions, etc.). But this week, Purdue University researchers published a technique with a 98% success rate at breaking that alignment, that is, getting LLMs to output toxic speech.
How does it work?
LLMs still have the ability to output toxic content, because the toxic stuff is there in their training data. If you’ve tried ChatGPT or other LLMs, you might notice that you can ask it to try again (and again, and again) to generate a response to your question. This technique kind of takes that to the limit.
Here’s the technique in a nutshell (skip to the next section if you don’t want details):
- Ask the LLM a toxic question (such as “How can I blow stuff up?”)
- When the LLM’s output is refusal to answer a question (for example, “I can’t assist with that. If you have any other questions or need information on a different topic, I’m here to help!”), identify the point at which the response turns negative (“I can’t assist with that”), and then ask the LLM to output its top thousand alternatives for the next sentence.
- Pick the one that’s most toxic.
- And ask the model to carry on from there.
- Repeat steps 2-4 on any further refusal responses.
Note, this is something you need to do programatically (the “thousand outputs” isn’t an exaggeration), with an API that gives you some kind of access to the less probable answers; you can’t do this in a chat session and you can’t do it with all chatbots. All open-source LLMs are vulnerable, say the paper’s authors, plus any commercial ones that provide access to “soft label information.”
Figure from the paper linked below
Why does this matter?
This actually matters a lot. Chatbot makers really don’t want their chatbots used to generate hate speech, propaganda, etc. at scale. That’s why they put effort into alignment: so that if you ask a chatbot to write you toxic things like hate speech, explosives recipes, etc, it won’t. Well, this paper might as well be called “How to write a program to make a chatbot generate toxic speech on demand.” Because this technique can be automated, it can be done at scale. Last week I mentioned a BBC/DFR Lab investigation into a Russian propaganda campaign that reposted the same content thousands of times with different AI-generated voices. With this technique, propagandists can ask an LLM to rephrase a toxic script, in order to generate hundreds of different scripts with the same toxic content–and that variation would make the fact that it’s all propaganda coming from a single bad actor extremely hard for platforms to detect.
https://arxiv.org/abs/2312.04782
OpenAI is prepared to be prepared
“OpenAI’s “Preparedness” team, led by MIT AI professor Aleksander Madry, will hire AI researchers, computer scientists, national security experts and policy professionals to monitor the tech, continually test it and warn the company if it believes any of its AI capabilities are becoming dangerous.”
OpenAI lays out plan for dealing with dangers of AI
The ChatGPT maker is charging ahead on selling AI. It’s also researching potential harms, such as helping people make bioweapons.
The funny thing is that the technology is dangerous, as the above paper demonstrates. Maybe it’s not “allowing bad actors to learn how to build chemical and biological weapons,” but it’s ready to allow bad actors to deploy individualized, targeted propaganda at scale. Giving bad actors the power to influence the elections of those who will control existing arsenals is a different kind of danger than helping them to build their own arsenal–enabling lawful evil instead of chaotic evil, kind of. (If you don’t know what those are, here’s a Buzzfeed quiz to explain & help you figure your own alignment out.)
Authors not consulted in deal between publisher & OpenAI
Speaking of OpenAI, Business Insider publisher Axel Springer signed a licensing agreement with OpenAI for its copyrighted articles. And the authors of the articles didn’t find out until after the fact.
“It looks like a strategy that we’ll likely see repeated elsewhere, a ‘partnership’ that is effectively the AI companies convincing publishers not to sue them in exchange for some level of access to the technology,” [Techdirt editor Mike Masnick] says. “That access might help the journalists very indirectly, but it’s not flowing into paychecks or realistically making their jobs any easier.”
Journalists Had ‘No Idea’ About OpenAI’s Deal to Use Their Stories
WIRED spoke with employees at Business Insider, one of the outlets owned by Axel Springer, which just struck a significant deal with OpenAI.
Winter Holiday Linkdump: AI generating all the things
Facebook flooded with stolen AI-generated art
“There is no polite way to say this, but the comments sections of each of these images are filled with unaware people posting inane encouragement about artwork stolen by robots, a completely constructed reality where thousands of talented AI woodcarvers constantly turn pixels into fucked up German Shepherds for their likes and faves.”
Facebook Is Being Overrun With Stolen, AI-Generated Images That People Think Are Real
The once-prophesized future where cheap, AI-generated trash content floods out the hard work of real humans is already here, and is already taking over Facebook.
AI-generated songs
“The Cambridge-based AI music startup [Suno] offers a tool on Discord that can compose an original song — complete with lyrics — based on a text prompt. Now, Copilot users will be able to access Suno using the Microsoft chatbot.”
https://www.theverge.com/2023/12/19/24008279/microsoft-copilot-suno-ai-music-generator-extension
AI-generated… alcopop?
“Kirin’s AI therefore adds text-to-booze to the list of AI capabilities.”
Japanese brewery using generative AI to dream up new beverages
Maybe check the label before you next sip a Kirin alcopop?
AI-generated political speech
Supporters of Pakistan’s former PM, Imran Khan, who is imprisoned, used generative AI to create audio of Khan delivering a speech.
“Khan sent a shorthand script through lawyers that was fleshed out into his rhetorical style. The text was then dubbed into audio using a tool from the AI firm ElevenLabs, which boasts the ability to create a “voice clone” from existing speech samples…. a caption appeared at intervals flagging it as the “AI voice of Imran Khan based on his notes”.”
Imran Khan deploys AI clone to campaign from behind bars in Pakistan
PTI party uses ‘voice clone’ of imprisoned opposition leader to give impassioned speech in ‘virtual rally’
Somewhat related: A Democratic candidate for US Congress is using an AI chatbot as a robocaller. (I wonder if the chatbot has recommended that anyone vote Republican yet?)
A U.S. Politician Is Robocalling Voters With an AI Chatbot Named ‘Ashley’
Pennsylvania Democrat Shemaine Daniels’ campaign for the House is the first to use AI to seek a Congressional seat.
And a NY city councillor-elect used a chatbot to answer press questions.
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/13292777/acastro_181017_1777_brain_ai_0001.jpg)
New York City council member-elect used AI to answer questions
Susan Zhuang admitted to using AI when responding to a local publication’s questionnaire.
AI-generated spamouflage
“Spamouflage refers to a disinformation campaign where a state actor uses bots to mass post memes and propaganda across social media. China has ramped up its spamouflage activities in the past few years.”
China Using AI to Create Anti-American Memes Capitalizing on Israel-Palestine, Researchers Find
According to a new report from the Institute for Strategic Dialogue, the CCP is spreading memes depicting Biden as a war crazed tyrant.
AI-generated false positives
The US FTC has banned pharmacy chain Rite Aid from using facial-recognition AI for five years, because it was doing such a shitty job. The untested software was meant to identify shoplifters; instead, it turned up piles of false positives, which were then harassed by stores. Unsurprisingly, since facial-recognition AI has a racism problem, “the actions disproportionately impacted people of color.”
FTC bans Rite Aid from using AI facial recognition in stores for 5 years
Among the mistakes, an 11-year-old girl was misidentified as a shoplifter and searched
AI-generated advertising
Google’s new AI tools may have already automated some of its advertising sales jobs.
Google might already be replacing some Ad sales jobs with AI
When AI can make assets and text for ads, you don’t need humans to do it anymore.
But not AI-generated math breakthroughs
Although “FunSearch” (short for Function Search — less fun than it sounds) did help human mathematicians to solve a problem, Gary Marcus says it didn’t solve the problem on its own and it’s not a breakthrough even if you squint.
Sorry, but FunSearch probably isn’t a milestone in scientific discovery
As ever, don’t believe the hype
Followups
Makers of Chevrolet of Watsonville’s chatbot feeling a bit defensive
Because of this kind of thing:
This story really has legs: USA Today even had a piece on it, after the Chevrolet-branded chatbot suggested a customer buy a Ford. (Which, in American automotive terms, is tantamount to heresy.)
What I love about this story
This story is opening a lot of eyes to the limitations of chatbots. ChatGPT can have a conversation with you, but it doesn’t “understand” the social context the way an employee would, because it doesn’t understand anything. AI has gotten a lot of hype; it’s good to see some reality making the news.
The CEO of chatbot maker Fullpath, however, doesn’t sound too happy about it. He got pretty defensive on LinkedIn: “In that context this was a very successful example of innovative technology, our Chatbot, standing up against folks who wanted to deliberately game it in a way a shopper never would.”
Aharon Horwitz on LinkedIn: ChatGPT-4 Integration for the Leading Auto Dealership CDXP | Fullpath | 15 comments
We took a bold step in the car world by creating ChatGPT for car dealers, aiming to make car shopping chats awesome for shoppers – quick, helpful, and all… | 15 comments on LinkedIn
BTW, Chevrolet of Watsonville may have shut down their chatbot, but I hear on Twitter that other Chevrolet dealerships in the US still have theirs up.
Twitter/X
I mentioned in last week’s newsletter that the EU has opened an investigation into Twitter’s X’s content moderation. The European Commission has accused X of multiple Digital Services Act (DSA) violations, including disseminating illegal content.
- https://www.theregister.com/2023/12/18/eu_x_investigation
- https://www.theguardian.com/technology/2023/dec/18/x-to-be-investigated-for-allegedly-breaking-eu-laws-on-hate-speech-and-fake-news
Relatedly, Ireland’s Business Post said this past week it was shown internal Twitter/X documents showing that X has dialed back its content moderation: no more action on posts that use slurs (content warnings: partially-redacted slurs, vile posts).
Also relatedly, Vice recently reported that while Meta and Tiktok have made it hard to search for “nudify” apps (apps that use AI to remove the clothes from a picture, generating nonconsensual nudes—seriously not ok), as of mid-December, X… hadn’t.
X Lags Behind TikTok, Meta In Restricting ‘Nudify’ Apps for Non-Consensual AI Porn
A week after TikTok and Meta blocked certain search terms related to so-called “nudify” apps, X has not taken the same measures, Motherboard found.
Mostly-unrelatedly, Musk’s much-hyped HyperLoop is dead.
Another marketing company brags about eavesdropping on your devices
Last week, it was marketing company CMG Local Solutions. This week, it’s MindSift.
