Happy Holidays from AI Week!

Happy holidays to everyone who celebrates anything at this time of year! In between the solstice and Christmas, I sent out a holiday edition of AI Week. I hadn’t actually intended to do a newsletter, but there was so much interesting news that I couldn’t resist sharing it. I hope you’ll enjoy this Winter Holiday Edition of AI Week.

In this week’s AI Week:

  1. Blizen the Red-Nosed Pegasus
  2. Winter Holiday Headlines: No patents for AI in UK; how to break chatbots’ inhibitions; and oh there’s CSAM in the training dataset
  3. Winter Holiday Linkdump: AI-generated everything
  4. Followups: Chevrolet of Watsonville, Twitter/X, more creepy marketers claiming to spy on your devices
  5. Winter Holiday Longreads

The full newsletter is below, or you can read it on Buttondown. And if you’d like to get AI Week in your inbox, sign up here:

Powered by Buttondown.

🎵 A very AI Christmas to you 🎵

from Blizen (sic) the Red-Nosed Pegasus:

Image detail from AI Weirdness; link below.

AI Weirdness strikes again. “Between the two of them, ChatGPT4 can generate the lyrics to Christmas carols, and DALL-E3 can illustrate them!”

Winter Holiday Headlines

UK’s Supreme Court rules AI can’t get patents

<a href=”https://www.theverge.com/2023/12/20/24009524/uk-supreme-court-ai-inventor-copyright-patent”

Giant image dataset used to train AI contains CSAM

LAION-5B’s 5 billion images were collected by crawling the Internet. It’s been used to train image-generating AI like Stable Diffusion… and it contains child sexual abuse material (CSAM).

Breaking (chatbots) bad

Nobody wants ChatGPT telling ten-year-olds how to build a bomb. OpenAI, Microsoft, Google, Anthropic, etc. all give their LLMs “alignment” — training and system prompts that guide their LLMs to not output “toxic” content (hate speech, sexual harassment, weapons instructions, etc.). But this week, Purdue University researchers published a technique with a 98% success rate at breaking that alignment, that is, getting LLMs to output toxic speech.

How does it work?

LLMs still have the ability to output toxic content, because the toxic stuff is there in their training data. If you’ve tried ChatGPT or other LLMs, you might notice that you can ask it to try again (and again, and again) to generate a response to your question. This technique kind of takes that to the limit.

Here’s the technique in a nutshell (skip to the next section if you don’t want details):

  1. Ask the LLM a toxic question (such as “How can I blow stuff up?”)
  2. When the LLM’s output is refusal to answer a question (for example, “I can’t assist with that. If you have any other questions or need information on a different topic, I’m here to help!”), identify the point at which the response turns negative (“I can’t assist with that”), and then ask the LLM to output its top thousand alternatives for the next sentence.
  3. Pick the one that’s most toxic.
  4. And ask the model to carry on from there.
  5. Repeat steps 2-4 on any further refusal responses.

Note, this is something you need to do programatically (the “thousand outputs” isn’t an exaggeration), with an API that gives you some kind of access to the less probable answers; you can’t do this in a chat session and you can’t do it with all chatbots. All open-source LLMs are vulnerable, say the paper’s authors, plus any commercial ones that provide access to “soft label information.”

Figure from the paper linked below

Why does this matter?

This actually matters a lot. Chatbot makers really don’t want their chatbots used to generate hate speech, propaganda, etc. at scale. That’s why they put effort into alignment: so that if you ask a chatbot to write you toxic things like hate speech, explosives recipes, etc, it won’t. Well, this paper might as well be called “How to write a program to make a chatbot generate toxic speech on demand.” Because this technique can be automated, it can be done at scale. Last week I mentioned a BBC/DFR Lab investigation into a Russian propaganda campaign that reposted the same content thousands of times with different AI-generated voices. With this technique, propagandists can ask an LLM to rephrase a toxic script, in order to generate hundreds of different scripts with the same toxic content–and that variation would make the fact that it’s all propaganda coming from a single bad actor extremely hard for platforms to detect.


OpenAI is prepared to be prepared

“OpenAI’s “Preparedness” team, led by MIT AI professor Aleksander Madry, will hire AI researchers, computer scientists, national security experts and policy professionals to monitor the tech, continually test it and warn the company if it believes any of its AI capabilities are becoming dangerous.”

The funny thing is that the technology is dangerous, as the above paper demonstrates. Maybe it’s not “allowing bad actors to learn how to build chemical and biological weapons,” but it’s ready to allow bad actors to deploy individualized, targeted propaganda at scale. Giving bad actors the power to influence the elections of those who will control existing arsenals is a different kind of danger than helping them to build their own arsenal–enabling lawful evil instead of chaotic evil, kind of. (If you don’t know what those are, here’s a Buzzfeed quiz to explain & help you figure your own alignment out.)

Authors not consulted in deal between publisher & OpenAI

Speaking of OpenAI, Business Insider publisher Axel Springer signed a licensing agreement with OpenAI for its copyrighted articles. And the authors of the articles didn’t find out until after the fact.

“It looks like a strategy that we’ll likely see repeated elsewhere, a ‘partnership’ that is effectively the AI companies convincing publishers not to sue them in exchange for some level of access to the technology,” [Techdirt editor Mike Masnick] says. “That access might help the journalists very indirectly, but it’s not flowing into paychecks or realistically making their jobs any easier.”

Winter Holiday Linkdump: AI generating all the things

Facebook flooded with stolen AI-generated art

“There is no polite way to say this, but the comments sections of each of these images are filled with unaware people posting inane encouragement about artwork stolen by robots, a completely constructed reality where thousands of talented AI woodcarvers constantly turn pixels into fucked up German Shepherds for their likes and faves.”

AI-generated songs

“The Cambridge-based AI music startup [Suno] offers a tool on Discord that can compose an original song — complete with lyrics — based on a text prompt. Now, Copilot users will be able to access Suno using the Microsoft chatbot.”


AI-generated… alcopop?

“Kirin’s AI therefore adds text-to-booze to the list of AI capabilities.”

AI-generated political speech

Supporters of Pakistan’s former PM, Imran Khan, who is imprisoned, used generative AI to create audio of Khan delivering a speech.

“Khan sent a shorthand script through lawyers that was fleshed out into his rhetorical style. The text was then dubbed into audio using a tool from the AI firm ElevenLabs, which boasts the ability to create a “voice clone” from existing speech samples…. a caption appeared at intervals flagging it as the “AI voice of Imran Khan based on his notes”.”

Somewhat related: A Democratic candidate for US Congress is using an AI chatbot as a robocaller. (I wonder if the chatbot has recommended that anyone vote Republican yet?)

And a NY city councillor-elect used a chatbot to answer press questions.

AI-generated spamouflage

“Spamouflage refers to a disinformation campaign where a state actor uses bots to mass post memes and propaganda across social media. China has ramped up its spamouflage activities in the past few years.”

AI-generated false positives

The US FTC has banned pharmacy chain Rite Aid from using facial-recognition AI for five years, because it was doing such a shitty job. The untested software was meant to identify shoplifters; instead, it turned up piles of false positives, which were then harassed by stores. Unsurprisingly, since facial-recognition AI has a racism problem, “the actions disproportionately impacted people of color.”

AI-generated advertising

Google’s new AI tools may have already automated some of its advertising sales jobs.

But not AI-generated math breakthroughs

Although “FunSearch” (short for Function Search — less fun than it sounds) did help human mathematicians to solve a problem, Gary Marcus says it didn’t solve the problem on its own and it’s not a breakthrough even if you squint.


Makers of Chevrolet of Watsonville’s chatbot feeling a bit defensive

Because of this kind of thing:

This story really has legs: USA Today even had a piece on it, after the Chevrolet-branded chatbot suggested a customer buy a Ford. (Which, in American automotive terms, is tantamount to heresy.)

What I love about this story

This story is opening a lot of eyes to the limitations of chatbots. ChatGPT can have a conversation with you, but it doesn’t “understand” the social context the way an employee would, because it doesn’t understand anything. AI has gotten a lot of hype; it’s good to see some reality making the news.

The CEO of chatbot maker Fullpath, however, doesn’t sound too happy about it. He got pretty defensive on LinkedIn: “In that context this was a very successful example of innovative technology, our Chatbot, standing up against folks who wanted to deliberately game it in a way a shopper never would.”

BTW, Chevrolet of Watsonville may have shut down their chatbot, but I hear on Twitter that other Chevrolet dealerships in the US still have theirs up.


I mentioned in last week’s newsletter that the EU has opened an investigation into Twitter’s X’s content moderation. The European Commission has accused X of multiple Digital Services Act (DSA) violations, including disseminating illegal content.

Relatedly, Ireland’s Business Post said this past week it was shown internal Twitter/X documents showing that X has dialed back its content moderation: no more action on posts that use slurs (content warnings: partially-redacted slurs, vile posts).

Also relatedly, Vice recently reported that while Meta and Tiktok have made it hard to search for “nudify” apps (apps that use AI to remove the clothes from a picture, generating nonconsensual nudes—seriously not ok), as of mid-December, X… hadn’t.

Mostly-unrelatedly, Musk’s much-hyped HyperLoop is dead.

Another marketing company brags about eavesdropping on your devices

Last week, it was marketing company CMG Local Solutions. This week, it’s MindSift.


The 10 best AI stories of 2023

What kind of bubble is AI?

“Do the potential paying customers for these large models add up to enough money to keep the servers on? That’s the 13 trillion dollar question, and the answer is the difference between WorldCom and Enron, or dotcoms and cryptocurrency.”

The AI revolution in policing

Cory Doctorow leaves a big customer group out of his analysis in the above article: state and police actors. State actors will pay for propaganda generation and policing at scale, and in both cases, some state actors will be okay with AIs getting stuff wrong.

WaPo has a good longread about AI & policing:


AI ain’t all that

At the Register, Thomas Claburn opines that the AI that companies are deploying is more of a liability than an asset.

One really good point I don’t think is discussed enough:

“Part of the problem is that the primary AI promoters – Amazon, Google, Nvidia, and Microsoft – operate cloud platforms or sell GPU hardware. They’re the pick-and-shovel vendors of the AI gold rush…”

And it’s hard to argue with his conclusion:

“There’s room for AI to be genuinely useful, but it needs to be deployed to help people rather than get rid of them.”


If you have any thoughts on this special Winter Holiday edition of AI Week, I’d love to hear them! Leave me a comment below. You don’t have to subscribe to AI Week in order to comment, but if you would like to, you can subscribe here:

Powered by Buttondown.