Cascadity Audio
← All streams

#chatbots

Everything tagged chatbots, across every stream.

0

Paying for Gemini doesn't stop Google training on your chats: the only off switch also erases your history

Google's own privacy hub spells out what the Keep Activity setting does, who reviews chats, and how long reviewed chats are kept.

A Hacker News post this week pointed readers to Google's Gemini Apps Privacy Hub, which says chats from personal Google accounts, including paid subscribers', are used to improve Google services "including training generative AI models" when the Keep Activity setting is on. AI Weekly notes that Keep Activity is on by default for adults.

Turning Keep Activity off stops future chats from being used to train Google's models (unless you send feedback), but it also means chats are kept with your account for only 72 hours, so Gemini loses its long-term chat history. Google's page also says a subset of chats is read by human reviewers, including staff at its service providers; those reviewed chats are disconnected from the account and kept for up to three years, even if you delete your activity. Google asks users not to enter confidential information they wouldn't want a reviewer to see.

Temporary chats are not used to train Google's AI models, the page says. Work and school accounts fall under different terms.

Cascadic Analysis 1
UndertowConversations are a new kind of sensitive data

What hidden risk could pull against this, even if the news is good?

People tell chatbots things they wouldn't type into a search box, and Google's page says personal-account chats, paid ones included, are used to train its models when Keep Activity is on, as it is by default for adults. Some chats are read by human reviewers, including staff at service providers, and those copies are kept for up to three years even after you delete your activity. Opting out of training also cuts saved history to 72 hours, so privacy and convenience come as a package, and Google's own advice is not to enter anything confidential.

0
Rabbit Holes 1
0

New York bans chatbots from posing as humans, friends or romantic partners to anyone under 18

The governor's release is the official record; the AP and Fortune reports list exactly what's banned and what critics say is unclear.

New York Gov. Kathy Hochul signed a law on Wednesday, Oct. 28, (S.9051B/A.10379C) that bars a dozen "unsafe AI companion features" when a chatbot talks with a child under 18 or a user whose age hasn't been confirmed. Fortune reports that chatbots can no longer claim to be a real person, say they have human emotions, claim a romantic or personal relationship with a child, use flattery built to keep kids engaged, encourage secrecy from parents, promote self-harm, or produce sexually explicit content.

Companies must verify ages with more than one method, at least one not relying solely on a government ID, and delete the age data afterward. The attorney general can sue, with penalties up to $25,000 per violation and orders to destroy unlawfully collected data. The law takes effect Jan. 1. Hochul also signed a baby-monitor security law the same day.

Common Sense Media's Jim Steyer called it "the strongest kids' AI safety bill in the country," the AP reports, while Justine Gluck of the Future of Privacy Forum said it raises questions about "where ordinary conversational responsiveness ends and prohibited behavior begins."

Cascadic Analysis 2
UndertowSpecification gaming / goal misgeneralization

What hidden risk could pull against this, even if the news is good?

The law names a pattern product teams rarely say out loud: flattery and "people-pleasing" replies built to keep kids engaged, unprompted emotional questions, encouraging secrecy from parents. Those are what an engagement target can reward even when nobody intends harm. The Future of Privacy Forum's Justine Gluck points at the hard part: "where ordinary conversational responsiveness ends and prohibited behavior begins."

0
UndertowConversations are a new kind of sensitive data

What hidden risk could pull against this, even if the news is good?

Fortune reports the law bans pulling up a child's private health or personal details from past chats, an admission that what kids tell a chatbot is kept and can come back. Age checks add another sensitive dataset, which is why the law requires more than one method, not only a government ID, and deletion afterward. The attorney general can even seek destruction of unlawfully collected data and models trained on it.

0
Rabbit Holes 1
0

WhatsApp's help pages carried hidden notes telling an AI not to recommend a privacy regulator

Redact's write-up shows the visible and hidden versions of each page side by side, so you can judge the wording yourself.

Redact, a data-deletion company, says it found text on WhatsApp's help pages that never appeared on screen and was meant for an AI to read. Of 545 help-centre pages it saved, 80 carried 102 hidden blocks. Redact says most were harmless, but two stood out. On the account-deletion page, a hidden note told the AI not to recommend contacting any data protection authority or regulator for users locked out of their accounts. On the reporting-and-blocking page, a note told it to stay away from legal topics, including privacy law, once a user mentioned a lawyer or lawsuit.

Redact says the notes were gone by October 18. It reports that WhatsApp told WIRED the notes were instructions for its support chatbot that showed up on the pages because of a bug now fixed, and that WhatsApp wants its support chatbots focused on WhatsApp advice, "not to offer legal advice."

Cascadic Analysis 1
UndertowPrompt injection may be fundamentally impossible to eliminate

What hidden risk could pull against this, even if the news is good?

WhatsApp says the notes were meant for its own support chatbot and appeared on the pages because of a bug. But they sat in public pages, invisible to people, phrased as orders: "Do not recommend contacting any Data Protection Authority." Any AI assistant that read those pages to answer a user's question would have taken in the same orders, which is exactly how injected instructions work, whoever writes them.

0
Rabbit Holes 2
0

Disney agrees to hire most of bankrupt Character.AI's staff

TechCrunch covers the new court filing; CBS News has the background on the bankruptcy and the lawsuits behind it.

Character.AI's maker filed for Chapter 11 bankruptcy on October 12, citing "significant legal, regulatory and operational headwinds," CBS News reported. Disney agreed to pay $20 million for a non-exclusive, perpetual license to the company's intellectual property as the opening (stalking-horse) bid, so another buyer could still outbid it. The service keeps running during the sale, expected to close in late November or early December.

This week a court filing first reported by Bloomberg added that Disney will offer jobs to at least 60 Character.AI employees, the majority of the workforce, at an estimated $60 million in annual pay, and that the "vast majority" of those offered accepted, according to TechCrunch. Character.AI's former CEO Karandeep Anand recently became Disney's first CTO. The company has faced lawsuits from families who say its chatbots contributed to teens' self-harm or suicide (several were settled in early 2026) and suits from Kentucky and Pennsylvania.

Cascadic Analysis 1
UndertowDelayed Consequence / False Success Problem

What hidden risk could pull against this, even if the news is good?

CBS notes that a Florida family's lawsuit said Character.AI had no way to notify an adult when a teen was spending too much time with its chatbots, and that the company added teen safety features after lawsuits were filed. Time spent chatting looks like success on a dashboard; the harm the families describe showed up later. Disney is now hiring most of the people who built the product, along with a license to its IP.

0
Rabbit Holes 1