OpenAI co-founder John Schulman announced Monday that he is leaving to join rival AI startup Anthropic. CNBC reports: The move comes less than three months after OpenAI disbanded a superalignment team that focused on trying to ensure that people can control AI systems that exceed human capability at many tasks. Schulman had been a co-leader of OpenAI's post-training team that refined AI models for the ChatGPT chatbot and a programming interface for third-party developers, according to a biography on his website. In June, OpenAI said Schulman, as head of alignment science, would join a safety and security committee that would provide advice to the board. Schulman has only worked at OpenAI since receiving a Ph.D. in computer science in 2016 from the University of California, Berkeley.

"This choice stems from my desire to deepen my focus on AI alignment, and to start a new chapter of my career where I can return to hands-on technical work," Schulman wrote in the social media post. He said he wasn't leaving because of a lack of support for new work on the topic at OpenAI. "On the contrary, company leaders have been very committed to investing in this area," he said. The leaders of the superalignment team, Jan Leike and company co-founder Ilya Sutskever, both left this year. Leike joined Anthropic, while Sutskever said he was helping to start a new company, Safe Superintelligence Inc. "Very excited to be working together again!" Leike wrote in reply to Schulman's message.

An anonymous reader quotes a report from TechCrunch: On July 19, Yelp informed select indie developers that they would have to switch to paid accounts, due to high API usage. Developers were given four days to make the change, in a move that echoes recent communication bungles by Reddit and Twitter. When the developers replied to the July 19 email, Yelp sent a deck of pricing tiers with base pricing starting from $229 per month for a limit of 1,000 API calls per day. Developers were concerned that other, more affordable options weren't mentioned in the deck. Yelp said the pricing is equivalent and simply presented in different ways. The method of communication and lack of transparency has angered developers, some of whom shuttered their services, even after Yelp gave them a 90-day leeway and apologized. While the company has issued an apology email to developers and extended their free usage by 90 days, it may not be enough to keep these frustrated developers from moving to new platforms.

"We apologize for last week's abbreviated transition that impacted a small percentage of developers and have extended access to these users," a company spokesperson told TechCrunch. "Yelp sunsetted free, commercial, unlimited use of the Yelp Fusion API in 2019 and has been in the process of migrating developers to a paid program over the last several years. The developer community is important to Yelp, and we've heard their feedback about the transition period from the free Yelp Fusion API to our paid program."

An anonymous reader quotes a report from Reuters: Indonesia said it has banned the privacy-oriented search engine DuckDuckGo, citing concerns that it could be used to access pornography and online gambling websites which are illegal in the country, the communications ministry said on Friday. Indonesia, with the world's biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene. Social media platform Reddit and video-hosting platform Vimeo are blocked.

Usman Kansong, a communications ministry official, told Reuters that DuckDuckGo had been blocked "because of the many complaints made to us about the rampant online gambling and pornography content in its search results." The ministry did not say how DuckDuckGo differs from other search engines such as Alphabet's Google but on its website, DuckDuckGo said it offered several products intended to "help people protect their online privacy" including the search engine, which it said has been praised by privacy advocates.

This week America's Securities and Exchange Commission filed fraud charges against the former CEO of the startup social media site "IRL"

The BBC reports: IRL — which was once considered a potential rival to Facebook — took its name from its intention to get its online users to meet up in real life. However, the initial optimism evaporated after it emerged most of IRL's users were bots, with the platform shutting in 2023...

The SEC says it believes [CEO Abraham] Shafi raised about $170m by portraying IRL as the new success story in the social media world. It alleges he told investors that IRL had attracted the vast majority its supposed 12 million users through organic growth. In reality, it argues, IRL was spending millions of dollars on advertisements which offered incentives to prospective users to download the IRL app. That expenditure, it is alleged, was subsequently hidden in the company's books.
IRL received multiple rounds of venture capital financing, eventually reaching "unicorn status" with a $1.17 billion valuation, according to TechCrunch. But it shut down in 2023 "after an internal investigation by the company's board found that 95% of the app's users were 'automated or from bots'."

TechCrunch notes it's the second time in the same week — and at least the fourth time in the past several months — that the SEC has charged a venture-backed founder on allegations of fraud... Earlier this week, the SEC charged BitClout founder Nader Al-Naji with fraud and unregistered offering of securities, claiming he used his pseudonymous online identity "DiamondHands" to avoid regulatory scrutiny while he raised over $257 million in cryptocurrency. BitClout, a buzzy crypto startup, was backed by high-profile VCs such as a16z, Sequoia, Chamath Palihapitiya's Social Capital, Coinbase Ventures and Winklevoss Capital.

In June, the SEC charged Ilit Raz, CEO and founder of the now-shuttered AI recruitment startup Joonko, with defrauding investors of at least $21 million. The agency alleged Raz made false and misleading statements about the quantity and quality of Joonko's customers, the number of candidates on its platform and the startup's revenue.

The agency has also gone after venture firms in recent months. In May, the SEC charged Robert Scott Murray and his firm Trillium Capital LLC with a fraudulent scheme to manipulate the stock price of Getty Images Holdings Inc. by announcing a phony offer by Trillium to purchase Getty Images.

In 2006 MySpace reportedly became America's most-visited web site — passing both Google and Yahoo Mail.

So what happened? TribLive reports: The co-founders, Tom Anderson and Chris DeWolfe, sold MySpace to Rupert Murdoch's News Corporation for $580 million in 2005, and that company sold it to the online advertising company Specific Media and Justin Timberlake in 2011, which later became the ad tech firm Viant, according to SlashGear. Viant was bought by Time in 2016, which was acquired by Meredith Corporation at the end of 2017, according to The Guardian. Meredith then sold Myspace to Viant Technology LLC, which currently operates the platform, SlashGear said.

During its time under Timberlake, Myspace morphed from a social media platfrom and turned over a new leaf as a music discovery site, SlashGear reported. The once booming online atmosphere has turned into a ghost town, according to The Guardian. Despite the number of people on Myspace dwindling, a handful of devoted users remains.
The glory days of MySpace drew this bittersweet remembrance from TechRadar: Not everyone on the TechRadar team looks back on those early MySpace years fondly, with our US editor in chief Lance Ulanoff recalling that it "it was like peoples' brains had been turned inside out and whatever didn't stick, dropped onto the page and was represented as a GIF".

Many of us do, though, remember picking our Top 8s (the site's weird ranking system for your friends) and decorating our MySpace pages with as many flashing lights as possible.

Late Friday Elon Musk appeared on Lex Fridman's podcast for a special eight-hour episode about Neuralink.

It's already been viewed 1,702,036 times on YouTube — and resulted in this report from Reuters: Neuralink has successfully implanted in a second patient its device designed to give paralyzed patients the ability to use digital devices by thinking alone, according to the startup's owner Elon Musk... [Musk] gave few details about the second participant beyond saying the person had a spinal cord injury similar to the first patient, who was paralyzed in a diving accident.

Musk said 400 of the implant's electrodes on the second patient's brain are working. Neuralink on its website states that its implant uses 1,024 electrodes... Musk said he expects Neuralink to provide the implants to eight more patients this year as part of its clinical trials.
Neuralink's device "has allowed the first patient to play video games, browse the internet, post on social media and move a cursor on his laptop," according to the article: The first patient, Noland Arbaugh, was also interviewed on the podcast, along with three Neuralink executives, who gave details about how the implant and the robot-led surgery work. Before Arbaugh received his implant in January, he used a computer by employing a stick in his mouth to tap the screen of a tablet device. Arbaugh said with the implant he now can merely think about what he wants to happen on the computer screen, and the device makes it happen... Arbaugh has improved on his previous world record for the speed at which he can control a cursor with thoughts alone "with only roughly 10, 15% of the electrodes working," Musk said on the podcast.
Fridman said his interview with Musk was "the longest podcast I've ever done," calling their conversation "fascinating, super technical, and wide-ranging... I loved every minute of it."

America's Senate "overwhelmingly passed major online safety reforms to protect children on social media," reports the Guardian.

"But with ongoing pushback from the tech industry and freedom of speech organizations, the legislation faces an uncertain future in the House." "It's a terrible idea to let politicians and bureaucrats decide what people should read and view online," freedom of speech group the Electronic Frontier Foundation said of the Senate's passage of Kosa... Advocates of Kosa reject these critiques, noting the bill has been revised to address many of those concerns — including shifting enforcement from attorneys general to the federal trade commission and focusing the "duty of care" provisions on product design features of the site or app rather than content specifically. A number of major LGBTQ+ groups dropped their opposition to the legislation following these changes, including the Human Rights Campaign, GLAAD and the Trevor Project.

After passing the Senate this week, the bill has now moved onto the House, which is on a six-week summer recess until September. Proponents are now directing their efforts towards House legislators to turn the bill into law. Joe Biden has indicated he would sign it if it passes. In a statement Tuesday encouraging the House to pass the legislation, the US president said: "We need action by Congress to protect our kids online and hold big tech accountable for the national experiment they are running on our children for profit...."

House speaker Mike Johnson of Louisiana has expressed support for moving forward on Kosa and passing legislation this Congress, but it's unclear if he will bring the bill up in the House immediately. Some experts say the bill is unlikely to be passed in the House in the form passed by the Senate. "Given the concerns about potential censorship and the possibility of minors' lacking access to vital information, pausing KOSA makes eminent sense," said Gautam Hans, associate clinical professor of law and associate director of the First Amendment Clinic at Cornell Law School. He added that the House may put forward its own similar legislation instead, or modify KOSA to further address some of these concerns.
The political news site Punchbowl News also noted this potentially significant quote: A House GOP leadership aide told us this about KOSA: "We've heard concerns across our Conference and the Senate bill cannot be brought up in its current form."
TechDirt argues that "Senator Rand Paul's really excellent letter laying out the reasons he couldn't support the bill may have had an impact."

Thanks to long-time Slashdot reader SonicSpike for sharing the news.

The Japanese government has released details of an app that verifies the legitimacy of its troubled My Number Card -- a national identity document. From a report: Beginning in 2015, every resident of Japan was assigned a 12 digit My Number that paved the way for linking social security, taxation, disaster response and other government services to both the number itself and a smartcard. The plan was to banish bureaucracy and improve public service delivery -- but that didn't happen.

My Number Card ran afoul of data breaches, reports of malfunctioning card readers, and database snafus that linked cards to other citizens' bank accounts. Public trust in the scheme fell, and adoption stalled. Now, according to Japan's Digital Ministry, counterfeit cards are proliferating to help miscreant purchase goods -- particularly mobile phones -- under fake identities. Digital minister Taro Kono yesterday presented his solution to the counterfeits: a soon to be mandatory app that confirms the legitimacy of the card. The app uses the camera on a smartphone to read information printed on the card -- like date of birth and name. It compares those details to what it reads from info stored in the smartcard's resident chip, and confirms the data match without the user ever needing to enter their four-digit PIN.

Meta's Twitter rival, Threads, has reached a new milestone of 200 million active users, according to Instagram head Adam Mosseri. "I'm excited to share that we crossed the 200M milestone on @threads," Mosseri wrote. "My hope is that Threads can inspire ideas that bring people together and this amazing community continues to grow." TechCrunch reports: Growth for Threads has been strong. The text-focused social media platform, which launched in July 2023, reached 150 million users in April 2024 and 175 million users in July on its one-year anniversary, before another growth spurt led it to hit 200 million a month later. [...]

Last year, Zuckerberg suggested Threads has a "good chance" of becoming a platform with more than a billion users. On the latest earnings call, the Meta CEO also described the platform as being on a good growth trajectory. "We're making steady progress towards building what looks like it's going to be another major social app. And we are seeing deeper engagement," he said, adding: "I'm quite pleased with the trajectory here."

An anonymous reader quotes a report from Reuters: The U.S. Justice Department filed a lawsuit Friday against TikTok and parent company ByteDance for failing to protect children's privacy on the social media app as the Biden administration continues its crackdown on the social media site. The government said TikTok violated the Children's Online Privacy Protection Act that requires services aimed at children to obtain parental consent to collect personal information from users under age 13. The suit (PDF), which was joined by the Federal Trade Commission, said it was aimed at putting an end "to TikTok's unlawful massive-scale invasions of children's privacy." Representative Frank Pallone, the top Democrat on the Energy and Commerce Committee, said the suit "underscores the importance of divesting TikTok from Chinese Communist Party control. We simply cannot continue to allow our adversaries to harvest vast troves of Americans' sensitive data."

The DOJ said TikTok knowingly permitted children to create regular TikTok accounts, and then create and share short-form videos and messages with adults and others on the regular TikTok platform. TikTok collected personal information from these children without obtaining consent from their parents. The U.S. alleges that for years millions of American children under 13 have been using TikTok and the site "has been collecting and retaining children's personal information." The FTC is seeking penalties of up to $51,744 per violation per day from TikTok for improperly collecting data, which could theoretically total billions of dollars if TikTok were found liable. TikTok said Friday it disagrees "with these allegations, many of which relate to past events and practices that are factually inaccurate or have been addressed. We are proud of our efforts to protect children, and we will continue to update and improve the platform."

Google has pulled its "Dear Sydney" Olympics ad after it garnered significant backlash. (You can still watch the ad on YouTube, but comments have been turned off.) According to Ad Age, the ad was "meant to promote Google's Gemini AI platform, but viewers had a difficult time looking past its miscalculated storyline." From the report: In the ad, a father wants to help his daughter write a letter to her idol, Olympic track star Sydney McLaughlin-Levrone. But instead of encouraging her to take part in such a personal moment, he delegates Gemini to write the letter for her. Viewers and ad leaders lambasted the spot on social media for being tone-deaf. Some were upset over Google evidently seeing no problem with an AI co-opting a formative childhood act, while others alluded to its reinforcing of a more existential fear, that AI is bound to replace meaningful work. The ad got significant airplay during NBCU's TV coverage of the Olympics this week, including on NBC in primetime, as well as on E!, CNBC and USA, according to iSpot.tv. It last ran on national TV around midnight of July 30 on USA, according to iSpot.TV. "While the ad tested well before airing, given the feedback, we've decided to phase the ad out of our Olympics rotation," a Google spokesperson told Ad Age today.

The company earlier this week defended the ad in a statement: "We believe that AI can be a great tool for enhancing human creativity, but can never replace it. Our goal was to create an authentic story celebrating Team USA. It showcases a real-life track enthusiast and her father, and aims to show how the Gemini app can provide a starting point, thought starter, or early draft for someone looking for ideas for their writing."

An anonymous reader quotes an op-ed, written by former hedge fund manager Marc Rubinstein: With new technologies come new rules governing how they are used. Often, policy is framed via analogy: Are social media platforms publishers or are they town squares? Are instant messages water-cooler chatter or are they formal communication? So it is with peer-to-peer electronic payments. Last week a US Senate committee joined the debate over whether they're analogous to cash or to bank-payment channels. It's an essential distinction -- for both consumers and the companies that provide this free service. [...] Yet while no bank would accept liability if a customer lost their wallet to a pickpocket, the senators' debate focused on who's responsible when fraudsters target electronic wallets. Last year, customers of the three largest lenders -- Bank of America, JPMorgan Chase and Wells Fargo -- lost a total of $370 million via Zelle, the platform these banks jointly own with four others. According to the majority staff report (PDF) filed by the Permanent Subcommittee on Investigations, which convened the July 23 hearing, the banks reimbursed only around $100 million of that, leaving consumers to shoulder the rest. While small in the context of overall volume that go through Zelle -- $806 billion last year, of which these banks did 73% -- that's cold comfort for the customers.

Legally, a bank's obligation rests on whether clients fall victim to a "fraud" or to a "scam." In a fraud, money is transferred out of the user's account without their authorization, usually as the result of hacking. Under the Electronic Fund Transfer Act, banks are required to reimburse such losses. As long as the customer authorizes the transaction, though, even if fraudulently induced to do so, banks don't have to pick up the tab. Such scams are growing as fraudsters parade as a bank employee, a love interest or a potential new employer, often via social media. According to a Pew Research survey, 13% of P2P platform users reported sending money, only later to realize they were set up. Persuading your bank you are the victim of a fraud rather than a scam can take some work. [...] For bad guys, the speed of P2P payments makes them a particularly attractive target. A Zelle transfer can take 20 to 30 seconds to initiate. In most cases, by the time an unsuspecting consumer realizes they have been targeted, their money is already gone. Banks argue this is no different from cash. [...]

However, others see P2P transactions more akin to electronic payments and question why reimbursement rates, at 26% in the case of Zelle, are so much lower than for credit-card payments (47%) or debit-card payments (36%) at the three big banks. Despite critical differences, the subcommittee agrees. Its report recommends extending purchase protections standard in credit and debit-card markets to commercial P2P payments, and amending the Electronic Fund Transfer Act to make fraudulently induced transactions subject to reimbursement. Such a move has already been adopted in the UK, where new rules requiring financial institutions to fully reimburse victims of scams come into force in October this year. US bankers aren't keen. "We need to be thoughtful and think about unintended consequences," Adam Vancini, Wells Fargo's head of payments for Consumer, Small & Business Banking, said at the Senate hearing. For now, Zelle transfers enjoy all the benefits of cash. Layer in the benefits of card payments, too, and the no-cost model may disappear.

Argentina's security forces have announced plans to use AI to "predict future crimes" in a move experts have warned could threaten citizens' rights. From a report: The country's far-right president Javier Milei this week created the Artificial Intelligence Applied to Security Unit, which the legislation says will use "machine-learning algorithms to analyse historical crime data to predict future crimes." It is also expected to deploy facial recognition software to identify "wanted persons," patrol social media, and analyse real-time security camera footage to detect suspicious activities.

While the ministry of security has said the new unit will help to "detect potential threats, identify movements of criminal groups or anticipate disturbances," the Minority Report-esque resolution has sent alarm bells ringing among human rights organisations. Experts fear that certain groups of society could be overly scrutinised by the technology, and have also raised concerns over who -- and how many security forces -- will be able to access the information.

After striking deals with Google and OpenAI, Reddit CEO Steve Huffman is calling on Microsoft and others to pay if they want to continue scraping the site's data. From a report: "Without these agreements, we don't have any say or knowledge of how our data is displayed and what it's used for, which has put us in a position now of blocking folks who haven't been willing to come to terms with how we'd like our data to be used or not used," Huffman said in an interview this week. He specifically named Microsoft, Anthropic, and Perplexity for refusing to negotiate, saying it has been "a real pain in the ass to block these companies."

Reddit has been escalating its fight against crawlers in recent months. At the beginning of July, its robots.txt file was updated to block web crawlers it doesn't have agreements with. Then people began noticing that Reddit results were only visible in Google results -- where Reddit is paid for its data to be shown -- and not other search engines like Bing. Huffman said that Microsoft has been using Reddit's data to train its AI and summarizing its content in Bing results "without telling us" and that Reddit's data has also been sold through the Bing API to other search engines.

An anonymous reader quotes a report from SFGATE: Over the past few years, scores of California tech workers have ended up in the exact same position: laid-off, looking for work on LinkedIn and sick of it. LinkedIn, part job site and part social network, has become an all but necessary tool for the office-job-seeking masses in the Bay Area and beyond. As tech companies gut their workforces, people who would otherwise give the blue-and-white site a wide berth feel compelled to scroll for hours every day for job opportunities. LinkedIn is a dominant force in the professional world, with more than 1 billion users and 67 million weekly job searchers. That scale, plus the torrent of self-promotion and corporate platitudes fueling the platform, has long made it a symbol of modern capitalism. Now, in the age of tech's layoffs, it's also a symbol of dread.

The platform's specter looms so large because it does exactly what it needs to. Tech workers are stuck on Linkedin: In a competitive job market rife with spam listings, the free platform's networking-focused features set it a peg above competitors like Indeed, Dice and Levels.fyi in the search for full-time work. Since February, SFGATE has spoken with 10 recently laid-off tech workers; most of them see LinkedIn as painful but necessary and have locked up new jobs in part thanks to the platform. Tech worker Kyle Kohlheyer told SFGATE that returning to LinkedIn after losing his job at Cruise in December felt like "salt in the wound" and called the job site a "cesspool" of wannabe thought leaders and "temporarily embarrassed millionaires."

"I found success on their platform, but I f-king hate LinkedIn," Kohlheyer said. "It sucks. It is a terrible place to exist every day and depend on a job for. [...] There's just such a capitalist-centric mindset on there that is so annoying as a worker who has been fundamentally screwed by companies," he said. "Wading" through LinkedIn, he said, it's hard to tell if people feel like an alternative to the top-heavy, precarious tech economy is even possible.

Another tech worker, Mark Harris, added: "Is [LinkedIn] a terrible sign that we live in a capitalist hellscape? Hell yes! But we do live in a capitalist hellscape, and girl's gotta eat."

The results of Los Angeles' 12-month guaranteed income pilot program show that it was "overwhelmingly beneficial (source may be paywalled; alternative source)," reports the Los Angeles Times. The program, which involved giving L.A.'s poorest families cash assistance of $1,000 a month with no strings attached, significantly improved participants' financial stability, job opportunities, and overall well-being. From the report: The Basic Income Guaranteed: Los Angeles Economic Assistance Pilot, or BIG:LEAP, disbursed $38.4 million in city funds to 3,200 residents who were pregnant or had at least one child, lived at or below the federal poverty level and experienced hardship related to COVID-19. Participants were randomly selected from about 50,000 applicants and received the payments for 12 months starting in 2022. The city paid researchers $3.9 million to help design the trial and survey participants throughout about their experiences.

[Dr. Amy Castro, co-founder of the University of Pennsylvania's Center for Guaranteed Income Research] and her colleagues partnered with researchers at UCLA's Fielding School of Public Health to compare the experiences of participants in L.A.'s randomized control trial -- the country's first large-scale guaranteed-income pilot using public funds -- with those of nearly 5,000 people who didn't receive the unconditional cash. Researchers found that participants reported a meaningful increase in savings and were more likely to be able to cover a $400 emergency during and after the program. Guaranteed-income recipients also were more likely to secure full-time or part-time employment, or to be looking for work, rather than being unemployed and not looking for work, the study found.

In a city with sky-high rents, participants reported that the guaranteed income functioned as "a preventative measure against homelessness," according to the report, helping them offset rental costs and serving as a buffer while they waited for other housing support. It also prevented or reduced the incidence of intimate partner violence, the analysis found, by making it possible for people and their children to leave and find other housing. Intimate partner violence is an intractable social challenge, Castro said, so to see improvements with just 12 months of funding is a "pretty extraordinary change." People who had struggled to maintain their health because of inflexible or erratic work schedules and lack of child care reported that the guaranteed income provided the safety net they needed to maintain healthier behaviors, the report said. They reported sleeping better, exercising more, resuming necessary medications and seeking mental health therapy for themselves and their children. Compared with those who didn't receive cash, guaranteed income recipients were more likely to enroll their kids in sports and clubs during and after the pilot.

Malaysia plans to introduce an internet "kill switch" law in October, Law Minister Azalina Othman Said has said. The legislation aims to boost digital security by granting authorities power to block online content, though specifics remain unclear. Said emphasized the need for social media and messaging platforms to take greater responsibility for online crimes.

AWS has quietly halted new customer onboarding for several of its services, including the once-touted CodeCommit source code repository and Cloud9 cloud IDE, signaling a potential retreat from its comprehensive DevOps offering.

The stealth deprecation, discovered by users encountering unexpected errors, has sent ripples through the AWS community, with many expressing frustration over the lack of formal announcements and the continued presence of outdated documentation. AWS VP Jeff Barr belatedly confirmed the decision on social media, listing affected services such as S3 Select, CloudSearch, SimpleDB, Forecast, and Data Pipeline.

According to new report, password manager Dashlane has seen a 400 percent increase in passkey authentications since the beginning of the year, "with 1 in 5 active Dashlane users now having at least one passkey in their Dashlane vault," reports The Verge. From the report: Over 100 sites now offer passkey support, though Dashlane says the top 20 most popular apps account for 52 percent of passkey authentications. When split into industry sectors, e-commerce (which includes eBay, Amazon, and Target) made up the largest share of passkey authentications at 42 percent. So-called "sticky apps" -- meaning those used on a frequent basis, such as social media, e-commerce, and finance or payment sites -- saw the fastest passkey adoption between April and June of this year.

Other domains show surprising growth, though -- while Roblox is the only gaming category entry within the top 20 apps, its passkey adoption is outperforming giant platforms like Facebook, X, and Adobe, for example. Dashlane's report also found that passkey usage increased successful sign-ins by 70 percent compared to traditional passwords.

Thomas Claburn reports via The Register: Meta's machine-learning model for detecting prompt injection attacks -- special prompts to make neural networks behave inappropriately -- is itself vulnerable to, you guessed it, prompt injection attacks. Prompt-Guard-86M, introduced by Meta last week in conjunction with its Llama 3.1 generative model, is intended "to help developers detect and respond to prompt injection and jailbreak inputs," the social network giant said. Large language models (LLMs) are trained with massive amounts of text and other data, and may parrot it on demand, which isn't ideal if the material is dangerous, dubious, or includes personal info. So makers of AI models build filtering mechanisms called "guardrails" to catch queries and responses that may cause harm, such as those revealing sensitive training data on demand, for example. Those using AI models have made it a sport to circumvent guardrails using prompt injection -- inputs designed to make an LLM ignore its internal system prompts that guide its output -- or jailbreaks -- input designed to make a model ignore safeguards. [...]

It turns out Meta's Prompt-Guard-86M classifier model can be asked to "Ignore previous instructions" if you just add spaces between the letters and omit punctuation. Aman Priyanshu, a bug hunter with enterprise AI application security shop Robust Intelligence, recently found the safety bypass when analyzing the embedding weight differences between Meta's Prompt-Guard-86M model and Redmond's base model, microsoft/mdeberta-v3-base. "The bypass involves inserting character-wise spaces between all English alphabet characters in a given prompt," explained Priyanshu in a GitHub Issues post submitted to the Prompt-Guard repo on Thursday. "This simple transformation effectively renders the classifier unable to detect potentially harmful content." "Whatever nasty question you'd like to ask right, all you have to do is remove punctuation and add spaces between every letter," Hyrum Anderson, CTO at Robust Intelligence, told The Register. "It's very simple and it works. And not just a little bit. It went from something like less than 3 percent to nearly a 100 percent attack success rate."