[Plura-list] Even if you think AI search could be good, it won't be good
Cory Doctorow
doctorow at craphound.com
Wed May 15 12:34:50 EDT 2024
Read today's issue online at: https://pluralistic.net/2024/05/15/they-trust-me-dumb-fucks/
^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^
TONIGHT (May 15), I'm in North Hollywood for a screening of Stephanie Kelton's *Finding the Money*:
https://www.laemmle.com/film/finding-money?date=2024-05-15
Friday (May 17), I'm in San Francisco at the Internet Archive to keynote the tenth anniversary of the Authors Alliance:
https://www.authorsalliance.org/2024/03/15/authors-alliance-10th-anniversary-event-authorship-in-an-age-of-monopoly-and-moral-panics/
^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^
Today's links
* Even if you think AI search could be good, it won't be good: It's just too goddamned easy to cheat.
* Hey look at this: Delights to delectate.
* This day in history: 2004, 2009, 2014, 2019, 2023
* Upcoming appearances: Where to find me.
* Recent appearances: Where I've been.
* Latest books: You keep readin' em, I'll keep writin' 'em.
* Upcoming books: Like I said, I'll keep writin' 'em.
* Colophon: All the rest.
^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^
🐿 Even if you think AI search could be good, it won't be good
The big news in search this week is that Google is continuing its transition to "AI search" - instead of typing in search terms and getting links to websites, you'll ask Google a question and an AI will compose an answer based on things it finds on the web:
https://blog.google/products/search/generative-ai-google-search-may-2024/
Google bills this as "let Google do the googling for you." Rather than searching the web yourself, you'll delegate this task to Google. Hidden in this pitch is a tacit admission that Google is no longer a convenient or reliable way to retrieve information, drowning as it is in AI-generated spam, poorly labeled ads, and SEO garbage:
https://pluralistic.net/2024/05/03/keyword-swarming/#site-reputation-abuse
Googling used to be easy: type in a query, get back a screen of highly relevant results. Today, clicking the top links will take you to sites that paid for placement at the top of the screen (rather than the sites that best match your query). Clicking further down will get you scams, AI slop, or bulk-produced SEO nonsense.
AI-powered search promises to fix this, not by making Google search results better, but by having a bot sort through the search results and discard the nonsense that Google will continue to serve up, and summarize the high quality results.
Now, there are plenty of obvious objections to this plan. For starters, why wouldn't Google just make its search results better? Rather than building a LLM for the sole purpose of sorting through the garbage Google is either paid or tricked into serving up, why not just stop serving up garbage? We know that's possible, because other search engines serve *really good* results by paying for access to Google's back-end and then filtering the results:
https://pluralistic.net/2024/04/04/teach-me-how-to-shruggie/#kagi
Another obvious objection: why would anyone write the web if the only purpose for doing so is to feed a bot that will summarize what you've written without sending anyone to your webpage? Whether you're a commercial publisher hoping to make money from advertising or subscriptions, or - like me - an open access publisher hoping to change people's minds, why would you invite Google to summarize your work without ever showing it to internet users? Nevermind how *unfair* that is, think about how *implausible* it is: if this is the way Google will work in the future, why wouldn't every publisher just *block Google's crawler*?
A third obvious objection: AI is bad. Not morally bad (though maybe morally bad, too!), but *technically* bad. It "hallucinates" nonsense answers, including dangerous nonsense. It's a supremely confident liar that can get you killed:
https://www.theguardian.com/technology/2023/sep/01/mushroom-pickers-urged-to-avoid-foraging-books-on-amazon-that-appear-to-be-written-by-ai
The promises of AI are grossly oversold, including the promises Google makes, like its claim that its AI had discovered millions of useful new materials. In reality, the number of useful new materials Deepmind had discovered was *zero*:
https://pluralistic.net/2024/04/23/maximal-plausibility/#reverse-centaurs
This is true of *all* of AI's most impressive demos. Often, "AI" turns out to be low-waged human workers in a distant call-center pretending to be robots:
https://pluralistic.net/2024/01/31/neural-interface-beta-tester/#tailfins
Sometimes, the AI robot dancing on stage turns out to literally be just a person in a robot suit pretending to be a robot:
https://pluralistic.net/2024/01/29/pay-no-attention/#to-the-little-man-behind-the-curtain
The AI video demos that represent "an existential threat to Hollywood filmmaking" turn out to be so cumbersome as to be practically useless (and vastly inferior to existing production techniques):
https://www.wheresyoured.at/expectations-versus-reality/
But let's take Google at its word. Let's stipulate that:
a) It can't fix search, only add a slop-filtering AI layer on top of it; and
b) The rest of the world will continue to let Google index its pages even if they derive no benefit from doing so; and
c) Google will shortly fix its AI, and all the lies about AI capabilities will be revealed to be premature truths that are finally realized.
AI search is *still* a bad idea. Because beyond all the *obvious* reasons that AI search is a terrible idea, there's a *subtle* - and incurable - defect in this plan: AI search - even excellent AI search - makes it far too easy for Google to cheat us, and Google can't stop cheating us.
Remember: enshittification isn't the result of worse people running tech companies today than in the years when tech services were good and useful. Rather, enshittification is rooted in the collapse of *constraints* that used to prevent those same people from making their services worse in service to increasing their profit margins:
https://pluralistic.net/2024/03/26/glitchbread/#electronic-shelf-tags
These companies always had the *capacity* to siphon value away from business customers (like publishers) and end-users (like searchers). That comes with the territory: digital businesses can alter their "business logic" from instant to instant, and for each user, allowing them to change payouts, prices and ranking. I call this "twiddling": turning the knobs on the system's back-end to make sure the house always wins:
https://pluralistic.net/2023/02/19/twiddler/
What changed wasn't the character of the leaders of these businesses, nor their capacity to cheat us. What changed was the *consequences* for cheating. When the tech companies merged to monopoly, they ceased to fear losing your business to a competitor.
Google's 90% search market share was attained by bribing everyone who operates a service or platform where you might encounter a search box to connect that box to Google. Spending tens of billions of dollars every year to make sure no one ever encounters a non-Google search is a cheaper way to retain your business than making sure Google is the very best search engine:
https://pluralistic.net/2024/02/21/im-feeling-unlucky/#not-up-to-the-task
Competition was once a threat to Google; for years, its mantra was "competition is a click away." Today, competition is all but nonexistent.
Then the surveillance business consolidated into a small number of firms. Two companies dominate the commercial surveillance industry: Google and Meta, and they collude to rig the market:
https://en.wikipedia.org/wiki/Jedi_Blue
That consolidation inevitably leads to regulatory capture: shorn of competitive pressure, the companies that dominate the sector can converge on a single message to policymakers and use their monopoly profits to turn that message into policy:
https://pluralistic.net/2022/06/05/regulatory-capture/
This is why Google doesn't have to worry about privacy laws. They've successfully prevented the passage of a US federal consumer privacy law. The last time the US passed a federal consumer privacy law was in 1988. It's a law that bans video store clerks from telling the newspapers which VHS cassettes you rented:
https://en.wikipedia.org/wiki/Video_Privacy_Protection_Act
In Europe, Google's vast profits lets it fly an Irish flag of convenience, thus taking advantage of Ireland's tolerance for tax evasion *and* violations of European privacy law:
https://pluralistic.net/2023/05/15/finnegans-snooze/#dirty-old-town
Google doesn't fear competition, it doesn't fear regulation, and it also doesn't fear rival technologies. Google and its fellow Big Tech cartel members have expanded IP law to allow it to prevent third parties from reverse-engineer, hacking, or scraping its services. Google doesn't have to worry about ad-blocking, tracker blocking, or scrapers that filter out Google's lucrative, low-quality results:
https://locusmag.com/2020/09/cory-doctorow-ip/
Google doesn't fear competition, it doesn't fear regulation, it doesn't fear rival technology and it doesn't fear its workers. Google's workforce once enjoyed enormous sway over the company's direction, thanks to their scarcity and market power. But Google has outgrown its dependence on its workers, and lays them off in vast numbers, even as it increases its profits and pisses away tens of billions on stock buybacks:
https://pluralistic.net/2023/11/25/moral-injury/#enshittification
Google is fearless. It doesn't fear losing your business, or being punished by regulators, or being mired in guerrilla warfare with rival engineers. It certainly doesn't fear its workers.
Making search worse is good for Google. Reducing search quality increases the number of queries, and thus ads, that each user must make to find their answers:
https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan
If Google can make things worse for searchers without losing their business, it can make more money for itself. Without the discipline of markets, regulators, tech or workers, it has no impediment to transferring value from searchers and publishers to itself.
Which brings me back to AI search. When Google substitutes its own summaries for links to pages, it creates innumerable opportunities to charge publishers for preferential placement in those summaries.
This is true of any algorithmic feed: while such feeds are important - even vital - for making sense of huge amounts of information, they can also be used to play a high-speed shell-game that makes suckers out of the rest of us:
https://pluralistic.net/2024/05/11/for-you/#the-algorithm-tm
When you trust someone to summarize the truth for you, you become terribly vulnerable to their self-serving lies. In an ideal world, these intermediaries would be "fiduciaries," with a solemn (and legally binding) duty to put your interests ahead of their own:
https://pluralistic.net/2024/05/07/treacherous-computing/#rewilding-the-internet
But Google is clear that its first duty is to its shareholders: not to publishers, not to searchers, not to "partners" or employees.
AI search makes cheating *so* easy, and Google cheats *so much*. Indeed, the defects in AI give Google a readymade excuse for any apparent self-dealing: "we didn't tell you a lie because someone paid us to (for example, to recommend a product, or a hotel room, or a political point of view). Sure, they *did* pay us, but that was just an AI 'hallucination.'"
The existence of well-known AI hallucinations creates a zone of plausible deniability for even more enshittification of Google search. As Madeleine Clare Elish writes, AI serves as a "moral crumple zone":
https://estsjournal.org/index.php/ests/article/view/260
That's why, even if you're willing to believe that Google *could* make a great AI-based search, we can nevertheless be certain that they *won't*.
plausible deniability (end of the Farrell essay)
^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^
🐿 Hey look at this
* Sam Bankman-Fried Is Not Entirely Wrong https://prospect.org/health/2024-05-13-sam-bankman-fried-not-wrong-steward-health-bankruptcy/
* My Mother’s Letters Should Belong to Me — Not a Company That Works With Prisons https://truthout.org/articles/my-mothers-letters-should-belong-to-me-not-a-company-that-works-with-prisons/
* Big Tech to EU: "Drop Dead" https://www.eff.org/deeplinks/2024/05/big-tech-eu-drop-dead
^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^
🐿 This day in history
#20yrsago Mayor dispatches cops to bust blogger-critic https://web.archive.org/web/20040605152433/https://www.loiclemeur.com/english/2004/05/a_french_blogge.html
#20yrsago England’s love affair with the utility bill https://web.archive.org/web/20040706124142/https://cede.blogspot.com/2004_05_01_cede_archive.html#108455091554008455
#20yrsago RIAA’s funny bookkeeping turns gains into losses https://web.archive.org/web/20040607052730/http://www.kensei-news.com/bizdev/publish/factoids_us/article_23374.shtml
#20yrsago Read this and understand the P2P wars https://papers.ssrn.com/sol3/papers.cfm?abstract_id=532882
#15yrsago Sarah Palin’s legal team doesn’t understand DNS https://www.huffpost.com/entry/crackhocom-sarah-palins-n_n_202417
#15yrsago Was 1971 the best year to be born a geek? https://www.raphkoster.com/2009/05/14/the-perfect-geek-age/
#15yrsago Charlie Stross on the future of gaming http://www.antipope.org/charlie/blog-static/2009/05/login_2009_keynote_gaming_in_t.html
#15yrsago UK chiropractors try to silence critic with libel claim https://gormano.blogspot.com/2009/05/two-things.html
#15yrsgo The Yggyssey: Pinkwater takes on The Odyssey https://memex.craphound.com/2009/05/15/the-yggyssey-pinkwater-takes-on-the-odyssey/
#15yrsago Sony Pictures CEO: “Nothing good from the Internet, period.” https://wwd.com/feature/memo-pad-uniqlo-nabs-deyn-bad-internet-classic-martha-2136751-1496073/
#10yrsago FCC brings down the gavel on Net Neutrality https://www.eff.org/deeplinks/2014/05/prepare-take-action-defend-net-neutrality-heres-how-fcc-makes-its-rules
#10yrsago IETF declares war on surveillance https://www.rfc-editor.org/rfc/rfc7258.txt
#10yrsago Rob Ford: a night of drunk driving, racism, drugs, beating friends and demeaning his wife https://www.thestar.com/news/insight/rob-ford-one-wild-night-in-march/article_7167c4f4-2a92-5444-b0d1-54d16a6a3f5b.html
#10yrsago Aussie politician calls rival a “c*nt” in Parliament, gets away with it https://www.youtube.com/watch?v=5TsNL3uBw1g
#10yrsago Mozilla CAN change the industry: by adding DRM, they change it for the worse https://www.eff.org/deeplinks/2014/05/mozilla-and-drm
#10yrsago De-obfuscating Big Cable’s numbers: investment flat since 2000 https://www.techdirt.com/2014/05/14/cable-industrys-own-numbers-show-general-decline-investment-over-past-seven-years/
#10yrsago Nude closeups of people who are more than 100 years old https://web.archive.org/web/20140516055234/http://anastasiapottingerphotography.com/gallery/art/centenarians/
#10yrsago Cable lobbyists strong-arm Congresscritters into signing anti-Net Neutrality petition https://web.archive.org/web/20140527030122/http://www.freepress.net/blog/2014/05/12/tell-congress-dont-sign-cable-industry-letter-against-real-net-neutrality
#10yrsago London property bubble examined https://timharford.com/2014/05/when-a-man-is-tired-of-london-house-prices/
#5yrsago A year after Meltdown and Spectre, security researchers are still announcing new serious risks from low-level chip operations https://www.wired.com/story/intel-mds-attack-speculative-execution-buffer/
#5yrsago Jury awards $2b to California couple who say Bayer’s Roundup weedkiller gave them cancer https://www.cnn.com/2019/05/14/business/bayer-roundup-verdict/index.html
#5yrsago AT&T promised it would create 7,000 jobs if Trump went through with its $3B tax-cut, but they cut 23,000 jobs instead https://arstechnica.com/tech-policy/2019/05/att-promised-7000-new-jobs-to-get-tax-break-it-cut-23000-jobs-instead/
#5yrsago DOJ accuses Verizon and AT&T employees of participating in SIM-swap identity theft crimes https://www.vice.com/en/article/d3n3am/att-and-verizon-employees-charged-sim-swapping-criminal-ring
#5yrsago Collecting user data is a competitive disadvantage https://a16z.com/the-empty-promise-of-data-moats/
#5yrsago Three years after the Umbrella Revolution, Hong Kong has its own Extinction Rebellion chapter https://www.scmp.com/news/hong-kong/health-environment/article/3010050/hong-kongs-new-extinction-rebellion-chapter-looks
#5yrsago Lawyer involved in suits against Israel’s most notorious cyber-arms dealer targeted by its weapons, delivered through a terrifying Whatsapp vulnerability https://www.nytimes.com/2019/05/13/technology/nso-group-whatsapp-spying.html
#5yrsago The New York Times on Carl Malamud and his tireless battle to make the law free for all to read https://www.nytimes.com/2019/05/13/us/politics/georgia-official-code-copyright.html
#5yrsago Alex Stamos on the security problems of the platforms’ content moderation, and what to do about them https://memex.craphound.com/2019/05/15/alex-stamos-on-the-security-problems-of-the-platforms-content-moderation-and-what-to-do-about-them/
#5yrsago Axon makes false statements to town that bought its police bodycams, threatens to tase their credit-rating if they cancel the contract https://www.muckrock.com/news/archives/2019/may/09/algorithms-axon-fontana/
#5yrsago After retaliation against Googler Uprising organizers, a company-wide memo warns employees they can be fired for accessing “need to know” data https://www.vice.com/en/article/pajkkz/its-almost-impossible-to-tell-if-iphone-has-been-hacked
#5yrsago Foxconn promised it would do something with the empty buildings it bought in Wisconsin, but they’re still empty (still no factory, either) https://www.theverge.com/2019/5/13/18565408/foxconn-wisconsin-innovation-centers-factories-empty-tax-subsidy
#1yrago Ireland's privacy regulator is a gamekeeper-turned-poacher https://pluralistic.net/2023/05/15/finnegans-snooze/#dirty-old-town
#1yrago Google’s AI Hype Circle https://pluralistic.net/2023/05/14/googles-ai-hype-circle/
Upcoming appearances (permalink)
* "Finding the Money" screening (LA), May 15
https://www.laemmle.com/film/finding-money?date=2024-05-15
* Authorship in an Age of Monopoly and Moral Panics (SF), May 17
https://www.authorsalliance.org/2024/03/15/authors-alliance-10th-anniversary-event-authorship-in-an-age-of-monopoly-and-moral-panics/
* Media Ecology Association keynote (Amherst, NY), Jun 6-9
https://media-ecology.org/convention
* HOPE XV, Jul 14 (Queens, NY)
https://www.hope.net/talks.html
* American Association of Law Libraries keynote (Chicago), Jul 21
https://www.aallnet.org/conference/agenda/keynote-speaker/
* "Finding the Money" screening (LA), May 15
https://www.laemmle.com/film/finding-money?date=2024-05-15
* Authorship in an Age of Monopoly and Moral Panics (SF), May 17
https://www.authorsalliance.org/2024/03/15/authors-alliance-10th-anniversary-event-authorship-in-an-age-of-monopoly-and-moral-panics/
* Media Ecology Association keynote, Jun 6-9 (Amherst, NY)
https://media-ecology.org/convention
* HOPE XV, Jul 14 (Queens, NY)
https://www.hope.net/talks.html
* American Association of Law Libraries keynote, (Chicago), Jul 21
https://www.aallnet.org/conference/agenda/keynote-speaker/
Recent appearances (permalink)
* Suur Futuroloogiline Kongress
https://www.youtube.com/watch?v=6hITj793htg&t=398s
* That Word Chat
https://www.youtube.com/watch?v=miwEy_mACEY
* Come sfuggire al “Merdocene” e costruire un Internet migliore (Torino Biennale Tecnologia)
https://www.youtube.com/watch?v=Z5NC2EZCYBg
Latest books (permalink)
* The Bezzle: a sequel to "Red Team Blues," about prison-tech and other grifts, Tor Books (US), Head of Zeus (UK), February 2024 (the-bezzle.org). Signed, personalized copies at Dark Delicacies (https://www.darkdel.com/store/p3062/Available_Feb_20th%3A_The_Bezzle_HB.html#/).
* "The Lost Cause:" a solarpunk novel of hope in the climate emergency, Tor Books (US), Head of Zeus (UK), November 2023 (http://lost-cause.org). Signed, personalized copies at Dark Delicacies (https://www.darkdel.com/store/p3007/Pre-Order_Signed_Copies%3A_The_Lost_Cause_HB.html#/)
* "The Internet Con": A nonfiction book about interoperability and Big Tech (Verso) September 2023 (http://seizethemeansofcomputation.org). Signed copies at Book Soup (https://www.booksoup.com/book/9781804291245).
* "Red Team Blues": "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books http://redteamblues.com. Signed copies at Dark Delicacies (US): and Forbidden Planet (UK): https://forbiddenplanet.com/385004-red-team-blues-signed-edition-hardcover/.
* "Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin", on how to unrig the markets for creative labor, Beacon Press/Scribe 2022 https://chokepointcapitalism.com
* "Attack Surface": The third Little Brother novel, a standalone technothriller for adults. The *Washington Post* called it "a political cyberthriller, vigorous, bold and savvy about the limits of revolution and resistance." Order signed, personalized copies from Dark Delicacies https://www.darkdel.com/store/p1840/Available_Now%3A_Attack_Surface.html
* "How to Destroy Surveillance Capitalism": an anti-monopoly pamphlet analyzing the true harms of surveillance capitalism and proposing a solution. https://onezero.medium.com/how-to-destroy-surveillance-capitalism-8135e6744d59?sk=f6cd10e54e20a07d4c6d0f3ac011af6b) (signed copies: https://www.darkdel.com/store/p2024/Available_Now%3A__How_to_Destroy_Surveillance_Capitalism.html)
* "Little Brother/Homeland": A reissue omnibus edition with a new introduction by Edward Snowden: https://us.macmillan.com/books/9781250774583; personalized/signed copies here: https://www.darkdel.com/store/p1750/July%3A__Little_Brother_%26_Homeland.html
* "Poesy the Monster Slayer" a picture book about monsters, bedtime, gender, and kicking ass. Order here: https://us.macmillan.com/books/9781626723627. Get a personalized, signed copy here: https://www.darkdel.com/store/p2682/Corey_Doctorow%3A_Poesy_the_Monster_Slayer_HB.html#/.
Upcoming books (permalink)
* Picks and Shovels: a sequel to "Red Team Blues," about the heroic era of the PC, Tor Books, February 2025
* Unauthorized Bread: a graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, 2025
Colophon (permalink)
Today's top sources:
Currently writing:
* A Little Brother short story about DIY insulin PLANNING
* Picks and Shovels, a Martin Hench noir thriller about the heroic era of the PC. FORTHCOMING TOR BOOKS JAN 2025
* Vigilant, Little Brother short story about remote invigilation. FORTHCOMING ON TOR.COM
* Spill, a Little Brother short story about pipeline protests. FORTHCOMING ON TOR.COM
Latest podcast: Precaratize Bosses https://craphound.com/news/2024/04/28/precaratize-bosses/
This work - excluding any serialized fiction - is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.
https://creativecommons.org/licenses/by/4.0/
Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.
How to get Pluralistic:
Blog (no ads, tracking, or data-collection):
Pluralistic.net
Newsletter (no ads, tracking, or data-collection):
https://pluralistic.net/plura-list
Mastodon (no ads, tracking, or data-collection):
https://mamot.fr/@pluralistic
Medium (no ads, paywalled):
https://doctorow.medium.com/
Twitter (mass-scale, unrestricted, third-party surveillance and advertising):
https://twitter.com/doctorow
Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):
https://mostlysignssomeportents.tumblr.com/tagged/pluralistic
"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xBF3D9110957E5F4C.asc
Type: application/pgp-keys
Size: 4820 bytes
Desc: OpenPGP public key
URL: <http://mail.flarn.com/pipermail/plura-list/attachments/20240515/5472c26b/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://mail.flarn.com/pipermail/plura-list/attachments/20240515/5472c26b/attachment.sig>
More information about the Plura-list
mailing list