[Plura-list] Descartes' God has failed and Thompson's Satan rules our computers

[Cory Doctorow]

Today's links

* Descartes' God has failed and Thompson's Satan rules our computers:
Computers, trust, and the knowability of the universe.

* This day in history: 2005, 2010, 2019

* Colophon: Recent publications, upcoming appearances, current writing
projects, current reading

_,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,_

👒 Descartes' God has failed and Thompson's Satan rules our computers

Security researchers are alarmed: the already-notorious Trickbot malware
has been spottied probing infected computers to find out which version
of UEFI they're running. This is read as evidence that Trickbot has
figured out how to pull off a really scary feat.

To understand why, you have to understand UEFI: a fascinating, deep,
philosophical change to our view of computers, trust, and the
knowability of the universe. It's a tale of hard choices, paternalism,
and the race to secure the digital realm as it merges with the physical.

Computers were once standalone: a central processing unit that might be
augmented by some co-processors for specialized processes, like a
graphics card or even a math co-processor.

These co-pros were subordinate to the CPU though. You'd turn on the
computer and it would read a very small set of hardcoded instructions
telling it how to access a floppy disk or other storage medium for the
rest of the boot sequence, the stuff needed to boot the system.

The hardwired instructions were in a ROM that had one job: wake up and
feed some instructions to the "computer" telling it what to do, then go
back to sleep. But there's a philosophical conundrum here.

Because the world of computing is adversarial and networked computing is
doubly so: there are people who want your computer to do things that are
antithetical to your interests, like steal your data or spy on you or
encrypt all your files and demand ransom.

To stop this, you need to be able to examine the programs running on
your computer and terminate the malicious ones. And therein lies the
rub: when you instruct your computer to examine its own workings, how do
you know if you can trust it?

In 1983, Ken Thompson (co-creator of C, Unix, etc) was awarded a Turing
Award ("computer science's Nobel Prize"). He gave a fucking bombshell of
an acceptance speech, called "Reflections on Trusting Trust."

https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf

Thompson revealed that he had created a backdoor for himself that didn't
just live in Unix, but in the C compiler that people made to create new
Unix systems.

Here's what that means: when you write a program, you produce
"high-level code" with instructions like "printf("Hello, World!");".
Once your program is done, you turn it into machine code, a series of
much shorter instructions that your CPU understands ("mov  dx, msg" etc).

Most programmers can't read this machine code, and even for those who
can, it's a hard slog. In general, we write our code, compile it and run
it, but we don't examine it. With nontrivial programs, looking at the
machine code is very, very hard.

Compilers are treated as intrinsically trustworthy. Give 'em some
source, they spit out a binary, you run the binary. Sometimes there are
compiler bugs, sure, and compiler improvements can be a big deal. But
compilers are infrastructure: inscrutable and forgotten.

Here's what Thompson did: he hid a program in his compiler that would
check to see whether you were compiling an operating system or a
compiler. If you were compiling an OS, it hid a secret login for him
inside of it.

If you were compiling a compiler, it hid the program that looked for
compilers or operating systems inside of it.

Think about what this means: every OS you compiled had an intentional
security defect that the OS itself couldn't detect.

If you suspected that your compiler was up to no good and wrote your own
compiler, it would be compromised as soon as you compiled it. What
Thompson did was ask us to contemplate what we meant when we "trusted"
something.

It was a move straight out of Rene Descartes, the reasoning that leads
up to "I think therefore I am." Descartes' "Discourse on the Method"
asks how we can know things about the universe.

He points out that sometimes he thinks he senses something but is wrong
- he dreams, he hallucinates, he misapprehends.

If all our reasoning depends on the impressions we get from our senses,
and if our senses are sometimes faulty, how can we reason at all?

Descartes wants a point of certainty, one thing he *knows* to be
absolutely true. He makes the case that if you can be certain of one
thing, you can anchor everything else to this point and build up a
massive edifice of trustable knowledge that all hangs off of this anchor.

Thompson is basically saying, "You thought you had descartesed your way
into a trustable computing universe because of the axiom that I would
never poison your lowest-level, most fundamental tools.

"*Wrong*.

"Bwahahahaha."

(But, you know, in a nice way: an object lesson to serve as a wake-up
call before computers fully merged with the physical world to form a
global, species-wide digital nervous system whose untrustworthy
low-level parts were foolishly, implicitly trusted).

But processors were expensive and computers were exploding. PCs running
consumer operating systems like Windows and Mac OS (and more exotic ones
like GNU/Linux and various Unices) proliferated, and they all shared
this flawed security model.

They all relied on the operating system to be a faithful reporter of the
computer's internals, and operated on the assumption that they could use
programs supervised by the OS to detect and terminate malicious programs.

But starting in 1999, Ken Thompson's revenge was visited upon the
computing world. Greg Hoglund released Ntrootkit, a proof-of-concept
malware that attacked Windows itself, so that the operating system would
lie to antivirus programs about what it was doing and seeing.

In Decartesspeak, your computer could no longer trust its senses, so it
could no longer reason. The nub of trust, the piton driven into the
mountainface, was made insecure and the whole thing collapsed. Security
researchers at big companies like Microsoft took this to heart.

In 2002, Peter Biddle and his team from Microsoft came to EFF to show us
a new model for computing: "Trusted Computing" (codenamed "Palladium").

https://web.archive.org/web/20020805211111/https://www.microsoft.com/presspass/features/2002/jul02/0724palladiumwp.asp

Palladium proposed to give computers back their nub of Descartesian
certainty. It would use a co-processor, but unlike a graphics card or a
math co-pro, it would run before the CPU woke up and did its thing.

And unlike a ROM, it wouldn't just load up the boot sequence and go back
to sleep.

This chip - today called a "Secure Enclave" or a "Trusted Platform
Module" (etc) - would have real computing power, and it would remain
available to the CPU at all times.

Inside the chip was a bunch of cool cryptographic stuff that provided
the nub of certainty. At the start of the boot, the TPM would pull the
first stages of the boot-code off of the drive, along with a
cryptographic signature.

A quick crypto aside:

Crypto is code that mixes a key (a secret known to the user) with text
to produce a scrambled text (a "ciphertext") that can only be
descrambled by the key.

Dual-key crypto has two keys. What one scrambles, the other descrambles
(and vice-versa).

With dual-key crypto, you keep one key secret (the "private key") and
you publish the other one (the "public key"). If you scramble something
with a private key, then anyone can descramble it with your public key
and know it came from you.

If you scramble it *twice*, first with your private key and then with
your friend's public key, then they can tell it came from you (because
only your private key's ciphertexts can be descrambled with your public
key).

And *you* can be certain that only they can read it (because only their
private key can descramble messages that were scrambled with their
public key).

Code-signing uses dual-key crypto to validate who published some code.

Microsoft can make a shorter version of its code (like a fingerprint)
and then you scramble it with its private key. The OS that came with
your computer has a copy of MSFT's public key. When you get an OS
update, you can descramble the fingerprint with that built-in key.

If it matches the update, then you know that Microsoft signed it and it
hasn't been tampered with on its way to you. If you trust Microsoft, you
can run the update.

But...What if a virus replaces Microsoft's public keys with its own?

That's where Palladium's TPM comes in. It's got the keys hardcoded into
it. Programs running on the CPU can only ask the TPM to do very limited
things like ask it to sign some text, or to check the signature on some
text.

It's a kind of god-chip, running below the most privileged level of
user-accessible operations. By design, you - the owner of the computer -
can demand things of it that it is technically capable of doing, and it
can refuse you, and you can't override it.

That way, programs running even in the most privileged mode can't
compromise it.

Back to our boot sequence: the TPM fetches some startup code from the
disk along with a signature, and checks to see whether the OS has been
signed by its manufacturer.

If not, it halts and shows you a scary error message. Game over, Ken
Thompson!

It is a very cool idea, but it's also very scary, because the chip
doesn't take orders from Descartes' omnibenevolent God.

It takes orders from Microsoft, a rapacious monopolist with a history of
complicity with human rights abuses. Right from that very first meeting
the brilliant EFF technologist Seth Schoen spotted this (and made the
Descartes comparison):

https://web.archive.org/web/20021004125515/http://vitanuova.loyalty.org/2002-07-05.html

Seth identified a way of having your cake and eating it too: he proposed
a hypothetical thing called an "owner override" - a physical switch
that, when depressed, could be used to change which public keys lived in
the chip.

This would allow owners of computers to decide who they trusted and
would defend them against malware. But what it *wouldn't* do is defend
tech companies shareholders against the owner of the computer - it
wouldn't facilitate DRM.

"Owner override" is a litmus test: are you Descartes' God, or Thompson's
Satan?

Do you want computers to allow their owners to know the truth? Or do you
want computers to bluepill their owners, lock them in a matrix where you
get to decide what is true?

A month later, I published a multi-award-winning sf story called
"0wnz0red" in Salon that tried to dramatize the stakes here.

https://www.salon.com/2002/08/28/0wnz0red/

Despite Seth's technical clarity and my attempts at dramatization, owner
override did not get incorporated into trusted computing architectures.

Trusted computing took years to become commonplace in PCs. In the
interim, rootkits proliferated. Three years after the Palladium paper,
Sony-BMG deliberately turned 6m audio CDs into rootkit vectors that
would silently alter your OS when you played them from a CD drive.

The Sony rootkit broke your OS so that any filename starting with $SYS$
didn't show up in file listings, $SYS$ programs wouldn't show up in the
process monitor. Accompanying the rootkit was a startup program
(starting with $SYS$) that broke CD ripping.

Sony infected hundreds of thousands of US gov and mil networks. Malware
authors - naturally enough - added $SYS$ to the files corresponding with
their viruses, so that antivirus software (which depends on the OS for
information about files and processes) couldn't detect it.

It was an incredibly reckless, depraved act, and it wasn't the last.
Criminals, spies and corporations continued to produce rootkits to
attack their adversaries (victims, rival states, customers) and trusted
computing came to the rescue.

Today, trusted computing is widely used by the world's largest tech
companies to force customers to use their app stores, their OSes, their
printer ink, their spare parts. It's in medical implants, cars, tractors
and kitchen appliances.

None of this stuff has an owner override. In 2012, I gave a talk to
Google, Defcon and the Long Now Foundation about the crisis of owner
override, called "The Coming Civil War Over General Purpose Computing."

https://memex.craphound.com/2012/08/23/the-coming-civil-war-over-general-purpose-computing/

It proposed a way that owner override, combined with trusted computing,
could allow users to resist both state and corporate power, and it
warned that a lack of technological self-determination opened the door
to a parade of horribles.

Because once you have a system that is designed to override owners - and
not the other way around - then anyone who commands that system can, by
design, do things that the user can't discern or prevent.

This is the *real* trolley problem when it comes to autonomous vehicles:
not "who should a car sacrifice in a dangerous situation?" but rather,
"what happens when a car that is designed to sometimes kill its owner is
compromised by Bad Guys?"

https://this.deakin.edu.au/self-improvement/car-wars

The thing is, trusted computing with an owner override is pretty
magical. Take the Introspection Engine, a co-processor in a fancy Iphone
case designed by Edward Snowden and Bunnie Huang. It's designed to catch
otherwise undetectable mobile malware.

https://www.tjoe.org/pub/direct-radio-introspection/release/2

You see, your phone doesn't just run Ios or Android; the part that
interfaces with the phone system - be baseband radio - runs an ancient,
horribly insecure OS, and if it is infected, it can trick your phone's
senses, so that it can no longer reason.

The Introspection Engine is a small circuit board that sandwiches
between your phone's mainboard and its case, making electrical contact
with all the systems that carry network traffic.

This daughterboard has a ribbon cable that snakes out of the SIM slot
and into a slightly chunky phone case that has a little open source
hardward chip with fully auditable code and an OLED display.

This second computer monitors the electrical signals traveling on the
phone's network buses and tells you what's going on. This is a
user-accessible god-chip, a way for you to know whether your phone is
hallucinating when it tells you that it isn't leaking your data.

That's why it's called an "Introspection Engine." It lets your phone
perch at an objective remove and understand how it is thinking.

(If all this sounds familiar, it's because it plays a major role in
ATTACK SURFACE, the third Little Brother book)

https://attacksurface.com

The reason the Introspection Engine is so exciting is that it is
exceptional. The standard model for trusted computing is that it treats
everyone *except* the manufacturer as its adversary - including you, the
owner of the device.

This opens up many different sets of risks, all of which have been
obvious since 1999's Ntrootkit, and undeniable since 2005's Sony Rootkit.

I. The manufacturer might not have your interests at heart.

In 2016, HP shipped a fake security update to its printers, tricking
users into installing a system that rejected their third-party ink,
forcing them to pay monopoly prices for HP products.

II. An insider at the company may not have your interests at heart.

Multiple "insider threat" attacks have been executed against users.
Employees at AT&T;, T-Mobile, even Roblox have accepted bribes to attack
users on behalf of criminals.

https://www.vice.com/en/article/qj4ddw/hacker-bribed-roblox-insider-accessed-user-data-reset-passwords

III. A government may order the company to attack its users.

In 2017 Apple removed all working VPNs from its Chinese app stores, as
part of the Chinese state's mass surveillance program (1m members of
religious minorities were subsequently sent to concentration camps).

Apple's trusted computing prevents users from loading apps that aren't
in its app stores, meaning that Apple's decisions about which apps you
can run on your Iphone are binding on you, even if you disagree.

https://www.reuters.com/article/us-china-apple-vpn/apple-says-it-is-removing-vpn-services-from-china-app-store-idUSKBN1AE0BQ

IV. Third parties may exploit a defect in the trusted computing system
and attack users in undetectable ways that users can't prevent.

By design, TPMs can't be field updated, so if there's a defect in them,
it can't be patched.

Checkm8 exploits a defect in eight generations Apple's mobile TPM. It's
a proof-of-concept released to demonstrate a vulnerability, not malware
(thankfully).

https://checkm8.info/

But there have been scattered, frightening instances of malware that
attacks the TPM - that suborns the mind of God so that your computer
ceases to be able to reason. To date, these have all been associated
with state actors who used them surgicially.

State actors know that the efficacy of their cyberweapons is tied to
secrecy: once a rival government knows that a system is vulnerable,
they'll fix it or stop using it or put it behind a firewall, so these
tools are typically used parsimoniously.

But criminals are a different matter (and now, at long last, we're
coming back to Trickbot and UEFI) (thanks for hanging in there).

UEFI ("You-Eff-Ee") is a trusted computing that computer manufacturers
use to prevent unauthorized OSes from running on the PCs they sell you.

Mostly, they use this to prevent malicious OSes from running on the
hardware they manufacture, but there have been scattered instances of it
being used for monopolistic purposes: to prevent you from replacing
their OS with another one (usually a flavor of GNU/Linux).

UEFI is god-mode for your computer, and a compromise to it would be a
Sony Rootkit event, but 15 years later, in a world where systems are
more widespread and used for more critical applications from driving
power-plants to handling multimillion-dollar transactions.

Trickbot is very sophisticated malware generally believed to be run by
criminals, not a government. Like a lot of modern malware, there's a
mechanism for updating it in the field with new capabilities - both
attacks and defenses.

And Trickbot has been observed in the wild probing infected systems'
UEFI. This leads security researchers to believe that Trickbot's authors
have figured out how to compromise UEFI on some systems.

https://www.wired.com/story/trickbot-botnet-uefi-firmware/

Now, no one has actually observed UEFI being compromised, nor has anyone
captured any UEFI-compromising Trickbot code. The thinking goes that
Trickbot only downloads the UEFI code when it finds a vulnerable system.

Running in UEFI would make Trickbot largely undetectable and
undeletable. Even wiping and restoring the OS wouldn't do it. Remember,
TPMs are designed to be unpatchable and tamper-resistant. The physical
hardware is designed to break forever if you try to swap it out.

If this is indeed what's going on, it's the first instance in which a
trusted computing module was used to attack users by criminals (not
governments or the manufacturer and its insiders). And Trickbot's owners
are really bad people.

They've hired out to the North Korean state to steal from
multinationals; they've installed ransomware in big companies, and while
their footprint has waned, they once controlled 1,000,000 infected systems.

You can check your UEFI to see if it's vulnerable to tampering:

https://eclypsium.com/2019/10/23/protecting-system-firmware-storage/

and also determine whether it has been compromised:

https://eclypsium.com/2020/10/14/protecting-your-organizations-from-mosaicregressor-and-other-uefi-implants/

But this isn't the end, it's just getting started. As Seth Schoen warned
us in 2002, the paternalistic mode of computing has a huge, Ken
Thompson-shaped hole in it: it requires you trust the benevolence of a
manufacturer, and, crucially, they know you don't have a choice.

If companies knew that you *could* alter whom you trusted, they would
have to work to earn and keep your trust. If governments knew that
ordering a company to compromise on TPMs, they'd understand that their
targets would simply shift tactics if they made that order.

Some users would make foolish decisions about whom to trust, but they
would also have recourse when a trusted system was revealed to be
defective. This is a fight that's into its third decade, and the stakes
have never been higher.

Sadly, we are no closer to owner override than we were in 2002.

_,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,_

👒 This day in history

#15yrago Sony *finally* releases rookit uninstaller — sort of
https://web.archive.org/web/20051204015131/http://cp.sonybmg.com/xcp/english/updates.html

#10yrsago UK govt demands an end to evidence-based drug policy
https://www.theguardian.com/politics/2010/dec/05/government-scientific-advice-drugs-policy?

#1yrago Prasad’s Law: there’s always enough health spending to
concentrate wealth, never enough to diffuse it
https://memex.craphound.com/2019/12/06/prasads-law-theres-always-enough-health-spending-to-concentrate-wealth-never-enough-to-diffuse-it/

#1yrago The retreat of “scientific selfishness,” a literature review
https://www.the-tls.co.uk/articles/greed-is-dead/

#1yrago Radicalized is one of the CBC’s best books of 2019!
https://www.cbc.ca/books/the-best-canadian-fiction-of-2019-1.5382741

#1yrago German researchers with ties to for-profit “neuromarketing”
company want to use AI to guess peoples’ “intelligence” from their
writing
https://memex.craphound.com/2019/12/06/german-researchers-with-ties-to-for-profit-neuromarketing-company-want-to-use-ai-to-guess-peoples-intelligence-from-their-writing/

_,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,_

👒 Colophon

Today's top sources: Naked Capitalism (https://www.nakedcapitalism.com/).

Currently writing: My next novel, "The Lost Cause," a post-GND novel
about truth and reconciliation. Yesterday's progress: 515 words (86836
total).

Currently reading: The City We Became, NK Jemisin

Latest podcast: Someone Comes to Town, Someone Leaves Town (part 24)
https://craphound.com/podcast/2020/11/23/someone-comes-to-town-someone-leaves-town-part-24/

Upcoming appearances:

*  Monopoly, Not Mind Control: What's Really Happening With
"Surveillance Capitalism," Dec 8,
https://www.nuug.no/aktiviteter/20201208-doctorow/

* Colloquium on Information Security, Dec 14
https://www.eventbrite.co.uk/e/the-31st-hphpe-virtual-colloquium-on-information-security-tickets-128859336745

* Keynote, NISO Plus, Feb 22-25,
https://niso.plus/cory-doctorow-to-keynote-at-niso-plus-2021/

Recent appearances:

* A More Competitive Web (Techdirt Podcast):
https://www.techdirt.com/articles/20201201/10183045801/techdirt-podcast-episode-264-more-competitive-web-with-cory-doctorow-daphne-keller.shtml

* Big Tech Podcast:
https://www.cigionline.org/big-tech/cory-doctorow-true-dangers-surveillance-capitalism

* Nerdcanon Podcast:
http://nerdcanon.com/episode-25-cory-doctorow-and-attack-surface/

Latest book:

* "Attack Surface": The third Little Brother novel, a standalone
technothriller for adults. The *Washington Post* called it "a political
cyberthriller, vigorous, bold and savvy about the limits of revolution
and resistance." Order signed, personalized copies from Dark Delicacies
https://www.darkdel.com/store/p1840/Available_Now%3A_Attack_Surface.html

* "How to Destroy Surveillance Capitalism": an anti-monopoly pamphlet
analyzing the true harms of surveillance capitalism and proposing a
solution.
https://onezero.medium.com/how-to-destroy-surveillance-capitalism-8135e6744d59

* "Little Brother/Homeland": A reissue omnibus edition with a new
introduction by Edward Snowden:
https://us.macmillan.com/books/9781250774583; personalized/signed copies
here:
https://www.darkdel.com/store/p1750/July%3A__Little_Brother_%26_Homeland.html

* "Poesy the Monster Slayer" a picture book about monsters, bedtime,
gender, and kicking ass. Order here:
https://us.macmillan.com/books/9781626723627. Get a personalized, signed
copy here:
https://www.darkdel.com/store/p1562/_Poesy_the_Monster_Slayer.html.

This work licensed under a Creative Commons Attribution 4.0 license.
That means you can use it any way you like, including commercially,
provided that you attribute it to me, Cory Doctorow, and include a link
to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are
included either under a limitation or exception to copyright, or on the
basis of a separate license. Please exercise caution.

_,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,__,.-'~'-.,_

👒 How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/web/accounts/303320

Twitter (mass-scale, unrestricted, third-party surveillance and
advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

*When life gives you SARS, you make sarsaparilla* -Joey "Accordion Guy"
DeVilla

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://mail.flarn.com/pipermail/plura-list/attachments/20201205/7444504c/attachment.sig>