A month after my family and I moved into our new home, we received a letter from the utility company. Our water usage had increased sharply, we were told, and we should check for leaks and various other problems. We chuckled at this – it was no surprise that my family of five used more water than the kindly widow we’d bought the house from.
Of course the utility company has every reason to know about my usage of utilities, and this program is a fine thing that probably saves a lot of water. But I always remembered that letter. It permanently changed my mental understanding of my relationship with my utility, which has vast numbers of customers, and surely (I believed) would not pay any attention to a random, reliable bill payer like me. After the letter came, I became a bit more self-conscious about my water usage. I had a bit more of a sense that I was being watched.
The letter was certainly generated by a computer. And I prefer the image of a dumb computer churning through water-consumption levels, looking for spikes, and mindlessly firing off a form letter, to the thought of some utility official personally reviewing my water consumption each month.
But what happens when we take that concept – the automated, mass monitoring of behavior – and dial it up 1000%? At what point, if any, should we start getting concerned? When if ever does automated monitoring by a computer constitute an invasion of privacy? And does it make sense to worry less about our privacy when we are monitored by machines than when we are monitored by human beings?
Commentators from a range of viewpoints have taken it for granted that mere machine monitoring should not be regarded as a true invasion of privacy – human beings, in this view, are the true source of concern. I've been doing some thinking about this question over the past 8 years or so, and occasionally setting down some thoughts. This is the first in a series of planned posts on this subject.
Is the intuition that machines are more benign observers than our fellow humans correct? What is it about people, as opposed to machines, that makes us feel shy?
Anthropomorphism
The thing about the dumb
computer that checks my water usage is that I can’t help personalizing
it a little bit. (“I’m taking a long shower this morning, I wonder if
it will write me again?”) Maybe I’m especially self-conscious, but
anthropomorphism seems to come naturally to humans; we constantly
impute humanity to inanimate objects. Our minds are hungry for
relationships, and are apt to see them everywhere. Picture the most
simple, bone-dumb object – a hammer, say. It has only one part, and
it’s pretty hard to personalize. But move up to two and already we
start to see the effect; imagine that the hammer’s handle sometimes
comes loose from its head. As you work, it seems to hold – but then
repeatedly comes apart at the worst possible moments. This seemingly
perverse behavior makes you angry at – at what? The hammer. Already
it’s starting to feel like it has a personality.
We can think of all the devices in our lives as arranged along a spectrum according to the intelligence of their “minds.” The hammer (intact) would be at one end. Then comes progressively more complicated and “tempermental” machines and appliances. By the time you get to something truly complex, such as an operating system, the anthropomorphism can be intense. Finally, at the far end of our spectrum would be an artificially intelligent agent that is as smart as a human, or more. Here, “anthropomorphism” may no longer even be the right word. If the thesis of materialism is correct – that we are purely physical beings with no metaphysical soul or “divine spark” necessary to bring us to life – then it is actually quite likely that computer programs will be created (or more likely, evolved) that are at least as intelligent as we are. Even if true artificial intelligence proves impossible, it seems clear that our computers are bound to get a lot smarter than they are now. Indeed, as I've written, artificial intelligence could be the next big technology revolution to hit us.
So what happens when this rising intelligence is applied to monitoring people? More anon.
In Part 1 I raised the question of whether monitoring by machine could be an invasion of privacy, and raised the question of what happens as we develop machines of increasing intelligence.
One of the developments that first raised that question prominently is Google’s Gmail. Like Hotmail, Yahoo, and others, Gmail provides free e-mail – but with a difference: Google software scans the content of your e-mails and then sends you ads that appear to be relevant based on the content of your messages. So if you spend a lot of time emailing your friends about triathlons, they may serve you ads for triathlon products.
Many people find this concept creepy, and there was an initial flurry of media attention paid to the concept when it was first introduced in March 2004, and a fair amount of discussion in the technology and privacy communities. But a lot of people, including many sympathetic to privacy concerns, dismissed complaints over Gmail. Some libertarian-minded defenders simply pointed to the service’s voluntary nature. But others pointed out that after all, many ISPs and system administrators already scan our e-mail in order to filter out spam. Most people do not find virus scanning of e-mail messages, even by giant corporations like AOL or Yahoo, to be as spooky as Google’s content scanning. How, they asked, is it qualitatively different for messages to be scanned for content by Google for the purpose of serving ads?
The difference, the reason that Gmail seems creepier, has to do with the intelligence of the scanning agent – where they lie along the spectrum between hammer and human. With spam scanners, we understand that the agent is dumb, and that it is seeking only to answer a yes/no question: is this e-mail a piece of spam? True, a spam scanner could be described as a robot listening in to our e-mail conversation, but it is deaf to what we are really saying. It is a true idiot, interested only in one thing, and doesn’t know whether you’re talking about sex, food, music, exercise, or politics. When it comes to creating the feeling that our privacy is being invaded – to creating an anthropomorphic sense of “mind” – the spam scanner is not much different from a vacuum cleaner, or the network hardware through which your messages pass.
But the Gmail agent sounds like it might, in fact, understand a little bit of your conversations. It is a significant step further along the spectrum toward the sentient end where we feel like we’re no longer alone.
Truth be told, the Gmail agent is still in all likelihood pretty dumb. Most people, if they could test the system systematically, would quickly get a feel for this dumbness. (Computer gamers know the experience – how computerized opponents that initially seem wily and unpredictable are quickly exposed as automatons as our far more complex minds “get their number.”) But as we think about scanning agents with more and more intelligence, we reach the question of just what the line is between a computerized scanning agent and a low-level, not particularly bright, but fully human person scanning one’s mail.
This is an issue that we are bound to confront with increasing frequency in coming years.
The technology revolution - word processors, e-mail, sensors of all kinds, and the general saturation of our lives by computer chips - has left us drowning in data as never before. That is already and will increasingly create a constant demand for automated ways of filtering through all that noise, so that the high-quality, low bandwidth intelligences that are our brains can focus on the significant pieces.
The cutting edge of this problem is at the National Security Agency, which sucks in enormous volumes of communications from around the world – and, as we learned during the Bush Administration, courtesy of the New York Times, illegally from within the United States itself. I once mentioned to a former NSA data expert that I’d heard that the agency collects four petabytes per month (that’s equivalent to a million gigabytes or a billion megabytes). His reaction: “that’s kind of small” (this was around 2003 when 4 petabytes seemed bigger than it does now). Of course, he was quick to add, “most of that’s junk.”
And therein lies the problem for the NSA: most of it's junk. How can the agency make good use of the one resource that will never be limitless: its human analysts’ time? To make any sense of the oceans of data that it is pulling in, the agency must rely on computers to filter it, and flag any potentially interesting information. We know next to nothing what those artificial analysts’ assistants look like, how they are programmed, or how well they function. How much intelligence (as in smarts) is required to collect useful intelligence (as in espionage) in this way?
The NSA is clearly scanning through large volumes of communications foreign and domestic - in effect, running a national-security version of my local utility’s water-usage monitor. Yet, Bush officials repeatedly insisted that the only Americans whose privacy has been invaded were, as President Bush put it, “people with known links to al Qaeda and related terrorist organizations.” Assuming Bush officials were not simply lying, there is a simple explanation for this disconnect: they were not counting telephone calls and e-mails intercepted by a computer as an “invasion of privacy.” In other words, in this view, if a computer looks over your e-mail and decides that it contains no evidence of wrongdoing, your privacy has not been invaded.
In short, our question – when if ever does automated monitoring by a computer constitute an invasion of privacy – is central to one of the most significant scandals of the Bush Administration and one of the most significant debates to have emerged in the uneasy history of government surveillance. And this question will be central to the future relationship overall between individuals and governments in 21st century democracies.
Some defenders of the NSA program cite the machine nature of the NSA’s eavesdropping as reason not to worry. James Jay Carafano of the Heritage Foundation and Kim Taipale of the Center for Advanced Studies in Science and Technology Policy, for example, argued in a January 2006 piece that existing wiretap statutes are obsolete because they do not contemplate the “automated monitoring of suspected terrorist communications.” Such automation, they boast, “can monitor data to uncover terrorist connections without human beings ever looking at anybody's emails or listening in on their phone calls.” The authors do not feel they even need to explain how that makes a difference; the implication is that it is a ridiculous idea that eavesdropping by machines might require a warrant like eavesdropping by humans.
Similarly, Heather MacDonald of the conservative Hudson Institute, for example, responding to arguments against a proposed system for using computers to rate the security risk of airline passengers, wrote that:
Are they right, then? Just what is the
difference between humans and machines - the role of sentience, of
greater or lesser intelligence - when it comes to eavesdropping on us
or monitoring our activities? More anon.
In Part 2 I discussed the beginnings of intelligent machine monitoring in Gmail, and how our technology will likely create a push to do more and more monitoring and sifting of communications by machine. All of which intensifies the questions at hand: can machines invade our privacy, and what effect does their increasing "intelligence" have?
The first difference that occurs to most of us is that human beings are simply smarter than machines. Even a person who is significantly below average in intelligence still understands far, far more about the world than even the sharpest of today’s computer programs. When it comes to an eavesdropper, intelligence matters to us because it determines the quality of information that is flagged or retained about you. A smarter “mind” (whether man or machine) will evaluate a greater quantity and more subtle information about you, and with greater sophistication, while a dumber mind will miss much of what is going on, fixating perhaps on a single dimension of your behavior. For example, a smart mind might pick up the tone of disdain in your references to Starbucks, put that together with your references to Noam Chomsky as well as certain other signals, and conclude that you’re an anti-globalization activist. A dumber mind might reach the same conclusion based on your utterance of the keywords “globalization really sucks,” failing to discount for your ironic context and sarcastic tone, your glowing references to NAFTA, and your position on the board of directors of an international manufacturing conglomerate. A truly subtle mind might hear you making extremely vehement anti-globalization arguments, but pick up on the fact that you are actually ambivalent about the issue, and are merely staking out a radical position with some close friends in private for the fun of seeing how it stands up.
Of course, the smarter mind does not necessarily use more information in forming its judgment. In fact, part of what makes the smarter mind smarter is precisely the fact that it has learned to ignore the vast majority of the information that comes its way and select only that which is relevant. The smarter mind acts as a much more accurate filter. We never hear most of the noises that strike our ears, nor see most of the sights that strike our eyes, nor feel most of the things that touch us -- nor remember most of the things that we do hear, see or feel.
But at root, the key difference is that the smarter mind has a more accurate mental map of the world, and so it is able to make more sophisticated filtering decisions and evaluate the relevant information in ways that are much more likely to be accurate in predicting your future behavior (which is perhaps our key goal when we gather information about another human being).
But intelligence is not the only factor that matters to us when contemplating the kind of eavesdropper who might be overseeing our communications. Another factor, tightly related to but ultimately separate from intelligence, is its connectedness –- the degree to which the eavesdropping can reverberate in our lives. People live within social networks and communities, and as a result things that we say can spread through these networks and come back to bite us. We are all exposed to the danger that something we say will be overheard and spread around, because we all say things to some people that we don’t want other people to hear. That is part of human life. People say things to their families they wouldn’t want their co-workers to overhear; and things to close friends they wouldn’t want their families to hear, and things to their coworkers they wouldn’t want their friends to hear. The importance of being able to control the reverberation of private conversations has long been implicitly recognized, for example through such institutions as the doctor-patient confidentiality, attorney-client privilege, and the special protections against testifying that are given to priests and spouses. And it is why the prospect of having something we say to one audience spill over into another audience raises a special kind of social terror in most people.
Is connectedness really separate from intelligence? Yes, because like people, computers can also be tied into networks – networks of other computers, and networks of people. E-mail monitoring software, for example, could be programmed to sound an alarm for an intelligence analyst based on something it finds in one’s e-mail. Or – to remove the human being from the equation altogether – it could be programmed to alert other computers in a network, which then make future judgments of the individual based upon that alert. Being watched by a computer can be the equivalent of being monitored by a particularly dogmatic, knee-jerk and thuggish informant – but the reports from even a really dumb mind can reverberate significantly.
Many people have a very strong intuition that there is something different about sentience. But this is an illusion. It is just that we are accustomed to a situation where sane people do not worry about discussing their love lives in front of their refrigerators or laptops, while encounters with sentient minds can bring dangerous consequences if not handled much more carefully. At root, it is consequences that we fear, not some abstract sense of moral scrutiny by Sentience. Embarrassment is nothing more than a form of fear. More anon.
