Problems of a total surveillance system

by Andy Oram

Related link: http://www.nytimes.com/2002/11/09/politics/09COMP.html





The Pentagon is planning a comprehensive system for mining personal
data and tracking "potential enemies operating inside the United
States." Billed as an anti-terrorism measure, it will instead strike
most Americans as a key element of an upcoming totalitarian order.




There are plenty of healthy reasons to distrust this system, which bears the moniker "Total Information Awareness"
and is designed to "break down" the boundaries between government and
private data. You could potentially lose a security clearance or find
FBI agents at your door because of a university course you registered
for or a book you bought online. And the innocent people (or those who
commit minor infractions, like Los Alamos researcher Wen Ho Lee) who
find themselves harassed by this system will probably never learn
exactly why.




But there are some more subtle reasons to oppose the data mining
system, even if you support a bigger role for law enforcement.





Profiling becomes policy



U.S. law enforcement agencies often insist, "We don't single out
particular groups." (There are hard-fought exceptions, such as an FAA
system for profiling airline passengers.) Any algorithm for searching
data for clues is inherently a form of profiling. Whether or not the
system explicitly flags particular racial or national characteristics,
it will probably cause more snooping and harassment of particular
ethnic groups.





To be sure, some experts say that the only way to provide adequate
security in a large society is to do profiling. This issue should at
least be debated publicly, instead of quietly resolved by Pentagon
directions to computer programmers. Personally, I don't care in this case if profiling is necesary for security, because I consider profiling to be a basic
violation of human rights. It constitutes a form of collective
punishment for all people associated with misbehaving individuals.





No one takes responsibility for computer decisions



Since holding prejudices and following hunches are a necessary part of
any police work, one might feel safer entrusting the filtering
decisions to a computer program instead of leaving them in the hands
of individuals. But this begs the question of who makes decisions. As
Joseph Weizenbaum pointed out thirty years ago in Computer Power and
Human Reason
, one is asking for trouble by making computers issue
decisions that require subtle judgments about human life and
motivation. One ends up with near-parodies of the prejudices of the
programmers.





Basic tenets of privacy policy implode



The basis of the U.S. policy toward privacy in the computer era has
been to let businesses do pretty much what they want, on the basis
that they are private actors. If you don't like your health data
shared with pharmacies or your marital status shared with department
stores, that's something for you to balance in purchasing
decisions. There are many opponents to this view, but it has remained
the justification for the "self-regulation" and general government
hands-off policy in the U.S. for at least twenty years.




Well, it looks like it's time for consumers to demand that all
information on their purchases and behavior be destroyed. Businesses
are no longer private actors--they are an arm of the Pentagon. What
the businesses store is a matter of public policy, and should be the
concern of everyone who interacts with them.





Innocents suffer while sophisticated criminals evade the system



People unfairly smeared as "potential enemies" have little recourse
but to suffer whatever disruption that causes in their attempts to
pursue ordinary lives. But terrorist organizations can watch cases
carefully and pick clues about what is likely to land a person on the
hot seat. The profiling system will become a cat and mouse game, with
law enforcement constantly tweaking the criteria for flagging
suspects--and trying them out on a largely innocent population.







Nobody would claim that the Pentagon or law enforcement has an easy
job, and few would have the temerity to suggest that the right path
forward is clearly lit and delineated. Still, we can apply some basic
criteria to judging inititiatives.




We know that over-eager or panicky law enforcement agents have caused
lots of unwarranted suffering for innocent people. And we know, from
the truncated investigations of the September 11 attacks as well as
other incidents like the Washington-area sniper, that agencies don't
make the best use of the information they already have.




There are always trade-offs between giving law enforcement new tools
to do their job and giving them power that will inevitably be
misused. The conceptual weakness--apart from any technical problems
they'll run into--of the current Pentagon project marks it a bad
idea. It is up to everyone who values privacy and the First Amendment
to resist and protest the system. Even though such resistance and
protest--this is one aspect one can count on--will increase our
chances of ending up flagged by the system.