Tolkien vs. Orwell: Who understood modern surveillance best (article)

[Janx]

Anyways, this doesn't seem to be the right kind of topic for ENWorld.

TomB

That depends. We talk about all kinds of things.

I do like how we've managed to keep out of the politics of it.

I think it is worthwhile to consider the mechanics of this kind of surveilance. As ggroy points out, the numbers aren't likely to be very efficient. As I point out, the technical requirements for a super-watcher is pretty impractical with today's technology, and it pretty much keeps scaling out of reach as we increase the internet activity.

Which to wrap it back to the original topic, watching everybody is inefficient and likely ineffective. Even though it might be a fun problem to solve for "find the bad guy in the data."

 

[tomBitonti]

This is a very bad rate.

So ... is this a good rate or a bad rate?

If every year there are 10,000 people investigated to stop one Boston (or 9/11) bomber, how does that turn out?

The question is how exactly will the 10,000 people be further distinguished. Such as another computer program whittling down the list further, or actual teams of human analysts examining the data further by hand (or some combination).

If human analysts are examining the cases further, it will require a lot of manpower and resources to investigate 10,000 people. (ie. Stakeouts, following a suspect, gathering evidence, etc ...).

But is it? (Note: I'm not disagreeing with you. But, I'm needing to see reasoning based by numbers and by a clear evaluation of the trade offs.)

The problem needs to be stated in the original form to avoid fallacies:

1 Boston Bomber per 1,000,000 people

NSA type monitoring on all of those people.

9,999 false positives. As a result of these false positives, these folks will have their lives turned upside down. Let's say a year of law enforcement harassment, perhaps more.

1 real positive.

There are three immediate problems:

* No time periods are mentioned. Let's thrown in "every year" as an additional detail.

* The figures would imply about 300 Boston Bombings a year in the US.

* Identifying the real positive itself involves some uncertainty. Does this mean detecting an ongoing and provable plot? Or stray thoughts?

If we use 9/11 as the example, that was about 20 people out of 300,000,000. I'm working out that that would mean 150,000 false positives against 20 positives in the entire country.

There is still a problem: 9/11 type events aren't initiated (it would seem) every year. Plus, the pool of possible bombers is larger than the US population.

Let's say those 150,000 are simply put in jail, until they are shown to not be a threat. How does locking 150,000 people a year (for one year) compare with preventing a 9/11 attack every 20 years? (Or X years: There is no basis that I have for assigning a frequency to 9/11 type events.)

Thx!

TomB

 

[ggroy]

* No time periods are mentioned. Let's thrown in "every year" as an additional detail.

With a time dependence element added in, all it does is make the mathematics more complicated.

If one were to do so, the easiest thing to incorporate it is a Poisson process.

But as with any statistical tool, even a Poisson process is dependent on past measurements as input data.


This is something that would require a lot more research, in order say anything that is possibly meaningful.

 

[ggroy]

* Identifying the real positive itself involves some uncertainty. Does this mean detecting an ongoing and provable plot? Or stray thoughts?

This would be highly dependent on the details of how the positive hit was made, and how exactly the list of positive hits is being whittled down.

For example, how much analyst manpower will it take to eliminate enough of the false positives, so that they have a manageable list of candidates?

Let's say those 150,000 are simply put in jail, until they are shown to not be a threat. How does locking 150,000 people a year (for one year) compare with preventing a 9/11 attack every 20 years? (Or X years: There is no basis that I have for assigning a frequency to 9/11 type events.)

I suppose something like this could be done in a totalitarian state. In a state where people have legal rights against unreasonable arrest and seizure, this may be harder to pull off.

The original calculation was to show how ineffective a mass data surveillance system is, with an NSA-style agency going on a fishing expedition through everybody and anybody's data.

 

[ggroy]

I suspect human intelligence may be more effective at identifying badguys which are a part of an organized (or semi-organized) group, where feds (or informants) have made efforts to infiltrate the ranks of such organizations.

Though human intelligence might not be as effective in finding the "lone wolves" type badguys. Especially if such lone wolves don't a priori fit into existing profiles of a "badguy", and don't leave much of a paper/data trail behind.

 

[ggroy]

The only other reasons I can think of offhand for the existence of such programs (even if they're ineffective in practice), is for "security theater" and CYA (cover your ass) type of reasons.

http://en.wikipedia.org/wiki/Security_theater
http://en.wikipedia.org/wiki/Cover_your_ass

 

[tomBitonti]

I suppose something like this could be done in a totalitarian state. In a state where people have legal rights against unreasonable arrest and seizure, this may be harder to pull off.

The original calculation was to show how ineffective a mass data surveillance system is, with an NSA-style agency going on a fishing expedition through everybody and anybody's data.

I'm trying to quantify the ineffectiveness. A process can be ineffective, but yield a positive value. The effectiveness must be measured against the value of the scarce successes against the cost of the failures.

In this space, there seems to be a question of reasonableness. The US justice system is deliberately imperfect, with a goal of "to be efficient, effective, and fair", with some (perhaps a lot) of dispute over what that means. (e.g., see https://www.bja.gov/evaluation/guide/documents/documentI.html).

The curious matter (to me) is that we have had 2/3 of our government basically approve of the process. The courts seem to have tried to avoid the issue for now. (I wonder if they will be able to, still.) We can argue of how representative those 2/3 actually are, but their approval provide a large part of the functional definition of what is considered "reasonable". (Not that I agree. I'm stating what I see as how the US decided what is the right "reasonable" balance of liberty vs monitoring.)

In this space, this statistic seems relevant:

http://www.prb.org/articles/2012/us-incarceration.aspx

(August 2012) Since 2002, the United States has had the highest incarceration rate in the world. Although prison populations are increasing in some parts of the world, the natural rate of incarceration for countries comparable to the United States tends to stay around 100 prisoners per 100,000 population. The U.S. rate is 500 prisoners per 100,000 residents, or about 1.6 million prisoners in 2010, according to the latest available data from the Bureau of Justice Statistics (BJS).1

150,000 new incarcerations is about a 10% increase in the number of prisoners. *If* we accept that many incarcerations as acceptable, the 10% increase is a small increase. (I think that the correct conclusion is that 1.6 million prisoners is horribly high.)

Thx!

TomB

 

[ggroy]

I'm trying to quantify the ineffectiveness. A process can be ineffective, but yield a positive value. The effectiveness must be measured against the value of the scarce successes against the cost of the failures.

Are you examining how much $$$$ it costs to get the scarce successes while weeding out the false positives?

 

[tomBitonti]

Are you examining how much $$$$ it costs to get the scarce successes while weeding out the false positives?

Hi,

Yes. The clearest costs are to liberty, and a $$$$ both in terms of the resources spent monitoring, investigating, and detaining people, and the loss that those people incur.

Incarcerating 150,000 people sounds expensive, but, we are incarcerating 10x that many already.

Actually, wikipedia gives much larger count of prisoners: 2.26 Million in 2010.

From:

http://en.wikipedia.org/wiki/Incarceration_in_the_United_States

And of those 2.26 million, almost 500,000 (22%) are on drug charges.

Perhaps the single greatest force behind the growth of the prison population has been the national "war on drugs." The number of incarcerated drug offenders has increased twelvefold since 1980. In 2000, 22 percent of those in federal and state prisons were convicted on drug charges.

I don't know how many of those are for traffickers or users, but, there would seem to be already a policy of incarcerating quite many more than 150,000 to fight drugs. Adding 150,000 to fight terrorism seems a smallish increase.

(Be careful of one implied conclusion, that adding 150,000 is OK. The policy is consistent with the drug policy. Another possible conclusion is that the drug policy is seriously out of whack.)

Thx!

TomB

 

[ggroy]

Yes. The clearest costs are to liberty, and a $$$$ both in terms of the resources spent monitoring, investigating, and detaining people, and the loss that those people incur.

(Ignoring the liberty part).

One could calculate how much at minimum it costs in manpower to investigate 10,000 positive hits each year.

For one employee being paid a federal minimum wage ($7.25/hour) working 9-5 (8 hours), it will cost $58 per day per employee. (In practice, the salaries may be significantly higher). Working weekdays with weekends off, one employee will be working around 261 days a year. Hence around a $15,138 salary per employee per annum.

If one employee is handling one positive hit as a full time job, at minimum it will cost around $151 million dollars per annum in minimum wage salaries (excluding other expenses) to investigate 10,000 positive hits.

Maybe the NSA and other government agencies might have the funds to investigate 10,000 positive hits. (Recent estimates of NSA's budget is around $10 billion per annum).

A local police force might not have that luxury. (For example, the Redmond, WA police department had an annual budget of around $30 million per annum over 2009-2012 while the Los Angeles LAPD had an annual budget of around $1.189 billion per annum).