My understanding of the demands of that technology and my understanding of how NSA works leads me to believe that things are a lot more tiered than people seem to automatically presume. To the extent that there's a drift net on the domestic side (and I think there probably is) I think it's about using traffic analysis to flag communications of interest - this will produce a much, much, much (a significant number of orders of magnitude I would guess [I'd have to do some more informed thinking about the volume of traffic they're collecting vs. the scale of the targets to predict better, but I'd guess 6 or 8 or so]) smaller body of material that then gets fed through the sort of indexing that Nexidia provides.

"The spectacle of this great nation which does not know its own mind is as humiliating as it is dangerous." ~ Walter Lippmann

they identify phonemes, then put them in contexts that they're looking for and then drill down further from there.

This may mean that they are able to listen far more efficiently, thus ignoring irrelevant stuff which could invade your privacy.

So at a deeper level it may be less invasive; however, at the front end it seems quite invasive.

http://mauberly.blogspot.com/

What good does it do you to "flag communications of interest" if you can't cross reference them to anything historically?

If it were MY database and MY telecom network I'd pay AT&T and Global Crossing for facility space, install my splitters, build a backbone network to route domestic traffic to a central location with lots of inexpensive disks. Once in the door, I'd dump it to disk straight off the pipes in parallel. As my staging disk volume groups fill up I'd use background processes on something like IBM SP2s to read, write and index those bits by to/from, whether voice, internet or other. Once indexed, those bits, which might be voice, html, data, other can be used to do meaningful analysis of "communications of interest". Nothing would be "transcribed" real-time, unless required, but the equipment I'd install can do that too, and I have the backbone to watch real-time when needed.

The basic problem with "communications of interest" is that they expand from the point of origin. You are interesting. Who did you call in the past year? Who did they call? What email did they send? What web sites did they look at? To answer these questions I'd want to store it all, indexed by to/from in compressed digital format and would only uncompress the "interesting" things as I needed to look at them. The things that are interesting become interesting when they are uncompressed and "scored" for relevancy. They become even MORE interesting when they are cross referenced with other searchs. Everyone would have a "score". Just like your credit rating.

It is much more costly to pass very large amounts of digital information against large filter sets, than it is to simply dump to disk, index and coalesce later and then analyze. And determining what is "interesting" is historical, not predictive.

Two problems though. Once I have all of this on disk, it never goes away. Deleting all the stuff I don't need can never be done, because I'll NEVER know what I might need and it is VERY costly to sift through my thousands of terabytes of data to remove old stuff and free up a little disk space.

Second, and the most obvious, is that simply by having this stuff I can declare you, JustPlainDave, a terrorist, and make your life and everyone you know miserable. Happened in Greece. With Hayden in charge, and Rove sitting in meetings, what makes us think it can't happen here? What makes Rove so sure Hillary is a "fatally flawed candidate"? Perhaps Karl has been getting some transcripts from my little database...

Bonus questions:

1) For fun, who owns Hawaiian Telecom?
2) Which branches of our Armed Services are not represented on the Board of Directors of "public" company Global Crossing?
3) Who owns the facility where the trans-Pacific fiber loops terminate in Hawaii?
4) And lastly, why don't private companies care that their communications are completely compromised?

Time to put the tin foil hat to bed.
Cheers

1) The Carlyle Group
2) Nearly all the MIC is represented, just not necessarily by brass.
3) AT&T
4) That's the price they have to pay for a place at the trough.

Turn back to the Constitution - and

A+

Why would you presume that if traffic analysis is used to flag comms of interest that they become ahistorical and you can't reference them to anything? I'd guess that the highest weighting factors in identifying traffic come from past history on the numbers, geography, etc. and that that previous traffic is archived.

I've no doubt that they're collecting significant amounts of potential take for sifting later, but my sense is that's got to be heavily tiered by the traffic analysis as well - the mass of data otherwise is simply too large. How much storage does it take to collect an entire day of voice calls using a driftnet approach where you collect everything - now what about a week, month, year? I've seen back of the envelope calculations that lead me to be immensely skeptical - what are yours?

"The spectacle of this great nation which does not know its own mind is as humiliating as it is dangerous." ~ Walter Lippmann

was the best part.

http://mauberly.blogspot.com/

Had an absolutely miserable (stupid, micromanaging) boss. He asked for the weeks tasks lists every monday and would assemble them into a wonderful Gantt chart to show his bosses. So every Monday, we just randomly re-arranged our task lists. That kept him busy until Thurs afternoon. And he played poker Thurs nights, so Friday wasn't a problem.