[IP] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Case
----- Forwarded message from David Farber <dave@farber.net> -----
First of all, anyone have the latest word about Cryptome? Second of all, I took a look at these and what's becomming quite clear is that they don't really say that much. They basically just show how the LGXs are connected and interface into and out of the "secret room" via splitters.* It does say that a buttload of traffic is being split and sent off to NSA equipmet, but the real missing piece is just how that traffic gets back to NSA. I still believe that they just can't send back EVERYTHING, and have to have several layers of prioritzation, so that only fairly interesting traffic makes it back in real time (this is not to say that they don't possibly route and store uninteresting local traffic for future reference) but they can't get everything back to, for instance, DC in real time. Another question I've had for a while is how they get around the loss budgets in certain cases. Dropping a 3dB splitter into an OC-48 signal that's pushed to the limit will result in some signficant BER degredation. Do they just avoid those signals? DO they put in some kind of in-line optical amplifier? (That's not trivial, as they have to electrically power such a device.) -TD *: On the other hand, the existence of splitters alone proves that this is not standard CALEA, so when the case broke this was significant. But technologically it's not all that interesting nor surprising (for 'paranoid' Cypherpunks, at least).
From: Eugen Leitl <eugen@leitl.org> To: cypherpunks@jfet.org Subject: [IP] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Case Date: Tue, 12 Jun 2007 20:22:43 +0200
----- Forwarded message from David Farber <dave@farber.net> -----
From: David Farber <dave@farber.net> Date: Tue, 12 Jun 2007 14:08:25 -0400 To: ip@v2.listbox.com Subject: [IP] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Case X-Mailer: Apple Mail (2.752.2) Reply-To: dave@farber.net
Begin forwarded message:
From: EFF Press <press@eff.org> Date: June 12, 2007 1:29:33 PM EDT To: presslist@eff.org Subject: [E-B] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Case Reply-To: press@eff.org
Electronic Frontier Foundation Media Release
For Immediate Release: Tuesday, June 12, 2007
Contact:
Cindy Cohn Legal Director Electronic Frontier Foundation cindy@eff.org +1 415 436-9333 x108 (office), +1 415 307-2148 (cell)
Kurt Opsahl Senior Staff Attorney Electronic Frontier Foundation kurt@eff.org +1 415 436-9333 x106
Secret Surveillance Evidence Unsealed in AT&T Spying Case
Whistleblower Declaration and Other Key Documents Released to Public
San Francisco - More documents detailing secret government surveillance of AT&T's Internet traffic have been released to the public as part of the Electronic Frontier Foundation's (EFF's) class-action lawsuit against the telecom giant.
Some of the unsealed information was previously made public in redacted form. But after negotiations with AT&T, EFF filed newly unredacted documents describing a secret, secure room in AT&T's facilities that gave the National Security Agency (NSA) direct access to customers' emails and other Internet communications. These include several internal AT&T documents that have long been available on media websites, EFF's legal arguments to the 9th Circuit, and the full declarations of whistleblower Mark Klein and of J. Scott Marcus, the former Senior Advisor for Internet Technology to the Federal Communications Commission, who bolsters and explains EFF's evidence.
"This is critical evidence supporting our claim that AT&T is cooperating with the NSA in the illegal dragnet surveillance of millions of ordinary Americans," said EFF Legal Director Cindy Cohn. "This surveillance is under debate in Congress and across the nation, as well as in the courts. The public has a right to see these important documents, the declarations from our witnesses, and our legal arguments, and we are very pleased to release them."
EFF filed the class-action suit against AT&T last year, accusing the telecom giant of illegally assisting in the NSA's spying on millions of ordinary Americans. The lower court allowed the case to proceed and the government has now asked the 9th U.S. Circuit Court of Appeals to dismiss the case, claiming that the lawsuit could expose state secrets. EFF's newly released brief in response outlines how the case should go forward respecting both liberty and security.
"The District Court rejected the government's attempt to sweep this case under the rug," said EFF Senior Staff Attorney Kurt Opsahl. "This country has a long tradition of open court proceedings, and we're pleased that as we present our case to the Court of Appeals, the millions of affected AT&T customers will be able to see our arguments and evidence and judge for themselves."
Oral arguments in the 9th Circuit appeal are set for the week of August 13.
For the unredacted Klein declaration: http://eff.org/legal/cases/att/SER_klein_decl.pdf
For the internal documents: http://eff.org/legal/cases/att/SER_klein_exhibits.pdf
For the unredacted Marcus declaration: http://eff.org/legal/cases/att/SER_marcus_decl.pdf
For EFF's 9th Circuit brief: http://eff.org/legal/cases/att/9thanswerbrief.pdf
For more on the class-action lawsuit against AT&T: http://www.eff.org/legal/cases/att/
For this release: http://www.eff.org/news/archives/2007_06.php#005304
About EFF
The Electronic Frontier Foundation is the leading civil liberties organization working to protect rights in the digital world. Founded in 1990, EFF actively encourages and challenges industry and government to support free expression and privacy online. EFF is a member-supported organization and maintains one of the most linked-to websites in the world at http://www.eff.org/
-end-
_______________________________________________ presslist mailing list https://falcon.eff.org/mailman/listinfo/presslist
------------------------------------------- Archives: http://v2.listbox.com/member/archive/247/=now RSS Feed: http://v2.listbox.com/member/archive/rss/247/ Powered by Listbox: http://www.listbox.com
----- End forwarded message ----- -- Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
_________________________________________________________________ Dont miss your chance to WIN $10,000 and other great prizes from Microsoft Office Live http://clk.atdmt.com/MRT/go/aub0540003042mrt/direct/01/
On 6/13/07, Tyler Durden <camera_lumina@hotmail.com> wrote:
First of all, anyone have the latest word about Cryptome?
what?
Second of all, I took a look at these and what's becomming quite clear is that they don't really say that much. They basically just show how the LGXs are connected and interface into and out of the "secret room" via splitters.* It does say that a buttload of traffic is being split and sent off to NSA equipmet ...
right. tap all the interesting fibers, feed to narus.
but the real missing piece is just how that traffic gets back to NSA. I still believe that they just can't send back EVERYTHING, and have to have several layers of prioritzation, so that only fairly interesting traffic makes it back in real time (this is not to say that they don't possibly route and store uninteresting local traffic for future reference) but they can't get everything back to, for instance, DC in real time.
the narus is there specifically so they don't have to backhaul a mirror of the traffic. it does all the inspection to isolate interesting information, then sends back that interesting information to aggregation points, before that in turn is sent on to NSA. the bridgeton center att noc is a good example. there is a room controlled by multi-factor biometric authentication (print, retinal) with man trap doors. this is probably the room used for distributing configuration to the remote monitoring points (it's unlikely they store much of interest at the remote sites, since the security is much lower at these places) as well as aggregation of the feeds for backhaul to NSA. see also the new NSA facilities being built in denver, CO. this is an ideal place to aggregate traffic across the country...
Another question I've had for a while is how they get around the loss budgets in certain cases. Dropping a 3dB splitter into an OC-48 signal that's pushed to the limit will result in some signficant BER degredation. Do they just avoid those signals? DO they put in some kind of in-line optical amplifier? (That's not trivial, as they have to electrically power such a device.)
from the sounds of it, the taps did introduce some problems which were resolved quickly. probably not from signal loss, but who knows. in any case, i don't think powering an optical amplifier is difficult in the facilities in question. for transoceanic cables it becomes a bigger problem :) best regards,
Well, the need for optical amplifiers is not so much a power availability issue so much as a complexity issue. If one has dozens or hundreds of fibers, remember that each of these is going to be some random wavelength in the 1310nm or 1550nm band: You won't be able to use a single optical amplifier to amplifiy multiple wavelengths: You'll need one for each fiber. So now, you need the space and the manpower (not to mention electrical power and heat dissipation) to handle a lot of OFAs. Now I wouldn't be suprised if there existed a VERY small in-line EDFA (erbium-doped fiber amplifier) that could operate on a smalll amount of power (perhaps battery?) and just hang off the end of a splitter. No, they would obviously incorporate the WDFA into a 'powered splitter' (such a thing has to eexist now that I think about it). But the trick would be to lower the number of fibers needed to tap and then hope that most of those are operating more than 3dB above their floor, and then if you have to split and optically power such a fiber. This implies a level of prioritization even prior to reaching the 'secret room'. So what can that Narus box do? Layer 3/4 seems obvious. Layer 7? Can it detect the presence of encryption? Can it detect the type and strength of encryption? Can it detect the existence of jpegs, mpegs, mp3s and so on? Does it support routing protocols? So, could it handle an instruction like: If the source of the packet is located in Brooklyn If the destination is a cave in Aghanistan If encryption = YES, Then route to NSA... If destination is Kebab King in Jackson Heights If encryption=NO, then route to Local Storage etc... -TD
From: coderman <coderman@gmail.com> To: "Tyler Durden" <camera_lumina@hotmail.com> CC: cypherpunks@jfet.org Subject: Re: [IP] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Cas Date: Wed, 13 Jun 2007 10:45:17 -0700
On 6/13/07, Tyler Durden <camera_lumina@hotmail.com> wrote:
First of all, anyone have the latest word about Cryptome?
what?
Second of all, I took a look at these and what's becomming quite clear is that they don't really say that much. They basically just show how the LGXs are connected and interface into and out of the "secret room" via splitters.* It does say that a buttload of traffic is being split and sent off to NSA equipmet ...
right. tap all the interesting fibers, feed to narus.
but the real missing piece is just how that traffic gets back to NSA. I still believe that they just can't send back EVERYTHING, and have to have several layers of prioritzation, so that only fairly interesting traffic makes it back in real time (this is not to say that they don't possibly route and store uninteresting local traffic for future reference) but they can't get everything back to, for instance, DC in real time.
the narus is there specifically so they don't have to backhaul a mirror of the traffic. it does all the inspection to isolate interesting information, then sends back that interesting information to aggregation points, before that in turn is sent on to NSA.
the bridgeton center att noc is a good example. there is a room controlled by multi-factor biometric authentication (print, retinal) with man trap doors. this is probably the room used for distributing configuration to the remote monitoring points (it's unlikely they store much of interest at the remote sites, since the security is much lower at these places) as well as aggregation of the feeds for backhaul to NSA.
see also the new NSA facilities being built in denver, CO. this is an ideal place to aggregate traffic across the country...
Another question I've had for a while is how they get around the loss budgets in certain cases. Dropping a 3dB splitter into an OC-48 signal that's pushed to the limit will result in some signficant BER degredation. Do they just avoid those signals? DO they put in some kind of in-line optical amplifier? (That's not trivial, as they have to electrically power such a device.)
from the sounds of it, the taps did introduce some problems which were resolved quickly. probably not from signal loss, but who knows.
in any case, i don't think powering an optical amplifier is difficult in the facilities in question. for transoceanic cables it becomes a bigger problem :)
best regards,
_________________________________________________________________ Picture this share your photos and you could win big! http://www.GETREALPhotoContest.com?ocid=TXT_TAGHM&loc=us
On Thu, Jun 14, 2007 at 06:03:18AM -0400, Tyler Durden wrote:
So what can that Narus box do? Layer 3/4 seems obvious. Layer 7? Can it detect the presence of encryption? Can it detect the type and strength of encryption? Can it detect the existence of jpegs, mpegs, mp3s and so on? Does it support routing protocols? So, could it handle an instruction like:
If the source of the packet is located in Brooklyn
If the destination is a cave in Aghanistan If encryption = YES, Then route to NSA...
If destination is Kebab King in Jackson Heights If encryption=NO, then route to Local Storage
etc...
http://en.wikipedia.org/wiki/Narus ... NarusInsight The capabilities of the NarusInsight system are alarming to many privacy advocates. [edit] System specification
From the Key Features list of NarusInsight:
* Universal data collection from links, routers, soft switches, IDS/IPS, databases, etc. provides total network view across the world's largest IP networks. * Normalization, Correlation, Aggregation and Analysis provide a comprehensive and detailed model of user, element, protocol, application and network behaviors, in real-time. * Seven 9s reliability from data collection to data processing and analysis. * Industry-leading packet processing performance that supports network speeds of up to OC-192 at layer 4 and OC-48 at layer 7, enabling carriers to monitor traffic at either the edge of the network or at the core. * Scalability to support the world's largest, most complex IP networks. * Flexibility -- NarusInsight's functionality can easily be configured to meet any specific customer requirement (Narus SDK). * Extensibility -- NarusInsight's functionality can easily be configured to feed a particular activity or IP service such as security, lawful intercept or even Skype detection and blocking. OC-192 carries about 10 gigabit per second. Ten billion bits per second, monitored in real-time. Technically, this is an extremely powerful supercomputer. [edit] System capabilities * Packet-mode data intercepts for Service Providers and Carriers. * Wireline to wireless and Wi-Fi or dialup to broadband. * "Instant Compliance" with CALEA and ETSI for simple, fast and hands-free compliance. * Carrier-grade speeds, performance and scalability. * Supports all of your services, out-of-the-box. * Securely manages resources while simplifying audits and reporting. * Network and vendor agnostic. * Enables additional application for revenue generation or revenue protection. This data flows right into NarusInsight Intercept Suite, which enables packet-level, flow-level, and application-level usage information is captured and analyzed as well as raw user session packets for forensic analysis, surveillance or in satisfying regulatory compliance for lawful intercept. The Lawful Intercept module offers carriers and service providers compliance with regulatory requirements regarding lawful intercept. The Lawful Intercept module provides an end-to-end solution consisting of Administration, Access and Delivery functions. The Lawful Intercept module is compliant with CALEA and ETSI standards. It can be seamlessly integrated with third party products for testing/validation or as a complete law enforcement solution. The Directed Analysis module seamlessly integrates with NarusInsight Secure Suite or other DDoS, intrusion or anomaly detection systems, securely providing analysts with real-time, surgical targeting of suspect information (from flow to application to full packets). The Directed Analysis module provides industry standard formats and offers tools for archival and integration with third party investigative tools. These capabilities include playback of streaming media (i.e. VoIP), rendering of web pages, examination of e-mail and the ability to analyze the payload/attachments of e-mail or file transfer protocols. Narus partner products offer the ability to quickly analyze information collected by the Directed Analysis or Lawful Intercept modules. When Narus partners' powerful analytic tools are combined with the surgical targeting and real-time collection capabilities of Directed Analysis and Lawful Intercept modules, analysts or law enforcement agents are provided capabilities that have been unavailable thus far. [edit] Usage It is useful to examine the OSI model of seven layers which underpins all communication on the Internet. NarusInsight focuses on two layers: number four, the transport layer, built on standards like TCP and UDP, the physical building blocks of Internet data traffic, and number seven, the application layer, built on standards like HTTP and FTP, which are dependent on the application using them, i.e. Internet Explorer, Kazaa, Skype, etc. NarusInsight monitors 10 billion bits per second at level four and 2500 million bits per second at level seven. For reference, a 256K DSL line equals 0.25 million bits per second (Mbit/s), and a normal modem around 0.05 Mbit/s. So a single NarusInsight machine can monitor traffic equal to the maximum capacity of around 39,000 DSL lines or 195,000 modems. In practical terms 10 Gbit/s equals the combined traffic of millions of broadband users, so the number of subscribers monitored by one installation is several millions. It can also perform semantic analysis of the same traffic as it is happening, in other words analyze the content, meaning, structure and significance of this entire traffic, as it is happening. The exact use of this data is not fully documented. A starting point is the Internet Protocol Detail Record, used to record information about usage activity within the telecom infrastructure (such as a call completion). NDM-U stands for "Network Data Management - Usage". It refers to a functional operation within the Telecom Management Forum's Telecom Operations Map. The NDM function collects data from devices and services in a service providers network. Usage refers to the type of data which is the focus of this document. These standards were built into Narus' systems. "IPDR.org has been in existence since 1999 and more than a dozen vendors have actual IPDR implementations "etched in code". Their systems are actually able to talk to each other and interoperate. Version 2.5 and up of the NDM-U represents a stable basis for development. IPDR.org's Interoperability Pavilion is a working demonstration of multiple companies exchanging service usage data in that format." Service usage data. That would be data on the actual usage of the Internet. And what kind of data would this be? Way back in 1999, this article stated: "In an effort to provide more complex network traffic analysis, Narus is introducing its semantic network traffic service. The company cites research which predicts the fast-growing ISP sector will become stagnant without the ability to offer differentiated services. In order to gain significant revenues from these services, a technology was necessary to allow usage based pricing. "We realized that, at the heart of the data that is needed to accurately measure usage and enable 'pay-as-you-go' business models for Internet service providers, is what we call the 'semantics' of network traffic," said Ori Cohen, Narus' founder and chief executive officer. "In short, by seeing the 'semantics' of network traffic, service providers can see 'inside' the data, providing much more detailed insight about the use of the Internet and the perceived value of specific applications than existing technologies allow." "Semantic Traffic Analysis uses network technology to consistently capture and analyze all IP data streams on heavily trafficked networks remotely and non-invasively. In addition, the semantics of the data stream are determined also, as well as the protocol used and the application taking place. A variety of other data is available as well." In this context, semantics is not just the data, but rather the meaning of the data. It looks at the data in a more comprehensive way than looking for keywords. Each NarusInsight machine does this at 2500 million bits per second, in real-time. One website calls this "the biggest invasion of privacy in history by several orders of magnitude."
From Narus' Lawful Intercept and Regulatory Compliance page:
"Explosive Internet growth in recent years has transformed worldwide communications, yielding tremendous efficiencies and benefits, as well as many risks." "For example, terrorist attacks around the globe have been carefully orchestrated through Internet-based forms of communications such as e-mail, messaging, hidden Web pages and now VoIP, forcing governmental organizations and law enforcement agencies to re-evaluate how they are providing public security as it becomes so much easier and faster to communicate electronically." "Recent mandates and the resulting standards referenced under CALEA in the United States and ETSI in Western Europe aim to preserve the right of law enforcement agencies to conduct authorized electronic surveillance in an effort to protect the public and its right to privacy. However, these mandates create IT headaches for carriers as they struggle to meet the requirements." "With a suite of products targeted at meeting lawful intercept requirements, Narus simplifies lawful intercept tasks helping carriers and agencies meet requirements without experiencing any degradation in service quality." -- Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
Shee-IT. Layer 4 packet inspection at OC-192 is kinda surprising, but Layer 7 at OC-48 is for me the more difficult thing to swallow. This has interesting implications in terms of where they place one of these boxes. Another thing worth thinking about is the control channels they must use to update the policies to one of these boxes. It's obviously in-band. One wonders if one could tap one of the fibers and find the packet stream they use to program one of these things. -TD
From: Eugen Leitl <eugen@leitl.org> To: Tyler Durden <camera_lumina@hotmail.com>, cypherpunks@jfet.org Subject: Re: [IP] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Cas Date: Thu, 14 Jun 2007 12:33:56 +0200
On Thu, Jun 14, 2007 at 06:03:18AM -0400, Tyler Durden wrote:
So what can that Narus box do? Layer 3/4 seems obvious. Layer 7? Can it detect the presence of encryption? Can it detect the type and strength of encryption? Can it detect the existence of jpegs, mpegs, mp3s and so on? Does it support routing protocols? So, could it handle an instruction like:
If the source of the packet is located in Brooklyn
If the destination is a cave in Aghanistan If encryption = YES, Then route to NSA...
If destination is Kebab King in Jackson Heights If encryption=NO, then route to Local Storage
etc...
http://en.wikipedia.org/wiki/Narus
...
NarusInsight
The capabilities of the NarusInsight system are alarming to many privacy advocates.
[edit] System specification
From the Key Features list of NarusInsight:
* Universal data collection from links, routers, soft switches, IDS/IPS, databases, etc. provides total network view across the world's largest IP networks. * Normalization, Correlation, Aggregation and Analysis provide a comprehensive and detailed model of user, element, protocol, application and network behaviors, in real-time. * Seven 9s reliability from data collection to data processing and analysis. * Industry-leading packet processing performance that supports network speeds of up to OC-192 at layer 4 and OC-48 at layer 7, enabling carriers to monitor traffic at either the edge of the network or at the core. * Scalability to support the world's largest, most complex IP networks. * Flexibility -- NarusInsight's functionality can easily be configured to meet any specific customer requirement (Narus SDK). * Extensibility -- NarusInsight's functionality can easily be configured to feed a particular activity or IP service such as security, lawful intercept or even Skype detection and blocking.
OC-192 carries about 10 gigabit per second. Ten billion bits per second, monitored in real-time. Technically, this is an extremely powerful supercomputer.
[edit] System capabilities
* Packet-mode data intercepts for Service Providers and Carriers. * Wireline to wireless and Wi-Fi or dialup to broadband. * "Instant Compliance" with CALEA and ETSI for simple, fast and hands-free compliance. * Carrier-grade speeds, performance and scalability. * Supports all of your services, out-of-the-box. * Securely manages resources while simplifying audits and reporting. * Network and vendor agnostic. * Enables additional application for revenue generation or revenue protection.
This data flows right into NarusInsight Intercept Suite, which enables packet-level, flow-level, and application-level usage information is captured and analyzed as well as raw user session packets for forensic analysis, surveillance or in satisfying regulatory compliance for lawful intercept.
The Lawful Intercept module offers carriers and service providers compliance with regulatory requirements regarding lawful intercept. The Lawful Intercept module provides an end-to-end solution consisting of Administration, Access and Delivery functions. The Lawful Intercept module is compliant with CALEA and ETSI standards. It can be seamlessly integrated with third party products for testing/validation or as a complete law enforcement solution.
The Directed Analysis module seamlessly integrates with NarusInsight Secure Suite or other DDoS, intrusion or anomaly detection systems, securely providing analysts with real-time, surgical targeting of suspect information (from flow to application to full packets). The Directed Analysis module provides industry standard formats and offers tools for archival and integration with third party investigative tools.
These capabilities include playback of streaming media (i.e. VoIP), rendering of web pages, examination of e-mail and the ability to analyze the payload/attachments of e-mail or file transfer protocols. Narus partner products offer the ability to quickly analyze information collected by the Directed Analysis or Lawful Intercept modules. When Narus partners' powerful analytic tools are combined with the surgical targeting and real-time collection capabilities of Directed Analysis and Lawful Intercept modules, analysts or law enforcement agents are provided capabilities that have been unavailable thus far.
[edit] Usage
It is useful to examine the OSI model of seven layers which underpins all communication on the Internet. NarusInsight focuses on two layers: number four, the transport layer, built on standards like TCP and UDP, the physical building blocks of Internet data traffic, and number seven, the application layer, built on standards like HTTP and FTP, which are dependent on the application using them, i.e. Internet Explorer, Kazaa, Skype, etc. NarusInsight monitors 10 billion bits per second at level four and 2500 million bits per second at level seven. For reference, a 256K DSL line equals 0.25 million bits per second (Mbit/s), and a normal modem around 0.05 Mbit/s. So a single NarusInsight machine can monitor traffic equal to the maximum capacity of around 39,000 DSL lines or 195,000 modems. In practical terms 10 Gbit/s equals the combined traffic of millions of broadband users, so the number of subscribers monitored by one installation is several millions. It can also perform sem! antic analysis of the same traffic as it is happening, in other words analyze the content, meaning, structure and significance of this entire traffic, as it is happening.
The exact use of this data is not fully documented. A starting point is the Internet Protocol Detail Record, used to record information about usage activity within the telecom infrastructure (such as a call completion). NDM-U stands for "Network Data Management - Usage". It refers to a functional operation within the Telecom Management Forum's Telecom Operations Map. The NDM function collects data from devices and services in a service providers network. Usage refers to the type of data which is the focus of this document. These standards were built into Narus' systems.
"IPDR.org has been in existence since 1999 and more than a dozen vendors have actual IPDR implementations "etched in code". Their systems are actually able to talk to each other and interoperate. Version 2.5 and up of the NDM-U represents a stable basis for development. IPDR.org's Interoperability Pavilion is a working demonstration of multiple companies exchanging service usage data in that format."
Service usage data. That would be data on the actual usage of the Internet. And what kind of data would this be? Way back in 1999, this article stated:
"In an effort to provide more complex network traffic analysis, Narus is introducing its semantic network traffic service. The company cites research which predicts the fast-growing ISP sector will become stagnant without the ability to offer differentiated services. In order to gain significant revenues from these services, a technology was necessary to allow usage based pricing. "We realized that, at the heart of the data that is needed to accurately measure usage and enable 'pay-as-you-go' business models for Internet service providers, is what we call the 'semantics' of network traffic," said Ori Cohen, Narus' founder and chief executive officer. "In short, by seeing the 'semantics' of network traffic, service providers can see 'inside' the data, providing much more detailed insight about the use of the Internet and the perceived value of specific applications than existing technologies allow." "Semantic Traffic Analysis uses network technology to consistently capture and analyze all IP data streams on heavily trafficked networks remotely and non-invasively. In addition, the semantics of the data stream are determined also, as well as the protocol used and the application taking place. A variety of other data is available as well."
In this context, semantics is not just the data, but rather the meaning of the data. It looks at the data in a more comprehensive way than looking for keywords. Each NarusInsight machine does this at 2500 million bits per second, in real-time.
One website calls this "the biggest invasion of privacy in history by several orders of magnitude."
From Narus' Lawful Intercept and Regulatory Compliance page:
"Explosive Internet growth in recent years has transformed worldwide communications, yielding tremendous efficiencies and benefits, as well as many risks." "For example, terrorist attacks around the globe have been carefully orchestrated through Internet-based forms of communications such as e-mail, messaging, hidden Web pages and now VoIP, forcing governmental organizations and law enforcement agencies to re-evaluate how they are providing public security as it becomes so much easier and faster to communicate electronically." "Recent mandates and the resulting standards referenced under CALEA in the United States and ETSI in Western Europe aim to preserve the right of law enforcement agencies to conduct authorized electronic surveillance in an effort to protect the public and its right to privacy. However, these mandates create IT headaches for carriers as they struggle to meet the requirements." "With a suite of products targeted at meeting lawful intercept requirements, Narus simplifies lawful intercept tasks helping carriers and agencies meet requirements without experiencing any degradation in service quality."
-- Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
_________________________________________________________________ Make every IM count. Download Messenger and join the im Initiative now. Its free. http://im.live.com/messenger/im/home/?source=TAGHM_June07
On 6/14/07, Tyler Durden <camera_lumina@hotmail.com> wrote:
Shee-IT. Layer 4 packet inspection at OC-192 is kinda surprising, but Layer 7 at OC-48 is for me the more difficult thing to swallow.
you gotta love fast asic's for this kind of stuff. cloudstream also has success with the fpga approach. (there's a grad paper somewhere that describes a 10GigE inspection setup using fpga's and capable of ~100-600 snort style rules per chip. more rules == linear scale. would be fun to try L7, which does make things more difficult...)
Another thing worth thinking about is the control channels they must use to update the policies to one of these boxes. It's obviously in-band. One wonders if one could tap one of the fibers and find the packet stream they use to program one of these things.
what makes you say this? i'd be surprised if the control channel is pulled from the monitored flows. you need bi directional transport, for control and backhaul, among other reasons. maybe we'll find out when congress/judiciary orders the devices removed? *cough*
Another thing worth thinking about is the control channels they must use to update the policies to one of these boxes. It's obviously in-band. One wonders if one could tap one of the fibers and find the packet stream they use to program one of these things.
what makes you say this? i'd be surprised if the control channel is pulled from the monitored flows. you need bi directional transport, for control and backhaul, among other reasons.
maybe we'll find out when congress/judiciary orders the devices removed? *cough*
Hum...it's interesting to think about this. I assumed the control channel would be in-band for several reasons, all of which may be wrong. Let me first of clarify, in case it wasn't clear: I'm talking about downloading the policies that will 'program' what the Narus box looks at and how it will respond. The Narus box itself likely needs it's own control channel to upgrade its own software and do OAM&P, and this will probably be over the SONET DCC overhead. But the policies itself, I think, could be in band. Consider: 1) The Narus box already does layer 4: Since it's already opening up the STS-Nc container and reading the packets, seems trivial for them to grab their own control stream out of that. 2) Depending on the architecture, if the packets are in-band then they don't need to worry about getting their control channel terminated by putting it into SONET overhead. Of course, the path overhead might actually survive untouched the whole way, but that would prevent them from terminating at an intermediate router (which they might want the option to do so as to prevent backhauling a whole nation's worth of traffic). 3) Although not a BIG deal, if they used SONET overhead they would have to put their channel into unused overhead bytes. Some chipsets do that, but it's a constraint better avoided for various reasons (including rare interoperability issues if someone else along the way is using the same bytes for something). I don't understand the comment about bidirectional transport...this is necessary anyway, no? At least the DCC of SONET NEs need bidirectional or the SONET router (yes, there's a tiny OSI router inside SONET SEs) will declare the DCC down. Or maybe I misunderstand you... Of course, some of these considerations go away somewhat if NSA simply backhauls all the traffic over a proprietary coast-to-coast optical network, which is not inconceivable. -TD _________________________________________________________________ Play games, earn tickets, get cool prizes. Play nowit's FREE! http://club.live.com/home.aspx?icid=CLUB_hotmailtextlink1
On Thu, Jun 14, 2007 at 06:03:18AM -0400, Tyler Durden wrote:
Well, the need for optical amplifiers is not so much a power availability issue so much as a complexity issue. If one has dozens or hundreds of fibers, remember that each of these is going to be some random wavelength
Since you're about the only authority on WAN networking I know, and I've been slowly starting using low-end (GBIC based monomode and multimode WAN) recently, I have the following questions for you: a stupid way to wire up the neighbourhood with fiber is connect every household with medium converter or a GBIC to a large switch, and once you're running out of ports to build a tree of those. However, I've only come across some 8-16 porters which take optical input and are cheap. Do you use vanilla large switches, and use media converters for each, something like a large patch panel, with glass coming in, and copper going out? Is there a special device class for residential fiber Ethernet, and if yes, how much do these things cost? If you don't want to use routers, one has to use trees of switches. Newer switches can manage redudant links/loops with spanning-tree, and similiar. Is there a way to mesh up a tree of switches not using a real router? (Even though a low-end box like a X2100 M2 can probably saturate all onboard GBit ports, and then some 4 more), at wire-speed. -- Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
On 6/14/07, Eugen Leitl <eugen@leitl.org> wrote:
... a stupid way to wire up the neighbourhood with fiber is connect every household with medium converter or a GBIC to a large switch, and once you're running out of ports to build a tree of those.
s/stupid/expensive/ :)
Is there a special device class for residential fiber Ethernet, and if yes, how much do these things cost?
passive optical networking is used, so you get cost advantages of a point to multi-point last mile distribution architecture. you can still easily achieve OC3 to each endpoint, so bandwidth per customer is of little concern...
If you don't want to use routers, one has to use trees of switches. Newer switches can manage redudant links/loops with spanning-tree, and similiar. Is there a way to mesh up a tree of switches not using a real router?
if an ATM switch isn't a real router, then sure! *grin*
On Thu, Jun 14, 2007 at 08:34:58AM -0700, coderman wrote:
s/stupid/expensive/ :)
I'm not a believer in a tree model, so I'm interested in exploring residential-scale meshes with cheap (a GBIC is 100 EUR, or so, and stackable managed 48-port Netgears are 700 EUR, or around).
Is there a special device class for residential fiber Ethernet, and if yes, how much do these things cost?
passive optical networking is used, so you get cost advantages of a point to multi-point last mile distribution architecture. you can still easily achieve OC3 to each endpoint, so bandwidth per customer is of little concern...
I'm interested in 1-10 GBit throughput for each individual customers to any other individual customer, at least on the local loop. Throughput would decline over increased distance, of course.
If you don't want to use routers, one has to use trees of switches. Newer switches can manage redudant links/loops with spanning-tree, and similiar. Is there a way to mesh up a tree of switches not using a real router?
if an ATM switch isn't a real router, then sure! *grin*
I understand ATM mesh is an obsolete technology, with very limited throughput. I heard something recently about Foundry switches with have an advanced kind of STP, with active loops, not meshes, though. Probably there are no commercial devices on the market which would fit the description.
I understand ATM mesh is an obsolete technology, with very limited throughput. I heard something recently about Foundry switches with have an advanced kind of STP, with active loops, not meshes, though.
Probably there are no commercial devices on the market which would fit the description.
See my previous post. As for ATM being 'obsolete', well one might say this with good reason, but not because it isn't used. The core is still dominated by ATM. Waitaminue, not just the core but the edges too: DSL is basically over ATM still. As for bandwidth, that isn't a problem for ATM, though it's easy to bitch about the mapping penalties incurred by putting packets over ATM cells over OC-N. On the other hand, having the ATM layer in there can complicate matters, and you don't need a lot of flow control if you're willing to give everyone globs of bandwidth. For both of these reasons and more G-PON is probably going to win out, and I'd suspect prices have already started to drop precipitously. -TD _________________________________________________________________ Need a break? Find your escape route with Live Search Maps. http://maps.live.com/default.aspx?ss=Restaurants~Hotels~Amusement%20Park&cp=33.832922~-117.915659&style=r&lvl=13&tilt=-90&dir=0&alt=-1000&scene=1118863&encType=1&FORM=MGAC01
Well, this barely counts as WAN! My expertise increases exponentially as you get to the core. However, I do happen to know a small amount here, and the issues are numerous and tricky. Let's go for the main question:
Is there a special device class for residential fiber Ethernet, and if yes, how much do these things cost?
Well, basically yes: It's called PON (Passive Optical Network) and there are several flavors. The original was A-PON, wherein a home-based ONT that sits on the outside of a residence outputs ethenet into the home and maps the packets onto ATM over OC-12 upstream (as I remember) and OC-48 down. It's a weird technology in that a single OC-48 is split lots of times (32, I think) and each ONT just grabs the ATM VPI that corresponds to its owner (upstream is even weirder: All of the upstream ONTs time their burst-mode output so as to glue together a single coherent OC-12 upstream). The real winner here is probably going to be GPON, which is basically an optically split ethernet. Each ONT is given a 'window' wherein it can broadcast upstream packets. I'm not too in touch with this technology, but I'd suspect a single GbE port on the headend switch/router can support 64 endusers. And often, these endusers can be an apartment building with a switch in the basement. If you only have a single fiber coming out of the headend, then the 'obvious' upgrade is to use Coarse-DWDM which can easily provide up to 16 cheap wavelengths. In other words, that's 16 GbE ports (no, the PHY is not quite the same as the GBICs have to be burst mode). BT and other big companies have been building out fiber-to-the-home architectures using PONs, and I think I've heard prices quoted for the ONT as low as $250, but I could be wrong. Of course, there are lot of people who don't like PON, and depending on how much fiber you've got in the nieghborhood and the quality of the copper in the last mile, PON might easily not deliver as much bandwidth as a hybrid approach. -TD
From: Eugen Leitl <eugen@leitl.org> To: Tyler Durden <camera_lumina@hotmail.com>, cypherpunks@jfet.org Subject: Re: [IP] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Cas Date: Thu, 14 Jun 2007 12:50:53 +0200
On Thu, Jun 14, 2007 at 06:03:18AM -0400, Tyler Durden wrote:
Well, the need for optical amplifiers is not so much a power availability issue so much as a complexity issue. If one has dozens or hundreds of fibers, remember that each of these is going to be some random wavelength
Since you're about the only authority on WAN networking I know, and I've been slowly starting using low-end (GBIC based monomode and multimode WAN) recently, I have the following questions for you:
a stupid way to wire up the neighbourhood with fiber is connect every household with medium converter or a GBIC to a large switch, and once you're running out of ports to build a tree of those. However, I've only come across some 8-16 porters which take optical input and are cheap. Do you use vanilla large switches, and use media converters for each, something like a large patch panel, with glass coming in, and copper going out? Is there a special device class for residential fiber Ethernet, and if yes, how much do these things cost?
If you don't want to use routers, one has to use trees of switches. Newer switches can manage redudant links/loops with spanning-tree, and similiar. Is there a way to mesh up a tree of switches not using a real router? (Even though a low-end box like a X2100 M2 can probably saturate all onboard GBit ports, and then some 4 more), at wire-speed.
-- Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
_________________________________________________________________ Like puzzles? Play free games & earn great prizes. Play Clink now. http://club.live.com/clink.aspx?icid=clink_hotmailtextlink2
Well, this barely counts as WAN! My expertise increases exponentially as you get to the core. However, I do happen to know a small amount here, and the issues are numerous and tricky. Let's go for the main question:
Is there a special device class for residential fiber Ethernet, and if yes, how much do these things cost?
Well, basically yes: It's called PON (Passive Optical Network) and there are several flavors. The original was A-PON, wherein a home-based ONT that sits on the outside of a residence outputs ethenet into the home and maps the packets onto ATM over OC-12 upstream (as I remember) and OC-48 down. It's a weird technology in that a single OC-48 is split lots of times (32, I think) and each ONT just grabs the ATM VPI that corresponds to its owner (upstream is even weirder: All of the upstream ONTs time their burst-mode output so as to glue together a single coherent OC-12 upstream). The real winner here is probably going to be GPON, which is basically an optically split ethernet. Each ONT is given a 'window' wherein it can broadcast upstream packets. I'm not too in touch with this technology, but I'd suspect a single GbE port on the headend switch/router can support 64 endusers. And often, these endusers can be an apartment building with a switch in the basement. If you only have a single fiber coming out of the headend, then the 'obvious' upgrade is to use Coarse-DWDM which can easily provide up to 16 cheap wavelengths. In other words, that's 16 GbE ports (no, the PHY is not quite the same as the GBICs have to be burst mode). BT and other big companies have been building out fiber-to-the-home architectures using PONs, and I think I've heard prices quoted for the ONT as low as $250, but I could be wrong. Of course, there are lot of people who don't like PON, and depending on how much fiber you've got in the nieghborhood and the quality of the copper in the last mile, PON might easily not deliver as much bandwidth as a hybrid approach. -TD
From: Eugen Leitl <eugen@leitl.org> To: Tyler Durden <camera_lumina@hotmail.com>, cypherpunks@jfet.org Subject: Re: [IP] EFF: Secret Surveillance Evidence Unsealed in AT&T Spying Cas Date: Thu, 14 Jun 2007 12:50:53 +0200
On Thu, Jun 14, 2007 at 06:03:18AM -0400, Tyler Durden wrote:
Well, the need for optical amplifiers is not so much a power availability issue so much as a complexity issue. If one has dozens or hundreds of fibers, remember that each of these is going to be some random wavelength
Since you're about the only authority on WAN networking I know, and I've been slowly starting using low-end (GBIC based monomode and multimode WAN) recently, I have the following questions for you:
a stupid way to wire up the neighbourhood with fiber is connect every household with medium converter or a GBIC to a large switch, and once you're running out of ports to build a tree of those. However, I've only come across some 8-16 porters which take optical input and are cheap. Do you use vanilla large switches, and use media converters for each, something like a large patch panel, with glass coming in, and copper going out? Is there a special device class for residential fiber Ethernet, and if yes, how much do these things cost?
If you don't want to use routers, one has to use trees of switches. Newer switches can manage redudant links/loops with spanning-tree, and similiar. Is there a way to mesh up a tree of switches not using a real router? (Even though a low-end box like a X2100 M2 can probably saturate all onboard GBit ports, and then some 4 more), at wire-speed.
-- Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE
_________________________________________________________________ Get a preview of Live Earth, the hottest event this summer - only on MSN http://liveearth.msn.com?source=msntaglineliveearthhm
participants (3)
-
coderman -
Eugen Leitl -
Tyler Durden