The N.S.A. documents show that one of the main tools used for chaining phone numbers and e-mail addresses has the code name Mainway. It is a repository into which vast amounts of data flow daily from the agency’s fiber-optic cables, corporate partners and foreign computer networks that have been hacked.
The documents show that significant amounts of information from the United States go into Mainway. An internal N.S.A. bulletin, for example, noted that in 2011 Mainway was taking in 700 million phone records per day. In August 2011, it began receiving an additional 1.1 billion cellphone records daily from an unnamed American service provider under Section 702 of the 2008 FISA Amendments Act, which allows for the collection of the data of Americans if at least one end of the communication is believed to be foreign.
The overall volume of metadata collected by the N.S.A. is reflected in the agency’s secret 2013 budget request to Congress. The budget document, disclosed by Mr. Snowden, shows that the agency is pouring money and manpower into creating a metadata repository capable of taking in 20 billion “record events” daily and making them available to N.S.A. analysts within 60 minutes.
The spending includes support for the “Enterprise Knowledge System,” which has a $394 million multiyear budget and is designed to “rapidly discover and correlate complex relationships and patterns across diverse data sources on a massive scale,” according to a 2008 document. The data is automatically computed to speed queries and discover new targets for surveillance.
A top-secret document titled “Better Person Centric Analysis” describes how the agency looks for 94 “entity types,” including phone numbers, e-mail addresses and IP addresses. In addition, the N.S.A. correlates 164 “relationship types” to build social networks and what the agency calls “community of interest” profiles, using queries like “travelsWith, hasFather, sentForumMessage, employs.”
A 2009 PowerPoint presentation provided more examples of data sources available in the “enrichment” process, including location-based services like GPS and TomTom, online social networks, billing records and bank codes for transactions in the United States and overseas.
At a Senate Intelligence Committee hearing on Thursday, General Alexander was asked if the agency ever collected or planned to collect bulk records about Americans’ locations based on cellphone tower data. He replied that it was not doing so as part of the call log program authorized by the Patriot Act, but said a fuller response would be classified.
If the N.S.A. does not immediately use the phone and e-mail logging data of an American, it can be stored for later use, at least under certain circumstances, according to several documents.
One 2011 memo, for example, said that after a court ruling narrowed the scope of the agency’s collection, the data in question was “being buffered for possible ingest” later. A year earlier, an internal briefing paper from the N.S.A. Office of Legal Counsel showed that the agency was allowed to collect and store raw traffic, which includes both metadata and content, about “U.S. persons” for up to five years online and for an additional 10 years offline for “historical searches.”