I believe that there is a switch that law enforcement has. I have no idea when it is right or proper to flip that switch. I have no idea how many times it has been abused. But I do have an idea what is behind that switch. As the Vietnamese hooker said "Any thing you want".
I'm talking about the hackers that hack the hackers. I'm talking about the NSA.
Here's what I believe about how 'anything' comes from a limited set of somewhat anonymous things. First of all, what I don't doubt is the skills of investigators who are in the business of CT. What I suspect is that they have gained legitimate access to non-anonymous stuff that can be named in a warrant, but that they also have shady access to anonymous stuff that is collected on the sly. In otherwords, like every other police force, they have admissible evidence and non-admissible evidence. The difference between the two can always be attributed to 'a hunch'. If you bust a jaywalker and destroy his lunch plans, you can find out who's sitting in the restaurant you know he was going to and casually observe who is checking their watch.
What I am not likely to find out is the exact record layout of the sort that Verizon is giving up to the Feds. But somebody knows. If I had a peak at those, I could tell you what kind of information is inferrable. What I believe happens is that network analysis is done to identify the suspect's contacts and then observations are made based on suspicions of who those individuals are. In other words, it's up to the investigating agency to de-anonymize those anonymous, non-content records since they will have some key even if the lookup to that key is not made available.
Take a cell phone call. What can I know? I can take Tsarnaev's cell phone and determine its identification. Then, knowing where T has been on the day of his crime, I can isolate all cell traffic within X radius of the cell phone towers. Know that basically all towers carry all traffic. Either you're CDMA or GSM, Verizon and AT&T have a trading backchannel clearinghouse where they exchange information about 'cross' traffic. So even if I'm looking at AT&T's tower, I could still have (what I remember to be) about 40% of other carrier's traffic.
So I figure out what I know to be true, and exclude what I know to be false. I can deduce a great deal. For example. Who has called Russia in the past 12 months? Believe me, that cuts down a whole lot of traffic. Who called Russia in the past 12 months that was in Boston in May? This kind of programming is very simple. The question is merely whether it comes from admissible sources and whether or not those who have secret inadmissible sources are clueing in investigators as to what they may have proven to be impossible. IE, a senior guy saying "I have a very strong hunch that it's not Suspect 22". Or "Have you considered the possibility that Suspect 22 might have been using the public phones over at Harvard Square?"
The more data sources, the more triangulation. For example. It doesn't take much to figure out that the likelihood of a person writing a post this long onto Facebook wasn't composed on a mobile device. So it would be easy to find confirmation or denial of a person being at home vs not at home. After all, if Facebook is passing weblogs with a GUID, it would be very burdensome for them to scrub out the IP addresses of the web clients. In fact, my nickel says weblogs are probably open season because it costs so much to process them that website owners would only pass them in a pure unadulterated form.
Let us not forget that hackers hack. An NSA without a botnet is unthinkable. An FBI without a favor from the NSA is also inconceivable, as is an Eric Holder who doesn't play fast and loose with domestic firewalls. Our Congress refused to sunset the Patriot Act. So there are many things government agencies can know and do know. And the cost to know gets lower every month.
So I'm just talking around what I believe - which is that the best way to avoid getting busted by the cops is to be a cop. The next best thing is to know what cops know, and try not to be surprised. When it comes to domestic surveillance, nothing surprises me. Nothing at all.