Real-Time Botnet Threat Intelligence Data Feed Offered By Microsoft

Microsoft is planning to make the data it collected as part of its botnet takedown operations available as a real-time threat intelligence feed.

The company would distribute threat data obtained from captured botnets and other sources to foreign governments, law enforcement, Computer Emergency Response Teams and private corporations, two members of the Microsoft Digital Crimes Unit told attendees at the International Conference on Cyber-Security at Fordham University.

Partners would be able to access the information using application program interfaces (APIs) that would be provided free by Microsoft.

Joining up the bots

When the real-time threat intelligence data service goes live, Microsoft plans to offer three different feeds. These would enable partners to look for malware infections that often are part of botnet activity, or correlate host data with information on various Internet scams, such as click fraud, to connect the dots between them, according to T J Campana, a senior programme manager in the Microsoft Digital Crimes Unit. Microsoft declined to say when the new real-time feed would go live.

Sharing threat intelligence data was an oft-recurring theme at ICCS. Better co-ordination and information-sharing between law enforcement, security researchers and private organisations is essential in the battle against cyber-criminals, Barry Green, the president of the Internet Systems Consortium, told attendees on the first day of the conference. While public-private partnerships are essential, there needs to be more data-sharing and private-private collaboration within the industry, according to Green. RSA Netwitness already offers a real-time data feed of network threat intelligence data collected from multiple sources.

“We collect a tremendous amount of data from our global assets,” said Campana.

Microsoft has been collecting botnet data from a number of sources. The company’s vast Internet infrastructure, which includes a global load-balanced, 80Gbps network, can take over whole botnets and point the infected hosts to servers that Microsoft controls, according to Campana. This way, Microsoft is able to capture the botnet activity for its data collection purposes while effectively stopping the malicious traffic.

Data obtained during the recent operations that helped shut down the Kelihos, Rustock and Waledec botnets in the United States would also be included in the service. The takedowns, such as the operation for Rustock in March 2011 and the Waledec campaign in 2010, were a significant source of data, according to Richard Domingues Boscovich, a senior attorney in the Microsoft DCU who spoke during the same session as Campana. Microsoft has been collecting the data for three years, using a manual process and working with various partners such as CERT, according to Boscovich.

The data harvested from the Kelihos botnet back in September is already stored in the system, which includes IP addresses of infected systems, Autonomous System (AS) numbers as assigned by regional Internet registries (RIR) and reputation data provided by Microsoft’s Smart Data Network Services, Campana said. Microsoft will not include the personally identifiable information that had been collected by the botnet in the threat intelligence feed, according to Campana.

Botnet library services

Microsoft partners wanted a way to access the botnet data Microsoft had accumulated in a faster and more efficient manner, which provided the impetus for the data service. Once live, the service would segment the data, and only the relevant pieces of data will be shared with the partner. For example, national or regional CERTS would be more likely to prefer seeing threat intelligence related to their geographic area. The service will also give small organisations the security intelligence necessary to battle large global botnets without having to invest in complex analysis and forensics or engaging the professionals capable of using those tools.

Microsoft has been beta-testing the system internally, according to Campana. The system appears to be a 70-node cluster running Apache’s Hadoop software framework on top of Windows Server, Campana said. The open-source Hadoop software allows organisations to work with petabytes of data and thousands of nodes.

The third annual International Conference on Cyber Security: A White Hat Summit was a joint effort between the Federal Bureau of Investigation and Fordham University. Leaders from law enforcement, industry and academia discussed cyber-crime and real-life operations during the conference, which ran from 9-12 January at the Fordham University campus in New York.