Openness Customer Spotlight: Klout Uses Microsoft BI and Hadoop to Bolster Big Data Insights

  • Article

Posted by Kerry Godes
Senior Manager, Worldwide Marketing and Operations

Social networking accounts for 1 of every 6 minutes spent online, generating an enormous amount of content that could be useful to consumers, schools, governments and corporations – almost anyone interested in measuring trends – if only there were a way to harness it.

Enter Klout, a data services company, which offers fast, detailed insights from an index of the online activity of 100 million people, more than 2.7 billion pieces of content daily and 30 billion API calls per month on 15 leading social networks.

From these hundreds of terabytes of data, Klout customers and partners get nearly real-time access to information on what people are doing, searching and talking about on their social networks. “We take in raw data and make it into something that is actionable for our consumers, brands, and partners,” says David Mariani, Vice President of Engineering at Klout. 

Above video: Klout discussing SQL Server and Apache Hadoop solution

Founded in 2007, Klout still operates like a start-up and its founders are passionate about giving developers freedom to innovate. Its business intelligence solution was updated in 2011 to combine Microsoft SQL Server 2012 and open source Apache Hadoop.

“Klout chose the best of both worlds by marrying the Microsoft BI platform with Hadoop and Hive,” Mariani said. Based on previous experience with the Microsoft BI platform, Klout analysts knew that SQL Server offers extensive compatibility with third-party software and could manage big data sets. This gave Klout the ability to process data with an average query response time of less than 10 seconds while reducing cost and boosting efficiency.

“By using open source and commercial software, we don’t have to build everything from the ground up and we get a great ecosystem of tools and support,” Mariani said. For example, Klout will spend less time managing business logic and data connections. “By creating a single warehouse on Hive, and putting all of our business logic in SQL Server 2012 Analysis Services, we can easily expose all of the data that was previously tucked away in Hadoop.”

To learn more about how Klout is using a combination of Microsoft and open source technologies, read the full case study here.