Turning Big Data into Big Insights
Posted by Kerry Godes
Senior Manager, Worldwide Marketing and Operations
Big data is the term increasingly used to describe the process of applying serious computing power, even the latest in machine learning and artificial intelligence, to massive and often highly complex sets of information. This week Microsoft is showcasing what customers, partners, and the industry are doing to harness the power of big data to change the way organizations and people do business, discover insights and interact with one another.
What exactly is big data? It’s data that is so large, so complex or collected at such a fast pace that it challenges the capabilities of traditional data management systems. Any organization that looks at their current data infrastructure and sees a significant shift in the volume, variety or velocity of data is experiencing big data.
“Our daily lives generate an enormous collection of data,” said Dan Vesset, Program Vice President of IDC's Business Analytics research. “Whether you’re surfing the Web, shopping at the store, driving your smart car around town, boarding an airplane, visiting a doctor, attending class at university, each day you are generating a variety of data. A lot of the ultimate potential is in the ability to discover potential connections, and to predict potential outcomes in a way that wasn’t really possible before. Before, you only looked at these things in hindsight.”
In other words, there may be oceans of data out there, but making it into something you can use, particularly in real-time, is another matter entirely.
Structured data, most universally found in databases that use Structured Query Language (SQL), is organized in a way that lets users select exact pieces, rows or columns of that database – perhaps you’ll select all of the rows with a certain zip code, or the columns with a specific date. Unstructured data, however, has no such architecture and can often include text or images that aren’t part of the free-form data (emails, for example).
That’s why Microsoft is working with Hadoop, an open source data platform that helps manage unstructured data, to integrate Hadoop with SQL Server and Windows Azure and help customers combine all their data sources, structured and unstructured. Customers like Klout, a data services company that offers detailed insights from the online activity of more than 100 million people, are already benefiting from a combined Hadoop-SQL Server business intelligence solution today.
“What we’re trying to do is allow a broad set of skills, driving simplicity and ease of use into the area of big data,” said Eron Kelly, General Manager of Product Marketing for Microsoft SQL Server. “Taking very complex technical problems and simplifying them with easy-to-use tools – that’s been the Microsoft strategy over the last 30 years.”
And as many grapple with the “what” and “why” of big data, customers like The Weather Company (TWC), parent company of The Weather Channel, are beginning to lead the way, delivering better, faster insight. TWC knows firsthand that weather is at the core of many decisions that people make on a daily basis.
“Weather is probably the biggest big data platform. It plays a role in how you work, how you live, how you play and how you shop,” says Bryson Koehler, Chief Information Officer at The Weather Company. “It impacts a significant portion of the world’s activity, and big data is about understanding how consumer behavior intersects with and is influenced by weather. With big data, we are helping our customers understand the data and take action in real-time.”
You can read our full interview with Koehler in Microsoft’s Big Data Newsroom, as well as learn more about this week’s big data news and events.