This is part of a series of blog posts about the Microsoft data platform and advanced analytics opportunity for partners. Select Data & Analytics to read the series. Subscribe to this blog by email or RSS.
US Partner Technology Strategist
Many businesses today run their data platform infrastructure using on-premises hardware, on either physical servers or in a virtualized environment. As the benefits of the cloud continue to emerge, IT organizations are tasked with evaluating whether and how data platform applications can utilize the cloud. Customers are understandably cautious about relying solely on the cloud for their data platform needs. The good news is that customers can take advantage of the cloud without moving their entire platform. Just as a hybrid car takes a “best of both worlds” approach and helps save on fuel usage and costs, the hybrid cloud capabilities that are built into SQL Server 2016 can help offset IT costs without everything being moved to the cloud.
This hybrid approach to the data platform is a compelling story for customers as well as for partners with on-premises data platform practices that want to extend those practices to cloud. SQL Server 2016 will be generally available on June 1, and its capabilities open up many new partner opportunities, including with hybrid scenarios.
In this post, I’ll outline some of the hybrid scenarios built into SQL Server 2016 and explain how you can get started with each of them.
1. Migrate historical data transparently and securely to the cloud using Stretch Database
In many organizations, data can be categorized into different tiers based on their age. Typically, historical data is not frequently accessed by business applications, and is kept for compliance or other application-specific reasons. Yet these data may need to be readily accessible in case of an audit, for example, or for use by business application that do historical trend analysis. Storing and managing historical data in the same way as current data are stored and managed is not cost-effective. The cloud can be a good solution for storing and managing historical data—if it also allows that data to be queried and accessed quickly if and when needed.
Stretch Database utilizes the resources in Microsoft Azure to offload archival data storage and query processing. Once enabled at the table level, Stretch Database silently migrates your historical data to Azure, ensuring that no data are lost during migration. Stretch Database uses processing power in Azure to run queries against remote data by rewriting the query.
2. Access distributed big data using PolyBase
With the advent of big data and technologies like Hadoop, the cloud is often used to store vast volumes of unstructured data that are not suitable to be stored in a traditional RDBMS data store. This presents a challenge when analyzing this dataset, as some of the information required for analysis may be stored in an on-premises RDBMS data store like SQL Server.
For example, someone analyzing customer buying patterns may need to combine customer information from their CRM system stored in SQL Server database with information about the same customers from social platforms like Twitter and Facebook that’s stored in the cloud in a Hadoop system like Azure HDInsight. It is difficult to combine these datasets and relate them for meaningful analysis.
PolyBase is a technology that accesses and combines both non-relational and relational data, all from within SQL Server. It allows you to run queries on external data in Hadoop or Azure blob storage. The queries are optimized to push computation to Hadoop. You can do all of this simply, using the familiar T-SQL language—there’s no need to learn Java or other languages to query the Hadoop data. More importantly, there’s no need to bring that data from the cloud to an on-premises database system. The diagram below explains how PolyBase works.
3. Build reports in the cloud but keep data on-premises using Power BI Gateway – Enterprise:
Microsoft Power BI is a collection of software services, apps, and connectors that work together to turn unrelated sources of data into coherent, visually immersive, and interactive insights. The Power BI service is a SaaS offering that users can easily connect to their data sources, visualize or discover what’s important, and share those insights with others. Because it is a SaaS based solution, there is no setup or deployment and users can start building reports and dashboards in minutes. An issue customers face with a cloud solution is accessing data that may be in an on-premises data store like SQL Server or Oracle or that may be a file like an Excel worksheet. Power BI has connectors to a wide variety of data sources and so accessing this data is simple. The challenge is accessing this data without moving them to the cloud.
The Power BI Gateway – Enterprise is a windows service that acts as a bridge, providing quick and secure data transfer between the Power BI service and on-premises data sources. Data transfer between Power BI and the gateway is secured through Azure Service Bus. The Service Bus creates a secure channel between the Power BI service and your computer. Because the gateway provides this secure connection, there’s usually no need to open a port in your firewall.
The partner opportunity
If you haven’t yet spent time learning about SQL Server 2016, I encourage you to take a look at the product site and explore the resources there to understand the many new capabilities that are built in to provide customers with a consistent experience from on-premises to cloud.
If you are a Microsoft partner already delivering services for on-premises data platform solutions like SQL Server, the above hybrid scenarios can extend those capabilities into the cloud. You can focus on your existing strengths around on-premises services and provide additional value to customers by helping them save costs. The hybrid scenarios also let you start conversations with stakeholders outside of IT. For example, the Power BI gateway service is appealing to people in marketing, sales, and other business-focused roles who want to use data in a more compelling and beneficial way that doesn’t require a reliance on their IT departments, but that still meets security and compliance requirements.
Get started with these business and training resources
If you’re ready to start helping your customers implement a hybrid data platform solution using SQL Server 2016, here are my resource recommendations:
- Stretch Database: Leverage infinite storage and compute capacity in Azure with SQL Server 2016
- Data Exposed: SQL Server Stretch DB
- PolyBase: Gaining insights from semi-structured data from HDFS and relational data in SQL Server 2016
- Data Exposed: PolyBase in SQL Server 2016
- Data Management Gateway in Power BI for connectivity to on premise and hybrid scenarios