With almost every business going through a period of digital transformation right now, many are using the latest generation of Microsoft data platform technologies to help them store and analyse their data. While that may not sound like anything new, the types and volumes of data they handle are often quite different compared to just a few years ago. So too are their requirements for agile and scalable services with innovative feature sets. SQL Server, along with Azure’s data services, have now matured to the extent that they’re now regularly being used to power everything from a developer’s prototypes to a SaaS vendor’s global-scale services.
What is digital transformation?
Digital transformation describes the period of commercial reinvention, powered by new technologies, that businesses are currently going through. It’s changing what businesses sell and how they engage with their customers. While this reinvention often introduces new technologies, digital transformation is primarily about business innovation.
Imagine you were an ERP software vendor. For many years, your customers would have installed your software on their servers in their data centres and their PCs in their offices. Once or twice a year, you would send them updates which might upgrade version 4.0 of your software to 4.5 or even 5.0. Your customers would probably have paid a large one-off licence fee in year one, followed by smaller support fees in the following years. In this situation, digital transformation isn’t about simply shipping v6 or even a completely new v10. It would be about rewriting your ERP software to make it a web based SaaS application service that you host and your customers then access over the internet. It would also be about you becoming responsible for updating and upgrading the software your customers use – potentially monthly or even weekly - rather than them. Perhaps most importantly, it could be about you charging your customers on a per-user per-month basis, rather than for a one-off licence. Here we can see not just the software has changed, but also the customer experience and the software’s commercial model.
Thinking about how the ERP software would be deployed in this scenario, we most likely start imagining application platforms hosted with a cloud services provider, such as Microsoft with its Azure services. The ERP vendor is mostly likely to want to pay monthly for services to align this to how it charges its customers. They’re also likely to want a scalable platform so they only pay for the capacity they need but know that extra capacity is available when they need it. Finally, they’re going to want to have a functionally rich platform to avoid them having to source and operate third party technologies to perform what they consider common tasks.
Microsoft’s digital data platform
Microsoft’s portfolio of data services is now as broad as it is deep. Regardless of what format or volume of data an application platform needs to store and analyse, the Microsoft data platform now has an option for it:
- Transactional structured data - Azure SQL Database or SQL Server
- Transactional semi-structured data - DocumentDB
- Archive structured data - Azure SQL Data Warehouse or SQL Server
- Archive semi-structured data – HDInsight or Azure Data Lake
When data needs to be transferred between systems, then are the Azure Data Factory and SQL Server Integration Services platforms to choose between.
When data needs to be analysed, there are several options each with their own strengths: Azure Analysis Services, SQL Server Analysis Services, Azure Data Lake and HDInsight. To complement that layer of analysis, there is Microsoft R Server and support for Python in SQL Server 2017 to support data science.
Finally, for data visualisation, Microsoft has its Power BI and SQL Server Reporting Services platforms.
Supporting digital transformation
It’s clear that SQL Server and Azure’s data services now have more technologies than most people could name, but how do they support digital transformation? To answer that, we should look at three specific capabilities they provide.
Rapidly deploying or destroying cloud services has been the norm in the application tier space for a long time, but now the same capabilities exist for data services. Unlike most application services though, the capacity of these rapidly deployable data services can be immense. It’s now possible for a data scientist to trial new queries on a 2TB data set using their own copy of the data, potentially for tests lasting just a few hours. An application platform can now also automate the deployment of additional database, data services, or data loading processes every time a new customer signs up for a SaaS service. This level of agility available in the data tier of cloud applications means the whole stack can be managed as an agile asset whereas for a long time, data platforms could be slow and clunky movers.
Being able to pay for data storage as it’s required, rather than up front for all that might ever be needed, helps a data platform’s costs align to the digital era’s subscription based commercial models. If data services can have storage added to them as and when they need it, costs can be aligned to platform usage. This scalability on demand doesn’t though compromise the quality or performance of computer or storage resources Microsoft’s immense Azure platforms allow for data services to have terabytes of fast, resilient, and often replicated storage added to them with the click of a mouse – rather than the rev of a delivery truck with more disks.
From a data querying perspective, the format of data has probably changed more in the last few years than in the previous few decades. Capturing gigabytes, if not terabytes, of semi-structured data can now be the norm for retailers. Likewise, transactional finance applications now often support real-time analytics as well as end user processes. The flexibility of SQL Server and Azure’s data services now mean there are often storage and querying engines for every time of data and every volume of data. In fact, the lines of their capabilities are blurring. SQL Server, which is a structured database engine, can now query semi-structured big data. Likewise, Microsoft’s big data technologies, such as Azure Data Lake, are as comfortable querying structured data as they are semi-structured data.
The breadth of the Microsoft data platform then along with the agility, scalability, and functionality of its services now provides a data storage and analysis tool for most if not all modern application platforms. Application platforms that often support the business innovation happening around them, whether that’s new internally facing applications or externally facing revenue generating systems. Either way, the days of the data centre hosted static n-tier application systems are coming to an end.
- For an example of how a global business is digitally transforming, this summary of a business school lecture by L’Oréal’s chief digital officer gives a good overview - http://www.sbs.ox.ac.uk/school/news/revolution-will-be-digitised
- To see how Microsoft’s strategy is supporting its customers with the technical aspect of their digital transformation, this concise summary is a good read - https://enterprise.microsoft.com/en-us/digital-transformation/
- For an overview of what’s new in SQL Server 2017, Microsoft has recently published this summary - https://www.microsoft.com/en-us/sql-server/sql-server-2017
- Finally, to read more about some of the Azure data services, there is the Azure product directory - https://azure.microsoft.com/en-gb/services/?sort=&filter=databases