This post is authored by Jaya Mathew & Hong Lu, Data Scientists at Microsoft, in collaboration with the team at ServusNet.
Governments worldwide are actively promoting the use of renewable energy because of the growing concern over climate change and carbon emissions. Among the available renewable energy sources, harnessing energy through wind turbines offers a compelling alternative. There are a growing number of wind farms throughout the world now, and, as the industry matures at a rapid pace, it is increasing its focus on operational efficiency and cost-competitiveness.
Given the inherent variability in wind energy production, the ability to analyze the data generated by wind turbines to characterize historical behavior and create a framework to predict future performance can be a critical asset and business differentiator.
ServusNet is a software company that develops Operations & Maintenance (O&M) and Operational Intelligence (OI) solutions for the renewable energy industry. The growth in the renewable energy sector is driving the need for sophisticated software tools to efficiently manage larger portfolios of wind and other distributed energy generation technologies. The ServusNet team has many years of experience in telecom network management and is building on this expertise to deliver innovative operations software products designed to reduce operating costs, improve production efficiency and increase planning accuracy. This post outlines Microsoft’s collaboration with ServusNet to migrate their existing wind power forecasting model to the Azure Cloud.
ServusNet currently has an on-premises solution which uses daily weather forecasts to generate the final wind power forecasts at a farm level. ServusNet would now like to deploy an end to end operationalized cloud-based solution so that they can scale out their product offering, catering to multiple wind farm portfolios and customers globally.
Cortana Analytics Suite was used to build this scalable, cloud based solution. Azure Web App was used to retrieve the weather data from a remote server and the data was then stored in cost-effective Azure Blob storage. The easy to use user interface in Azure ML was used to quickly replicate the functionality of their existing turbine wind power forecasting algorithm. The performance of the final cloud-based model matched that of ServusNet’s on-prem solution. At a click of a button, ServusNet was able to wrap their final experiment in a web service API that could then be consumed by various devices and platforms. Azure Data Factory (ADF) was used to orchestrate data movement and invoke the Azure ML web services at a predetermined schedule for similar wind farm portfolios.
The end to end pipeline as shown below:
End-to-end automated Azure pipeline
This data pipeline can be easily scaled for a large number of wind farms or portfolios using the Azure Resource Management template. Alternatively, this generalized pipeline can be programmatically reused using the ADF.NET SDK. The Azure Management Portal enables ServusNet to monitor their solution in real time, as show in the image below. An alert mechanism was set up to notify operators in the event of unexpected failure, and the underlying problem can then be backtracked on the Azure Management Portal.
Monitoring the process in the Azure portal
With their advanced ML algorithm embedded in Azure ML, ServusNet is now able to provide a world-class, reliable and scalable cloud-based solution that predicts the power generated by wind farms. Using their Cortana Analytics -based framework, ServusNet also has access to additional big data components, for instance the Azure SQL Data Warehouse, when more wind farm data becomes available – this ensures that they are all set to customize and extend their existing solution for much larger wind farm portfolios but with minimal incremental development costs.