Developing a Cloud Capability Model


By Stephen Armory, Cloud Solution Architect at Microsoft

Executive Overview

The process of Cloud Adoption can be a daunting one, requiring both high-level strategic thinking and detailed, execution level understanding. Over the last few years there has been an explosion of offerings in the cloud arena, everything from Cognitive Services providing AI and Computer Vision through to the lowly Web Application Firewall. This leads us to the problem of how to develop a cloud strategy that is both meaningful to non-technical stake holders, but also useful to those tasked with its implementation.

We often see that big ideas can easily become detached from reality while an overly delivery focused agenda can be myopic in its scope. We can spend our time producing beautiful slide decks, detailed project plans and delightfully complex business process diagrams, but the problem with these approaches is that you still end up with a high risk of “Failure to Implement”. Their weakness, when used to drive a Cloud Transformation, is that they are typically very poor at delivering the following key requirements:

1: Easy to understand.
2: Able to encompass a strategic vision.
3: Technically meaningful.
4: Able to explain the implementation process.
5: Drive consistency of approach where appropriate.
6: Describe the skills required to develop and manage the platform.
7: Able to remain relevant and up to date.

Once technical and business requirements have been considered and worked through, how do we then communicate this within the organisation in a meaningful way? Several clients we have worked with have benefited from the following approach to “Cloud Capability Modelling”, in one instance still benefiting from and enhancing the original vision 4 years into their Cloud Transformation programme.

Introduction

The purpose of this document is to introduce an approach that I have been using with clients over the last four years to help address the issues above. The approach is intended to work alongside a “Cloud First” strategy, so you may recognise parts that have been “borrowed” from traditional approaches, and some may be new to you. The goal of this document is to help you build a vendor agnostic capability model.

As with all sensible methodologies, you have the freedom to pick and choose the parts that you think are useful. There is only one golden rule “If you can’t draw it, you can’t build it.”

Step 1: Gathering the required capabilities

From a cloud perspective there are several key capabilities that are considered as standard, which include things like Authentication, Operations Management, Secure Communications etc. It is recommended that you arrange a small number of meetings with stakeholders and IT staff to ensure that the list of requirements is complete and aligns with the business strategy going forward. Getting people to write the capabilities that they need on Post-It notes and put them on a wall is my preferred approach.

Next, work through the Post-It notes to remove duplicates or fine distinctions, and try to merge items that don’t materially impact the strategy or are technically identical. We’re trying to add simplicity at this stage. If your organisation has a universally known tool that isn’t going to be replaced any time soon, put it in, as it will help people understand what’s being designed.

Don’t worry if you’ve missed some as this is intended as a living document. Finally, under no circumstances use this as an opportunity to select products or services. The credibility of the Cloud Capability Model rests on its objectivity, and you don’t know what the best tools are at this stage.

Appendix A has a “Straw Man” list of capabilities to get you started, so feel free to crib from this list. I would recommend that you don’t rush to exclude items though, including something like Machine Learning in your strategy is sensible, even if you don’t have an immediate need for it.

Step 2: Draw Out “Cloud Capability Model" Version 1

Use of a design or presentation tool is required here. Follow your own approach - what you are aiming for is something visually appealing. Typically I would use Visio, though PowerPoint or Adobe In-Design could be used. See Appendix B for an example.

We can now use colours to separate out the location of services. Also, where a capability is present now, colour it in grey to denote its existence. Don’t be afraid to work in A3 or A2 sizes. Once drafted, we would look to gain agreement that this is what was discussed and then seek executive approval of the “Cloud Capability Model”. Explain that this is not going to be used to start a grand cloud roll-out process, merely a considered vision of the future. After all, if you don’t know where you are going, how can you get there?

Loop around this step until all parties agree, or at least until you have sign off. At this stage, I would recommend printing of a large/full-size copy to mount in a prominent location.

Step 3: Shortlist of Projects and Technologies

The next task is to go through your upcoming project list and identify, for example, your top 3 projects that will use the new architecture. These first 3 will be used to draw out the stages you will go through as part of your Cloud Transformation.

Each project will have dependencies on certain technologies:

  • Phase 1: A Data Warehouse project may need to utilise a Relational Data Warehouse, Semantic Layer, Data Lake, Orchestration Engine, Bulk Loading technology, Dashboarding and Ad-hoc Analytics.
  • Phase 2: A “Customer BOT” project may need a Relational Database, BOT Service, Language Understanding, B2C Authentication and Integration with your existing CRM system.
  • Phase 3: An IOT solution may require Edge Compute, Stream Analytics, Big Data Store, Device Management, Time series Analytics, Machine Learning and Dashboarding.

Each cloud vendor publishes several “Reference Architectures”, and these can help you identify suitable approaches. When building architectures on Azure, please visit the Azure architecture centre.

Note: The first project will need to enable key security, management and communications capabilities (AKA “a scaffold”). It’s therefore advisable to not pick an overly complex project for your first.

Step 4: Cloud and Service selection

Once you are satisfied that your Shortlist of Projects and Required Technologies is complete, it’s time to select your cloud vendor(s) and appropriate services. Depending on your organisations purchasing practises, you may have to either carry out a competitive tender or simply make a shortlist and select from that. My recommendation here is to bear in mind three key things:

  1. Do not select different technologies between projects. We are looking for consistent technology choices for each requirement. e.g. Do not use a Microsoft Orchestration engine on one project and Amazon's on another.
  2. You should consider sacrificing “best in class” for simplicity and consistency of vendor and experience. You wouldn’t mix Lego with Meccano unless you had good reason.
  3. If you do decide to adopt a multi-cloud approach, carefully consider “Data Egress Costs” and performance implications. All cloud providers will charge you for moving data out of their service. So, don’t have your data lake with a separate vendor to your big data compute because it could be expensive and slow.

Step 5: Updating your Cloud Capability Model

Once you have made your service selection, work through each project in your list to produce a Cloud Capability Model for them. Use what you know about the required capabilities and associated data flows for each project to develop a variant of the Cloud Capability Model created in Step 2. You may want to use a colour to denote the new capability being “Activated”; I’ve chosen red. Also include the name of the product or service chosen in bold.

Once you have documented a Phase, duplicate the diagram and use it as the basis of the next. Remove the Data Flows and change the red-coloured capabilities to grey.

See examples in Appendix C, D & E to see the progression through three phases of the Cloud Transformation Program.

Step 6: Keeping the program on-track

By developing your Cloud Capability Model and subsequent three phases you should look to entrench this approach with subsequent phases (Projects).

My recommendation is to use a “Technical Design Authority” as your primary means of ensuring subsequent development does not break with the approach or “backslide” to non-strategic point solutions. Any new project should be accompanied by its own version of the Cloud Capability Model.

For clarity, a Technical Design Authority is a group of skilled technologists and architects who convene to approve or decline projects based on several factors. In this instance, adoption of and adherence to the strategy laid down in the Cloud Capability Model.

Finally, to focus peoples’ minds on the Cloud Adoption program, I would recommend that you maintain and display a “Current” Cloud Capability Model which includes all the components that have been selected over the agreed phases. See Appendix F for an example showing the “state of play” after Phases 1,2 and 3.

Skills and Training

This phased approach can also aid in terms of identifying which skills will be required at which stages of a program. By clearly showing the technologies that have been selected you should also be able to produce a list of required skills. You will, I hope, see a reduction in the number of non-standard projects thereby reducing the overall skills required.

Downloads

We hope that this approach will prove to be as useful to you as it has been for our customers. You will also find a download of the Visio diagram used in this article:
CloudCapabilityModelv1 - PDF
CloudCapabilityModelv1 - Visio

Appendix A: Sample Capabilities

 

Capability Area Location Description
Relational DB Data Storage and Retrieval Cloud SMP – Symmetric Multi Processor
Relational DW Data Storage and Retrieval Cloud MPP – Massively Parallel Processing
File Storage Data Storage and Retrieval Cloud SMB File Sharing
OLAP Data Storage and Retrieval Cloud On-Line Analytical Processing
Blob Storage Data Storage and Retrieval Cloud Just a place to store large number of files.
Document Storage Data Storage and Retrieval Cloud JSON
Data Lake Data Storage and Retrieval Cloud Big Data Store (Hadoop)
Graph Data Data Storage and Retrieval Cloud Relationship modelling database
Table Storage Data Storage and Retrieval Cloud Key-Value store
Distributed Ledger Data Storage and Retrieval Cloud Blockchain
Event Ingestion Data Ingestion/Orchestration Cloud Store and process massive data streams
Service Bus Data Ingestion/Orchestration Cloud Communication system for Service Oriented Architecture
Business Rules Engine Data Ingestion/Orchestration Cloud Processing business logic (rules)
ETL Tool Data Ingestion/Orchestration Cloud Extract Transform Load – Typically used to create Data Warehouses
Orchestration Engine Data Ingestion/Orchestration Cloud Automated workflow engine.
Bulk Loading Data Ingestion/Orchestration Cloud Tool for bulk loading of data.
Integration Platform Data Ingestion/Orchestration Cloud Integration Platform as a Service (IPaaS). Set of automated tools for connecting software applications that are deployed in different environments.
Data Catalog Metadata / Governance Cloud Directory of information about files, data, databases etc. Used to assist data discovery and governance.
Data Quality Metadata / Governance Cloud Tool used to enforce Data Quality rules.
Master Data Management Metadata / Governance Cloud Tool used to define and manage business critical data.
API Management Metadata / Governance Cloud System used to publish, govern, secure and throttle API calls.
Big Data Processing Compute and Services Cloud Scale out, massively parallel data processing engine.
IaaS – Virtual Machines Compute and Services Cloud Infrastructure as a Service
Machine Learning Compute and Services Cloud Service providing statistical techniques based on learning algorithms.
BOTs Compute and Services Cloud Multi-channel software application able to communicate in natural language and carry out scripted tasks repetitively and at scale.
Stream Processing Compute and Services Cloud High velocity, complex event processing engine.
Batch Processing Compute and Services Cloud Service providing the ability to run large scale jobs.
Web/App Hosting Compute and Services Cloud Hosting environment for Web and API apps.
Cache Service Compute and Services Cloud Non-durable in-memory data store used to improve processing of data requests.
Mobile Services Compute and Services Cloud Services to assist Mobile Application development, publication, authentication and data distribution.
Language Understanding Compute and Services Cloud Cognitive service used to “Understand” intents and entities in spoken language.
Computer Vision Compute and Services Cloud Service to located objects, people from photographs and video
Text Analytics Compute and Services Cloud Service to derive information from unstructured text
Video Indexing Compute and Services Cloud Service to transcribe and index video.
Robotic Process Automation (RPA) Compute and Services Cloud Automation tool used to mimic repetitive tasks/process flows usually carried out by a person.
Low Code UI Compute and Services Cloud A service to help users build applications with minimal or no software development.
Firewalls Networking and Comms Networking and Comms Network security system used to control incoming and outgoing internet traffic.
Cloud Connectivity Networking and Comms Networking and Comms Internet connection
WAN Networking and Comms Networking and Comms Wide area network
VOIP/Video/IM Networking and Comms Networking and Comms VOIP and Video Conference
Data Gateways Networking and Comms Networking and Comms Software used to handle data movement from and to the cloud
Key Management Security and Authentication Security and Authentication Cryptographic key management service
Single Sign On Security and Authentication Security and Authentication Service to support use of single authentication against multiple applications
B2B Authentication Security and Authentication Security and Authentication Business to Business Authentication
B2C Authentication Security and Authentication Security and Authentication Business to Customer Authentication
Threat Analytics Security and Authentication Security and Authentication Security tooling to provide advanced threat analytics against sophisticated cyber-attacks.
Security Policy Management Security and Authentication Security and Authentication Service to help ensure compliance with company, regulatory and security requirements.
Edge Streaming Data Edge Computing and IOT Cloud Service to provide ingestion of huge volumes of telemetry data.
Device Management Edge Computing and IOT Cloud Service to help the management of large numbers of IOT devices. Providing “Digital Twin” capabilities.
Data Analysis Edge Computing and IOT Cloud High performance, large scale analytics focused on the analysis of Device Telemetry.
Data Capture Edge Computing and IOT Edge Digitisation/Protocol Transcoding equipment. E.g. Modbus Converter.
Data Transfer Edge Computing and IOT Edge Software to support Data Transfer approach from Edge to Cloud. E.g. MQTT
Edge Processing Edge Computing and IOT Edge Device and software capable of processing sensor telemetry locally prior to sending into the cloud or use locally.
Devices and Sensors Edge Computing and IOT Edge Sensors used to capture required telemetry.
Sensor Communications Edge Computing and IOT Edge Bluetooth LE, WiFi, RFC.
Remote Comms Edge Computing and IOT Edge e.g. 3G, Sigfox, Sat Comms.
Semantic Layer BI and Analytics Cloud Business representation of corporate data to simplify analysis by end users.
Ad-hoc Analytics BI and Analytics Cloud Interactive data analysis tool.
Paginated Reporting BI and Analytics Cloud Service for providing “Reporting of Record” paginated reports.
Dashboarding BI and Analytics Cloud Service for providing interactive dashboards.
Dev Platform Software Development Cloud Software Development platform.
Test Platform Software Development Cloud Software Testing platform.
Source Code Control Software Development Cloud Source code control / code repository.
Devops Software Development Cloud Software/Services required to support a DevOps approach. e.g.Docker.
Team Collaboration Software Development Cloud Service providing support for agile software development approaches. e.g. Kanban, RAD, Scrum, Minimum Viable Product.
Operations Management Operations Hybrid Service to provide support for IT Operations tasks typically including Log Analysis, Automation of tasks, Back and Recovery.
Disaster Recovery Operations Hybrid Service providing structured approach to defining and managing Disaster Recovery for both cloud and on-premise data.
Backup/Restore Operations Hybrid Service providing support for Backup and Recovery of both cloud and on-premise data.
Replication Operations Hybrid Service supporting the maintenance of a Cloud copy of On-Premise data.
Active Directory Management Operations Hybrid Service / Software providing for the maintenance of Active Directory users etc.
Device Management Operations Hybrid Service providing single sign on to devices, including BYOD (Bring Your Own Device) and associated authorisation and monitoring of activity.
Cloud Management and Monitoring Operations Hybrid Service providing analytics, protection, compliance and availability of corporate data and services.
Cloud Cost Management Operations Hybrid Budgeting, Financial and Optimisation of cloud expenditure.
Office Software Productivity Hybrid Productivity software/services including but not limited to Spreadsheets, Document Authoring, Presentation Authoring etc.
Intranet Productivity Hybrid Organisation, Team collaboration and publishing tool
VR / Telepresence Productivity Hybrid Use of Mixed Reality technology to provide immersive, augmented experience typically used for collaboration / training.
Knowledge Management Productivity Hybrid Services designed to facilitate knowledge sharing within an organisation. E.g. Enterprise Search Tools
ERP Enterprise Systems On-Premise Enterprise Resource Planning. Service/Software designed to automate the business processes required to operate a business.
Asset Management Enterprise Systems On-Premise Service/Software designed to support the full lifecycle of assets within an organisation. (Purchase through to Removal)
Finance Systems Enterprise Systems On-Premise Service/Software used to manage and govern financial income, expenses and assets.
CRM Enterprise Systems SaaS Service used to manage interactions with current and potential customers.
HR Management Enterprise Systems SaaS Human resource management Service.
Marketing Tools Enterprise Systems SaaS Various marketing services including email marketing, search engine marketing etc.

Appendix B: Example first cut Cloud Capability Model

 

This “First Cut” provides a clear, unambiguous view of the future. By socialising this within your organisation, people can begin to prepare for the changes that are coming and understand the larger context that surrounds their specific area. By avoiding using named technologies at this stage, you avoid challenges from individuals who have a vested interest in maintaining the status quo.

Appendix C: Example Phase 1 - Data Warehouse Cloud Capability Model

 

This Phase 1 model describes the software choices and data flows for a Data Warehouse project.

Appendix D: Example Phase 2 – Customer BOT Cloud Capability Model

 

This Phase 2 model describes the software choices and data flows for a Customer BOT project. Items in Grey have been “Stood Up” in earlier stages.

Appendix E: Example Phase 3 – IOT Cloud Capability Model

 

This Phase 3 model describes the software choices and data flows for an IOT (Internet of Things) project. Items in grey have been “Stood Up” in earlier stages.

Appendix F: Up to date Cloud Capability Model

 

This “Up to date” model describes the Target Architecture, showing software choices that have been made and capabilities that are yet to be decided upon.

Comments (3)

  1. Excellent Guidance. Thanks for the advice, framework and target architecture diagrams. Very impressive.

  2. Very useful guidance.
    In your article you mention: You will also find a download of the Visio diagram used in this article: CloudCapabilityModelv1.
    This points to a PDF file. Can you also provide the Visio file, this would speed up writing down capability overviews.

    1. Stevey A says:

      Hi Martijn, fyi, I have now added the Visio Diagram to the post as well as the PDF.

Skip to main content