Introducing the Intelligent Kiosk, a collection of experiences built on top of Microsoft Cognitive Services, and now an official Cognitive Services sample and open source project. Intelligent Kiosk is an ideal demo tool for partners. In this post, I’ll explain how you can access and customize the Intelligent Kiosk resources to incorporate into your demos and offerings. And, in our next Data and Analytics Partner community call on November 15, we’ll cover this topic in depth.
- Sign up for the November 15 Data and Analytics Partner community call
- Learn more about Microsoft Cognitive Services
The Intelligent Kiosk is a Universal Windows Platform app with a collection of demos that showcases the perceptual intelligence capability of Azure Machine Learning, Cortana Intelligence Suite, and Microsoft Cognitive Services.
The Kiosk is an example of an intelligent application that can understand human faces and predict their age, gender, and emotion. It can do so from live images from a web cam, Bing Images, or local photos. It allows you to very quickly create and train a model to perform face identification against a pre-determined set of faces.
Through the power of speech-to-text and text sentiment analysis, the Kiosk shows how an application can detect the sentiment of what someone is saying. Most of the experiences are hands-free and autonomous, using human faces in front of a web camera as the main form of input.
Cognitive Services scenarios in the Kiosk demos
The Kiosk uses many of the Cognitive Services across its scenarios, including Bing News Search, Bing Image Search, Bing Autosuggest, Text Analytics, Face, Emotion, Vision, and the LUIS/Bot Framework. Here are the scenarios in the Kiosk demos, and their underlying product features.
- Automatic Photo Capture. Capture an image and apply analytics when people approach the web camera. Features: Windows 10 Face Tracking; Face API for age & gender prediction and face identification.
- News Analytics. Analyze search terms and detect sentiments. Features: Bing News API, Bing Autosuggest API, Text Analytics API for sentiment analysis and key phrase extraction
- Emotion API Explorer. Analyze faces and detect emotion. Features: Windows 10 Face Tracking; Emotion API; Bing Image Search API; Bing Autosuggest API
- Face API Explorer. Analyze faces and detect age and gender. Features: Windows 10 Face Tracking; Face API for age & gender prediction and face identification; Bing Image Search API; Bing Autosuggest API
- Face Identification Setup. Train the machine learning model behind the Face APIs to recognize specific people. Features: Face API for face identification; Bing Image Search API; Bing Autosuggest API
- Mall Kiosk. Makes product recommendations based on the people in front of the camera and then analyzes their reaction to it.Features: Windows 10 Face Tracking; Face API for age & gender prediction and face identification; real-time sampling using Emotion API; Windows 10 Speech-To-Text; Text Analytics API for sentiment analysis.
- Realtime Crowd Insights – processes frames from a web camera to derive realtime crowd insights such as demographics, emotion and unique face counting. Features: Windows 10 Face Tracking; Realtime sampling; Face API for age and gender prediction, face identification, and unique face counting; Emotion API.
The Kiosk demos illustrate these scenarios in various ways. Many of them can be customized to be able to recognize a specific person or make specific recommendations based on criteria. For example, perhaps you’re going to meet with a customer CIO, and you know she loves sailing. With a few minutes of configuration beforehand, you can give her a personalized experience in the Face API Explorer, and offer her sailing-related suggestions in the Mall Kiosk.
- Bing News Analytics – Categorizes search results based on the sentiment of the news headlines, and uses Key Phrase extraction to create a word cloud with the top keywords of the headlines
- CaptionBot – Captions whatever is in front of the camera
- Crankify Kiosk – Turns the user into a Crank from the Maze Runner series
- Emotion API Explorer – Allows the user to load images from Bing or Local Files, or from the web-camera, and displays the top three emotions on each face
- Emotion Match (baby face edition) – Analyzes the emotion on each face and matches it with a face with similar emotion from a set of baby photos; performs real-time face tracking to detect when people are near the camera; once it detects people, it guides them through the rest of the experience
- Emotion Photo Booth – Challenges the user to express the eight emotions that are recognized by the Emotion API; can handle many people at the same time: whoever shows the most expressive look for each emotion wins
- Face API Explorer – Performs age and gender prediction as well as face identification by using web cam photos, local files on your hard drive, or images from Bing Images; for people who have been added to the Face Identification Setup model, it will show their names and age, for unknown people it will show their gender and age
- How Old – Guesses the age and gender of people; performs real-time face tracking to detect when people are near the camera; once it detects people it guides them through the rest of the experience; can also be trained to identify specific people
- Mall Kiosk – Makes product recommendations based on the person’s gender and age; can be configured to make personalized recommendations when specific people are recognized
- Realtime Crowd Insights – Analyzes emotion, age, gender, and identification while the people are looking at the camera, and displays those insights
- Realtime Video Insights – similar to the Realtime Crowd demo, except that it uses as input a video file (from a local file or YouTube) and provides a different visualization of the emotion by slicing it for each person in the video and plotting it on a timeline
- What Dog – similar to Emotion Match, except it matches each person with a dog, based on the age and gender of the person
- What If – uses Cognitive Services and the Bot Framework to analyze “what-if” like questions and answer them with an image
The partner opportunity
The Intelligent Kiosk is an ideal demo tool. It engages your audience quickly, and they can easily see the relevance to their own business needs and use cases. Consumers expect personalized experiences, and the Kiosk makes the personalization opportunity clear and achievable.
You can take the demo on the road with you using any Windows 10 device that has a webcam, and you can also set it up at your office or at your next conference booth.
Most the demos included with Intelligent Kiosk are hands-free and self-guided. They require only the web cam as input, using the live camera feed to detect when people are nearby and engaging with them through UI feedback.
Business scenarios where a user encounters a screen presents the opportunity for personalized interaction with that user:
- Digital ads and directories in public spaces like shopping malls
- Retail store experiences
- Hotel, resort, and casino environments
- Waiting rooms
- Gas pumps
- Public transit
Consider your customer portfolio and identify opportunities to customize an Intelligent Kiosk demo that provides them with a transformative solution.
Create your Intelligent Kiosk demo
For information about a pre-built app, read the Intelligent Kiosk note in the Data Platform and Advanced Analytics Partners Yammer group (instructions to join the US Partner Community Yammer network here)
Whichever way you prefer to set up the kiosk, this powerful demo tool will help your customers see new potential to engage their own customers – one of the four dimensions of digital transformation.