Using the HTTP OMS Data Collector API for real-world scenario’s–Part 1

  • Article

If you are a user of the Operations Manager Suite (OMS) you probably have seen the announcement we made about the public preview of the Log Analytics HTTP Data Collector API. This API (Application Programming Interface) enables you to send custom data directly to our Log Analytics endpoint via HTTP.

The  cool thing with this new API is the option to send any data, and yes I mean any data, to OMS and query that data in OMS. And you may have noticed reading this blog that I love to collect all kinds of data and do cool stuff with that data.

So what’s the real-world scenario I was talking about in the header of this first blog post about using the Log Analytics HTTP Data Collector API? Wouldn’t it be cool to leverage the Log Analytics HTTP Data Collector API for the following scenario?

Scenario:

  • Collect stock values of your favorite company or companies on the NASDAQ stock exchange.
  • Store the collected Stock information in OMS using the Log Analytics HTTP Data Collector API.
  • Create an OMS view for the collected data with the new OMS View Designer.
  • Get notified when to sell or buy your favorite stock using OMS Alerts

In this first blog we are going to create a PowerShell script that will retrieve the stock information which we will later use to push this data to our OMS workspace.

For retrieving stock information we are using a web service from Google. There are many more options but this was one of the first I found when doing some searching online.

The url for retrieving stock info is the following: https://finance.google.com/finance/info?client=ig&q=msft

If you run this in your favorite browser you see the following info. It already looks like a kind of JSON object being returned. To create a ‘real’ and usable JSON we need to do some PowerShell magic but this where I started..

image 

Let’s start getting this data using PowerShell.

 
[sourcecode language='bash'  padlinenumbers='true']
$stockname = 'MSFT'
$HttpRequestUrl = 'https://finance.google.com/finance/info?client=ig&q={0}'-f $stockname
$StockValue = Invoke-WebRequest -Uri $HttpRequestUrl -UseBasicParsing
$StockValue.content

This will return the following data:

image

Before we can use this data as input for the Log Analytics HTTP Data Collector API we need to do some cleaning up.

The Log Analytics HTTP Data Collector API only accepts a JSON object being an array or collection..

Above object needs to be converted to something like this:

image

Remark:

  • The square brackets indicate that this is collection or array of objects in the JSON (although you can argue that one object is not a collection).
  • The Log Analytics HTTP Data Collector API automatically converts object properties to certain types. See https://azure.microsoft.com/en-us/documentation/articles/log-analytics-data-collector-api/ for more info. So make sure that any numbers are not strings because otherwise you have problems when using OMS Log search queries which may expect numbers instead of string object properties..

Function Get-StockPrice:

 
[sourcecode language='bash' ]
function Get-StockPrice
{
  [CmdletBinding()]
  Param
  (
    # Param1 help description
    [Parameter(Mandatory = $true,
        ValueFromPipelineByPropertyName = $true,
    Position = 0)]
    $StockName
  )

  Process
  {
    $HttpRequestUrl = 'https://finance.google.com/finance/info?client=ig&q={0}'-f $StockName

    $stockvalue = Invoke-WebRequest -Uri $HttpRequestUrl -UseBasicParsing

    #Fixing output. ONLY works with one STOCK PRICE!
    $stockprice = ((($stockvalue.content).Replace('//','')).Replace('[','')).Replace(']','') | ConvertFrom-Json

    #Convert to correct types

    $stockprice.id = [int]$stockprice.id
    $stockprice.l = [double]$stockprice.l
    $stockprice.l_fix = [double]$stockprice.l_fix
    $stockprice.l_cur = [double]$stockprice.l_cur
    $stockprice.c = [double]$stockprice.c
    $stockprice.c_fix = [double]$stockprice.c_fix
    $stockprice.cp = [double]$stockprice.cp
    $stockprice.cp_fix = [double]$stockprice.cp_fix
    $stockprice.pcls_fix = [double]$stockprice.pclx_fix

    ConvertTo-Json -InputObject @($stockprice)

  }
}

In the following blog posts we are going to use above PowerShell Function to retrieve stock values and storing the results in OMS using the Log Analytics HTTP Data Collector API.

Teaser screenshot for the OMS View collecting Stock information and sending this to OMS.

image

Hope you find this an interesting start.

References: