Microsoft Big Data Hackathon Resources (Vancouver)

Article
04/17/2015

Category	Data Set
Demographics	Local census profile https://namara.io/#/display/9ebda3ec-ea3f-4caf-bbac-95274f4317c8
Transportation	EV locations: https://namara.io/#/display/7aee7c3c-009b-46e4-8b1d-797aa58d19ec Parking meter data: https://namara.io/#/display/b10f2212-b075-48ed-88e9-fe33cffa0cd9 Rapid transit stations and lines: https://namara.io/#/display/53cef883-5ef4-4822-bf47-047d2c3f0032
Social and community housing:	Homeless shelter: https://namara.io/#/display/50b3270e-78b6-4c95-b072-69a132f4b6ec
Environment	Fraser River Basin Long-term Water Quality Monitoring https://open.canada.ca/data/en/dataset/9ec91c92-22f8-4520-8b2c-0f1cce663e18 Surface water quality related to Alberta oil sands area https://open.canada.ca/data/en/dataset/48d2acb8-f5ed-4f8e-82eb-8c60f31e0682 Also of interest – Adjusted daily rainfall and snowfall dataset for Canada https://open.canada.ca/data/en/dataset/d8616c52-a812-44ad-8754-7bcc0d8de305 Water Discharge dataset from Water Survey of Canada’s Hydrometric Network https://open.canada.ca/data/en/dataset/29df4910-c400-41c9-9f09-262f1d7b8525 Fraser River buoy hour water quality monitoring dataset captured from 2007 to present https://1drv.ms/1cp2eU1 (program site is located here https://aquatic.pyr.ec.gc.ca/fraserriverbuoy/default.aspx)
Cultural / POI	Cultural spaces: https://namara.io/#/display/3c9754ad-02a3-452c-8084-39a13df6bf3d Schools: https://namara.io/#/display/e5cb3e2a-78a2-424d-85f4-6d52df76d25c Traffic signal: https://namara.io/#/display/50994967-953b-4fa7-931b-c6dabee92294 Libraries:https://namara.io/#/display/0fbbaf79-c377-42c6-a3c9-9d5afa0dcab4 Public art: https://namara.io/#/display/8d4587d1-9989-4048-98b2-cbb80f23afac Businesses: https://namara.io/#/display/c85e9eef-b516-433c-b25d-f3503edd6de6

ThinkData Works Data sets: https://namara.io/#/

The site consolidates data from open.data.ca, Statistics Canada, Provincial Data sources, GoodLife Fitness, SpotCrime and others. Here are the links to a few datasets of interest:

City of Vancouver Open Data Catalog: https://data.vancouver.ca/datacatalogue/index.htm

British Columbia Open Data Catalog: https://www.opendatabc.ca/dataset

Canadian Government Open Data Portal: https://open.canada.ca/en

Collection of Data sources:

https://mran.revolutionanalytics.com/documents/data/?utm_campaign=Data_Elixir_20&utm_medium=email&utm_source=Data%2BElixir

Finance, Economics and Society data: https://www.quandl.com/

You also can use Power Query to retrieve data from Facebook: please read article about it here

Example:

US/Canada Border Wait Times are available here

https://open.canada.ca/data/en/dataset/000fe5aa-1d77-42d1-bfe7-458c51dacfef

The data set is not large (around 1M records) and In itself is not very interesting – as analysis is pretty much limited to location and time - but if mangled with other widely available data sets, could be a basis for relatively interesting exploratory and predictive analysis.

You could integrate and correlate it with:

· Weather data from nearby weather stations: https://climate.weather.gc.ca/

· Canadian dollar exchange rates: https://www.canadianforex.ca/forex-tools/historical-rate-tools/historical-exchange-rates

· Fuel prices: https://www5.statcan.gc.ca/cansim/a26?lang=eng&retrLang=eng&id=3260009&paSer=&pattern=&stByVal=1&p1=1&p2=31&tabMode=dataTable&csid and https://www.energy.gov.on.ca/en/fuel-prices/

· Terror alert levels https://www.dhs.gov/how-do-i/check-national-terrorism-advisory-system-ntas

· …

Using these data sets you could perform both historical analysis (including geo-spatial visualizations) and attempt to build a predictive model.

Trial versions and subscriptions

· Office Professional Plus 2013 or Office 365 (we recommend to use Office 365 Pro Plus version)

· Excel Add-ons: Power Map, Power Query

· Azure ML Trial

Online trainings