You think you have backup challenges?

A few months ago at the last stop on the Core I|O Tour, Rick and myself were talking about Data Protection Manager and how it can be used to simplify backups and restores of your data.  Someone from the audience asked a question at the end that stumped us so we took it offline and responded after the event and blogged about it here.

The person was Dan Braun, an engineer with CHUM, the television station behind such channels as Much Music, StarTV, Bravo, Canadian Learning Television, Space and CityTV (including my favorite channel since moving to TO, CP24, for their traffic cams!).  In total they run 21 specialty channels and 12 independent local channels across Canada.  During our email conversation, Dan invited us down to take a tour of the infrastructure behind CHUM and look at their backup system.  Rick, being located in Ottawa, could not accept the invite but I jumped at the chance!  Afterall I am still a geek at heart.

What intrigued me most was the fact that the news department is a completly tapeless environment.  From the cameras out on the street, to the editing room, to the final video you see on the air, it is all disk based now.  You can imagine the amount of storage space and the backup requirements.  I counted 8 individual 3.2TB drive arrays in one half rack alone!  And I saw a few other drive arrays that were even larger!  With all this data, and more being added on a daily basis the question becomes how do you archive this all for backup and future use?

Dan took me on the tour and the solution was a Sony Petasite that could hold approximately 1.7 PB (yes that is Petabytes, as in 1,024 TB) and still has room for expansion!  Think of four full height racks of storage space and you start to get the picture.  There was also a DVD changer that held over 1400 dual sided DVDs which had me thinking how cool would that be to have a home!

One of the things that also interested me was how technology has changed in a way that allows for more to be done in less space.  For example, with some older equipment it took up to eight full height racks to house the equipment required for one TV channel.  The same thing can now be done with a half rack worth of equipment allowing for CHUM to expand the number of channels while actually reducing the amount of space required in their datacenter.  Another example was an A/V switcher.  The old switcher could switch 96 inputs to 96 outputs and took up about seven full height racks to contain.  The new switcher can switch 512 inputs with 512 outputs and takes up one full height rack.

With all the wiring, I have to give a big congrats to the guys who do the wiring there.  I have seen some pretty good wiring jobs, and way to many horrid wiring jobs, but I have never seen as much cabling as I did in their datacenter and never have I seen such neat, tidy, and well labeled cabling.  From standard Cat5e to fibre, to RG6 and most everything in between I cannot imagine how many miles of cable runs through there.  Thankfully they have it all well documented as well.  I know too many people who could learn a thing or ten from these guys.

Of course Dan took me on a tour of the "environment".  CHUM was the pioneer in studio-less broadcasting and walking around the main floor you have to be careful where you walk so as not to disturb a live broadcast.  As a hint, if you see bright lights, you are in the wrong area!  Lucky Dan steered me clear and the security kept me out of a live session on Much Music.  He showed me the Bravo environment, I got to stand in front of the weather "green screen" and read from the TelePrompTer, go into the news room and watch some of the anchors as they pull video from the archive, add their script and prepare for the news broadcast.  It is really amazing how automated the entire system has become.

One very cool room was the floating sound room for the Much Music environment.  It is about the size of my home office, but literally floats as it is completely decoupled from the building in order to prevent any vibration from affecting the very sensitive sound recording equipment inside.  All in all it was a pretty cool experience.  There is a whole lot of technology in the world doing things that we take for granted.  I enjoyed getting a behind the scenes look at how some of my favorite channels make if to the air.