Insufficient data from Andrew Fryer

The place where I page to when my brain is full up of stuff about the Microsoft platform

SQL or No SQL ?

I have a post on NoSQL but as Azure moves on it’s looking a little long in the tooth so I thought it was time for an update.. 

I am a big fan of Darwin because he was able to question his own beliefs and perceptions by taking a hard look at the evidence around him, and if his survival of the fittest is to be believed then the relational database will be replaced by something better. Mass extinctions generally occur when the environment changes and in the IT landscape the biggest change is the cloud and I can see that this alters the way we interact with data in online transactions.  So if we set aside the pros and cons of a traditional relational database for this work,what is there to replace it, that makes it more evolved  and better?

The recent arguments I have seen seem to advocate column based databases which are superb for querying but less efficient than row based solutions for oltp.  This isn’t really new,  I was using Sybase IQ back in 2004 and it really made Business Object fly so I can understand why SAP have acquired Sybase given they now own Business Objects.  So was it ahead of its time and will it now flourish as the cloud takes off.  Possibly possibly not and I don’t really care because SQL Server is evolving itself to adapt to the demands of the cloud and to handle the many kinds of data we need to keep track of:

  • SQL has always been able to store various forms of unstructured data, and this has got better over  time: SQL Server 2008 filestream and remote binary storage allow massive files to be indexed and retrieved at nearly the speed of native file access using the win32 api.  Denali (SQL Server vNext)  will have file tables which takes this even further and embodies a lot of the principles of the WinFS work of a few years ago.
  • Denali will also have column based storage which gives massive query improvements like I saw in Sybase IQ but without the need to have a replica of the data in another product. So you can use the same SQL language and decide on whether to optimise for OLTP or reporting needs.
  • As hardware evolves and operating systems address these changes then databases will make use of these, be that multiple cores solid state storage or the availability of huge amounts of RAM. 

I can’t speculate how much of this ends up in future iterations of SQL Azure, but here’s a couple of things to bear in mind:

  • 70% plus  of Microsoft development resource is working on cloud services
  • SQL Azure gets updated every 90 days, it has 5x the storage capacity it did a year ago (and sharding allows you to exceed this if you need to) , it now has spatial data types and reporting services is on the way.

So my assertion is that databases like SQL Server have continually adapted and survived over the last 35 years and will probably continue for the foreseeable future,.

Happy as ever to debate this online or over coffee