Voice of the DBA

Category: Editorial

  • A Good Use for Hadoop

    Netflix is an amazing company. I’ve been a customer for over a decade, starting with the well known red envelopes that moved DVDs through the mail. Since then I’ve continued to subscribe as a streaming company and have enjoyed quite a few of their original programming efforts.

    However I’ve also been fascinated by the way in which Netflix has made use of technology to grow a unique kind of company. They own very little, having used Amazon’s AWS to host their infrastructure and mostly serve content they acquire from contracts with various media companies. They are famous for their Chaos Monkey approach, having machines randomly fail to test the fault tolerance of their entire infrastructure. I read recently about the closing of their last, small data center, so outside of employees’ laptops, they keep all systems in the cloud.

    However I ran across a post on the scale of their data flow, and it’s amazing. Apparently their events are generating 1.6PB a day of data. That’s incredible, and a scale at which very, very few of us will ever work. Personally, I think that’s interesting, but I’d prefer not to be working on a PB of changing data a day, but I might feel differently if I worked at a company like Netflix that has obviously been successful with dealing with data at that scale.

    The post notes that they put this data into Hadoop, where it can then be queried. I’ve often wondered what the domains are for using Hadoop. Most of the people I know that have tried it aren’t really working at large scales. I think that a well built ETL process would allow their data to easily work in a SQL Server data warehouse, and possibly even the Azure SQL Data Warehouse. However at 1.6PM a day, Hadoop seems like a much better choice.

    I’m curious how often that data is queried. Is most of that 1.6PB actually used in reporting, or is much of it lost in the shuffle and ignore? Is there a process to aggregate some of this raw data and then delete the older values? Can 1.6PBx365 actually be useful in analyzing your business? I’m sure some is, but wouldn’t a lot of those events lose their value over time?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.7MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • SQL Server 2016: The Data Protection Version

    I’ve spent a few years working with SQL Server encryption and security, trying to educate people on the various capabilities involved. There are some nice features available in the platform that can help you run a more secure SQL Server, but there has been a lot of room for improvement across the last few versions. The openness of the platform is somewhat tempered by the need for each of us to write a fair amount of code to properly secure our databases. That is the hard part as many developers don’t write secure code.

    That is changing a bit. I am really excited about the SQL Server 2016 release in that there are a number of data protection mechanisms that are going to greatly expand how we can protect our data moving forward. From Row Level Security to Always Encrypted to Dynamic Data Masking, the number of new features that will help make security a bit easier is growing dramatically.

    I certainly think Microsoft is definitely encouraging and supporting better, more secure coding practices. If you look in Books Online at MSDN, there is now a Security Center that covers a number of topics and organizes information in a much better way. Security has seemed like an afterthought in past documentation. From Encryption to SQL Injection to Auditing to even Metadata Visibility, it seems like the next version of SQL Server on premise, and in Azure, really is taking security more seriously.

    I look forward to watching the platform evolve and security increasing over time. It does seem as though SQL Server has the fewest security holes in the platform, but I hope that many of us can take advantage of the changes in SQL Server over time and build applications that will be viewed as being secure as well.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Data for Ransom

    Imagine you come into work and all of your database columns are encrypted. You have no idea what key was used, and your application can’t access the data. You receive a message that for a $100,000 payment, the decryption key will be sent to you. What do you do?

    Well, depending on your business, and how quickly you can restore backups, maybe your organization pays. I read a story recently about a hospital that found a number of their files encrypted by a virus, with a demand to pay a ransom for the decryption key. They did, received the key, and opened their files. However that set a dangerous precedent.

    Apparently attacks by ransomware are on the rise. Those of us with database software are not likely to be too affected as we (should) have regular backups we can restore. However the interruption to businesses can be costly, and this can result in more organizations paying the ransom, which then encourages more attacks.

    The new Always Encrypted capability in SQL Server 2016 is great, and I’m excited by it. However, we want to be sure a malicious user doesn’t enable this feature. I could certainly see that as an attack vector for web based systems. An attacker gets sysadmin privileges, enables Always Encrypted and places the certificate on the client web server. Some weeks later, they remove the client certificate, and suddenly no one can decrypt the data. Potentially, depending on backup procedures, there might not even be a capture of the certificate from the web server in any backup files.

    The world is becoming more dangerous, and more troublesome for our data. As with most things, vigilance and monitoring of security segments of our applications is important. While ransomware is unlikely to strike databases, one never knows, and you should ensure that you have good backups in the event that you are attacked.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Interesting Data Centers

    I’ve spent a couple decades in technology working with lots of companies. As an employee, a consultant, or a sales engineer, I’ve had the chance to visit quite a few data centers over the years, some of which were quite interesting. A few impressive, and perhaps a few more that were quite scary.

    When I started working in technology, the “server” was often just one box that lived on a desk in the office. In a few of my early small business clients, our server was often used by the secretary for Wordperfect typing while also serving applications and acting as a file server. In a few larger clients, including at a state government agency, multiple servers were piled on shelves in a closet.

    Over time, lots of small businesses, and a few larger ones, learned that hot closets don’t work for multiple servers, nor does a lack of clean, UPS power serve them well.  I’ve worked with quite a few companies to upgrade their facilities to include better power and cooling, often racing to keep up with the proliferation of server systems.

    As time progressed I found many companies that didn’t want to invest in a data center, including us here at SQLServerCentral. In the early 2000s, companies began to trust co-location facilities, those professional data centers built and run as a business. I toured many and saw some well built environments, and some not so well built. One small company SQLServerCentral visited had two corners of an office building downtown, on separate floors, with cool hoses (about 2ft in diameter) run outside, up the side of the building.

    As we look to move to the cloud, which is the next evolution of the co-location facility, I expect that more and more of us may never visit a data center in our lives. Some small startup companies don’t even own servers, outsourcing their email, VCS, build systems, and more to some vendor.

    That’s not for everyone, but it is becoming more commonplace. Vendors are even trying new ideas to lower their costs, while still maintaining the level of service we expect. There are data centers being built in mines, which provide cheaper cooling than traditional buildings. There are container based systems, and while electricity and water don’t usually mix, Microsoft is looking to try putting data centers in the ocean. Of course, not all ideas are good ones, as the ill-fated Sealand showed.

    The future of data centers and where and how we run server hardware will certainly be interesting. I wonder how many future DBAs may never physically touch the actual hardware that contains all their bits and bytes.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.2MB) podcast or subscribe to the feed at iTunes and LibSyn.