Voice of the DBA

Category: Editorial

  • Keep Data Forever

    Will this keep data forever?

    How long do you expect to keep a piece of data in any of your databases? Chances are if you’re like me, you rarely think about it and expect that you need to store all the data perpetually. As long as you have a job, and the database is being used by some system, you keep the data.

    Over time that becomes a problem. Some companies are starting to examine the legal issues of old data, mostly emails, but potentially other records, and digitally shredding or removing that data over time. For others there are the problems of storage, with the raw space needed to keep all this data growing faster than hardware budgets. More and more companies are looking to archive data that is less used to ease the stress on their storage and backup systems.

    However if you do decide to keep data forever, how do you do it? There are all sorts of issues with various techniques, and I have heard a few companies considering printing out the 1s and 0s and using a more durable storage medium like microfilm or even paper-like substrates that might last more millennia. There is a new type of disk that promises to last longer than ordinary discs thanks to it’s “etching” rather than “burning” process.

    Whether it really works is something we won’t know for years, but I do know that the best solutions for now are migrating your data to new (and larger) storage mediums and crossing your fingers that any archived data on disk, tape, or DVD is still readable if someone asks for it.

    Steve Jones

    The Voice of the DBA Podcasts

  • Pop Tarts and Hurricanes

    The second most dold item at Wal-Mart before a hurricane.

    What do people buy most of before a hurricane? Wal-Mart determined that batteries are the number ones sales item before a hurricane, but the second most sold item was surprising: pop-tarts. That’s the type of intelligence that can come from analysts asking the right questions about their customers and then IT delivering systems that can help them get an answer.

    More and more companies are starting to deal with big data. Big data is usually seen as extremely large sets of data. Those data sets  often exceed the capability of what a single database server can handle, and place a strain on existing IT infrastructures, especially when there is an explosion of new datab types. Hadoop is an open source framework that’s designed to deal with large sets of data and help with the processing and analysis of these large data sets.

    Microsoft recently added support for Hadoop to SQL Sever. Granted it’s only in the Parallel Data Warehouse, which is a highly specialized version of SQL Server integrated with specific hardware. However I am sure this will eventually make its way to other editions over time.

    Update: I made a mistake, there is a connector for non PDW SQL Server 2008 R2 instances.

    I don’t know if I would recommend learning Hadoop, but the techniques of processing large sets, and performning analysis on big data is important., Even more important might be the writing of well performing T-SQL code that will be used. I’d recommend you read about ways to write better code, and learn to integrate those skills into your daily work. You don’t always need them, but you often don’t know that when you are writing the code.

    It doesn’t take much more effort to write better code the first time, but it does take more effort to learn to do so. It’s an investment in your career that you ought to be regularly working on to become one of those talented people that is in demand in the future.

    Steve Jones

    The Voice of the DBA Podcasts

  • The Talent Crunch

    IT Jobs are everywhere. Check out the listings on any of these sites.

    Is it a good time to be in the technology business? I think so, as the US unemployment rate stands at just above 9%, but the rate in IT is listed as about 4%. There might be some errors in how they are classifying and counting IT jobs, and the unemployment is not evenly distribute across the US, but that still seems like a significant difference. As a comparison, much of the medical profession is similarly employed, as are teachers (yeah!), but construction trades are showing  > 20% unemployment.

    Not only are there lots of jobs in the technology area, but overall technology pays a much higher salary (or wage) than many other industries. Whether you think you are paid well or not, peruse some other professions on Dice, Monster, or another job board and compare salaries. Keep in mind the US median income is $48k, meaning half the people make more and half make less. I suspect most IT workers are in the “more” side of that range.

    I think it’s also likely that the unemployment rate in technology could decline further. There’s an interesting report from Dice (2MB PDF) that indicates we are in a talent crunch as well that could become more problematic in the future. More than a few places predict that we will have a shortage of skilled technology workers in the future. Some of those predictions are based on enrollment in college math and science programs, which can be faulty. IT work doesn’t require a college degree, and perhaps shouldn’t.

    However it does seem that we have an ever increasing need for people to work in technology. I’ve had friend worry that our own efforts to improve and automate things would result in fewer jobs, but it seems that’s not the case. The gains in individual productivity in both the development and administrative areas are easily outstripped by the huge growth in the numbers of applications and servers to manage.

    Hopefully you have a good career in IT, and will have one for as long as you want one. I’d caution you to think carefully about the title of this piece. There’s a crunch for talented people, not just people. You need to work on your technical skills, and your soft skills, regularly if you want to be in demand in the future.

    Steve Jones

    The Voice of the DBA Podcasts

  • Virtual Security

    The State of New Mexico is being careful with virtual machine security

    I have a few friends that are working virtualize almost their entire computer infrastructures.  They work in large and small companies, but there is a constant push to avoid the bare metal installation of any operating system onto physical hardware, making every Windows or Unix machine a virtual machine on top of a hypervisor. I was surprised to hear that companies were being to aggressive, but the cost benefits can be huge, and when virtualization is done in a smart way, performance doesn’t suffer.

    However virtualization can change security, especially when you have VMs that are allowed to move from physical host to physical host. The state of New Mexico embarked on a similar project, and were concerned over security of the virtual machines. Their department had dismissed some employees because of a security breach a few years earlier and security was on the forefront of their minds. Additional security as well as network controls were used in their project, and I hope they also implemented strong auditing procedures.

    As we move to newer infrastructures that include virtualization, physical security becomes more important, and additional controls are needed. The ability for someone to potentially move a VM outside of a data center, or even to a less secure remote data center becomes a point of concern. Moving the storage itself might be an even bigger problem as virtual storage becomes more commonplace.

    Ultimately, however, we can’t all have dedicated security employees, nor can we expect every DBA, sysadmin or even security officer to be able to protect against and mitigate all attack vectors. Auditing is ultimately the best way to handle breaches. We can’t prevent all of them, but responding quickly, learning, and perhaps more importantly informing the appropriate people to be ready to respond to the information disclosure.

    Steve Jones

    The Voice of the DBA Podcasts