Voice of the DBA

Category: Editorial

  • Lots of data in RAM

    I remember buying my first computer after college. At the time I was working part time and in graduate studies part time. My aging high school era Apple II wasn’t capable of handling the load, not to mention its 300baud modem wasn’t a lot of fun. I could likely have gotten a faster modem with a serial connection, but most of the modern Internet providers wanted Windows or a Macintosh OS.

    I decided to just spend the money and get a computer. Being a bit of a geek, I wanted to customize things a bit and get the most bang for my buck. At that time, I paged through Computer Shopper, looking for deals on parts and trying to find the cheapest 486 CPU I could. I made various choices, one of which was the RAM. I remember thinking hard at that time about whether I could get by with 2 1MB modules or did I need to go ahead and max out the motherboard and get 4. These days I think about GB, not MB, and 2 or 4 isn’t a good number.

    One of the first servers I built at one job was with 8MB chips, getting 32MB into a machine, which felt like a lot. Certainly more than I’d ever had in a computer before. I also remember helping write a PO for a Netware server that cost US$250,000, one with 256MB of RAM. It was the size of a washing machine and filled with 32MB HDDs, making it by far the largest machine I’d seen to date.

    That’s a nostalgic view back to 1992, when I was working on my CNE certification. I always think about that large, very expensive machine when I see some leap of hardware. This past week, it was a DD4 Gen-Z memory module that has 256GB of RAM in a stick. That’s not only a larger scalar value than the RAM sticks I started working with, but it’s an order of magnitude more memory. Imagine 4 of these sticks giving you a terabyte of RAM in a desktop. Who among us wouldn’t want this for our desktop SQL Server development?

    This level of tech isn’t ready for most of us, but it’s not that far off. 32GB laptops aren’t common, but they are easy to find. You can even get 64GB of RAM now if you want to. How long before we see 128GB+ desktop and laptop machines? I have no idea, as we seem to be in a bit of a hardware lull. Most developers run 8GB or 16GB, and it seems like this has been the standard for quite a few years.

    I don’t know if we’ll start to see some vendors pushing to add more RAM. Certainly the cloud has changed things with lots of processing and storage occurring off the local system. At this point I’d expect that three years after buying my last laptop, I’d be able to get 32GB in the same form factor for the price I paid a few years ago, but that’s not the case. The machine supports 32GB, but it’s pricey, the same price it was 3 years ago.

    I wonder how many of you are still power hungry, wanting more powerful machines. Or has the level of performance you get from your i7 on Windows or OSX been good enough the last few years? I think this machine works great for me, whether running VMs or containers. 16GB does the job well, and I’d look to get the same level of hardware when I replace this laptop.

    Steve Jones

  • Demo Data for Everyone

    As someone learning about DevOps, I follow a number of people, one of whom is Gene Kim. When I see him get excited about a post, I usually read it. That’s how I found this post on Demo Data as Code. It’s a short, but interesting read. I think this is actually something more people ought to implement in their environments and not just for demos.

    DevOps is about reliability and repeatability, among other things, but those two are tackled with automation for a known process. We don’t want simple, silly mistakes, or even complex errors that might undermine our ability to move forward and create value. We don’t want simple errors eating up resources and time from expensive talent with unnecessary work. Part of ensuring both repeatability and reliability involves using data in our databases to evaluate our application. This isn’t necessarily for demos, though it could be used for demos.

    Once of the areas that is often left out of the process is the data that we use in our building our systems. We need some data for developers, for QA, and often for demos. In all of those cases, when humans need to repeatedly look at how well the software performs, and want to re-test things, they need some consistent data. I’d also argue that the need for agility means that we need a manageable data set. I think SQL Provision from Redgate is amazing, but I still don’t want to always develop with 2TB of masked data. I certainly don’t want to demo this for customers from a laptop, and might not want to share this in the cloud.

    At Redgate, we sell masking with SQL Provision, and it supports most of the process that’s outlined in the Demo Data as Code article. What it needs, however, is a small set of data that can be masked in a deterministic fashion. What I recommend to most clients is that they build a known set of test data, which could be used for demos. This can include all your edge cases and show off new features. It’s helpful for developers, testers, and salespeople, who will always have a known, useful set of data.

    This can’t be a build it and forget it, much like what is emphasized in the article. This will need to be altered over time. There ought to be a process to build this dataset, likely from production data that gets sanitized. This can then be distributed through SQL Provision (or similar technology), with backups, or even as a set of scripts in your VCS. Ensure an environment can be hydrated instantly on any platform, from a developer workstation to a sales laptop to a QA server. Once you have this, everyone can work on evaluating your software from a known baseline.

    And if you find the need for more data, then just add it. You have a process, so add an additional step that will cover the holes you inevitably find.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Organized Learning

    I really wish I’d do something like the DBA Training plan from Brent. Honestly, I don’t have the time to do it the way he is, and certainly can’t respond to any volume of emails, but I applaud the way he’s doing this. Brent is sharing a lot of ideas and information on how to start getting a handle on your environment. As of this writing, there are 8 posts (part 8). I expect more in the future.

    Becoming a DBA is often a random collection of experiences that almost everyone goes through in a different order. Maybe you started by having to recover a database with a restore. Maybe it was the need to install the server software and configure users. Maybe you find yourself more interested in writing database queries in T-SQL than methods in C# and switch over. However you learn, there are two things I know: your environment will drive your learning and there’s always more to learn.

    My start as a DBA came about when I needed to install a SQL Server instance to handle a new application. It was important that I understood the general admin duties to ensure backups and manage security. At first I didn’t think much of the platform, but as I tried to troubleshoot performance issues, I started to learn about how connections are made, resources are used, and what bad T-SQL looks like. From there, I moved on to more development before coming back to the admin side later.

    It would be great to have a class that teaches you to be a DBA, but really, the job has somewhat varied at each position I have held. There are some specific things that are important at every job, but the exception is often the rule as to how you ensure your environment works well. Once the system is in production, it becomes very hard to change anything, from security to code, without lots of testing and approvals. I find that often trying to work within constraints drives a lot of learning, though not often deep or varied enough to investigate all the options.

    I don’t know everything about SQL Server, but I have developed two very important skills in my career. I’ve learned how to learn, by reading, researching, and practicing new skills. I feel comfortable that I can come up to advanced beginner on a topic very quickly, usually competent enough to make something work. Second, I’ve learned how to ask for help. I’m lucky in that I have lots of friends I can call on for questions in specific areas. If you don’t know someone that’s an expert, I hope you know about #sqlhelp on Twitter and the forums at SQLServerCentral. These are great places to get help on whatever is troubling you about SQL Server.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Upgrades are Hard

    I think the people that run StackOverflow are pretty sharp. They’ve built a well performing site, they’ve worked on some useful open source projects, and they think through their projects. They still have issues with upgrades.

    Taryn Pratt, from Stack Exchange, has written a few nice posts about her experiences upgrading the Stack Overflow databases. The first one was last year, where their multiple server AG environment upgraded from SQL Server 2012 to SQL Server 2017. Just recently she published another post on their Windows Server upgrade, from 2012 to 2016.

    If you’ve never done a complex upgrade, these posts are worth reading. Don’t second guess Taryn, but rather, just read as if you are following along. You have hindsight now, but in the middle of planning this, you will learn about things that might cause you issues. In the Windows upgrade, one of the interesting issues is a VM vs. physical machine issue with drivers. To me, this might be one reason to never bother with anything other than a VM, even if it’s the only one on the machine. A lightweight version of Hyper-V or Xen doesn’t eat much in the way of resources, but can provide some separation from these issues.

    One other thing to note is that you really need your runbook. I constantly see people asking for a checklist for how to upgrade, and there are good general steps to follow, but your environment likely needs to have a custom runbook that covers your situation. The Stack environment is complex, but even I was surprised with 35 pages of steps and notes.

    As with most plans, this one had issues when it was finally implemented. I think Mike Tyson sums it up nicely, which is why you practice your move. It’s also why we can’t necessarily upgrade every year. There’s a reason many companies still have old versions in production (Thanks, Brent Ozar).

    Plan, practice, test, repeat again, and then be prepared to think on your feet. That’s if you upgrade your systems. It’s time consuming and expensive, and I can see why a lot of companies have looked at cloud services, like Managed Instances. Reducing the time and cost to change your OS and/or SQL version is something we should all be thinking about.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.