Category: Editorial

  • We Need DevOps for Performance

    I read this nice piece on CosmosDB and setting performance levels. It covers how you set some level of performance for Request Units (RU) and CosmosDB handles the rest. Great, right? Set the level of performance you need and that gets handled. The question I have is how do you know what level of performance you need?

    This is one of the issues with the Cloud services that I see. It doesn’t matter if you use PaaS or IaaS, most of us really don’t know what level of performance works. We tend to guess, and far too many of us don’t use regular monitoring to decide if we are over or under powered. I find many DBAs and sysadmins would prefer to be over-provisioned and then let the server trundle along at a lower resource usage so that no one complains.

    When we move to a cloud type service, we tend to be more cost conscious, which makes sense when we’re paying by the minute. We want to minimum level of hardware we can get, but we don’t want to cause unnecessary complaints, either from customers because the system is slow, or from the CFO because costs are high. Using your old method of over-provisioning hardware (or DTUs) usually causes complaints from the CFO.

    The piece goes into some ways that you can start to evaluate your performance level for CosmosDB, and how to deal with throttling while you tune your RUs with a new application. This works great if you’re in a (more) greenfield area of development. Not so great if you’re lifting and shifting some application from another platform. Then we might need to ensure that we are responding quickly to issues, or better yet, have scripted responses that scale up or down.

    The same thing should be done for our relational systems. The Ops part of DevOps needs to be using monitoring and instrumentation to measure performance, adding capacity as appropriate, which should be before users realize there’s an issue. With today’s virtual systems, adding CPU and RAM usually is fairly easy, and it’s easy in the cloud as well.

    Of course, all this monitoring isn’t just to add capacity. Having a better sense of what’s going on can help you pinpoint poor code. Getting someone to fix that code becomes a lot easier if you can show that better code would cost less for our systems. It can be amazing how much more developers care about their code when the CFO gets involved.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Practice Until You Don’t Get It Wrong

    I wrote recently about a bad first day for an intern. He/she was fired, without cause in my opinion, when a production database was damaged while following a document for developer setup. The situation felt like a mistake, and one that wasn’t necessarily the fault of the individual. To me, this was extremely poor handling of the situation from a CTO.

    In the discussion for the piece, someone pointed out that it might not just be a new employee that makes a mistake that causes downtime. Certainly an unexperienced employee could have caused the issue, but I know there are people with lots of time in a position that make similar mistakes. It could be that someone that has been there a long time followed a poorly documented procedure, or applied the procedure to the wrong situation.

    In sports and the performance arts, I’ve heard a saying that says: amateurs practice until they get it right, but professionals practice until they don’t get it wrong. It’s the difference between just gaining a skill that you complete and continuing your learning to ensure that the execution of that skill is ingrained into your being. You internally build the muscle memory and skill to complete a task correctly over and over with practice.

    In the database world, this might be the difference between someone that has restored a database (the amateur) and someone that regularly practices restores of not only full database backups, but differentials and logs, tail log backups, filegroup restores, and more (the professional). The former has a skill, but the latter is confident and likely to complete the skill in a variety of situations without error.

    Practice won’t prevent mistakes, especially under pressure, but lots of practice does help us learn how to perform better. We learn what options are available, what issues might arise, and which choices we should be making. Just as professional athletes and musicians can make mistakes, we can as well. However, those mistakes should be rare. We should get most things right as the product of numerous hours of working deeply in, and understanding, our craft.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.9MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Most Wanted Upgrade

    I replaced the case on my desktop recently, though not without a few struggles. A damaged motherboard in the move, and then a DOA replacement caused me a bit of a headache while troubleshooting. Fortunately, I was able to get a few minutes with Glenn Berry to find the issues. Things are back up and running now, which is good. I like my desktop and I’m more comfortable doing most of my work there.

    The process got me thinking about upgrades, especially when I got a look at Glenn’s newest machine. He hasn’t published details, but he did post a few results for his Intel Optane 900p PCIe cards. He also dropped a picture on Twitter of his Threadripper CPU. That’s quite a Task Manager display for a desktop workstation. I don’t know that I need that much power, but it’s still impressive and exciting. Seeing a recent picture of Andrew Kelly’s 38″ 4K monitor for his new workstation has me drooling a little, even though I don’t really need more hardware.

    It’s Black Friday, and while I doubt many of you are out shopping for yourselves, I do wonder if there are upgrades that you’d be excited to get. If someone were to gift you some computer hardware, what would you want? It could be hardware or software, desktop, mobile, whatever type of upgrade you’d want. Let me know today what you wish someone would give you for a present.

    I’m not an enthusiast. I think my son spent almost as much on his video card upgrade as I did for a motherboard, CPU, and RAM. However, I did get a little excited in looking at the various options in the Denver Microcenter store last week. From 3D printers to CPU water coolers to fancy video cards, I was tempted by no shortage of high end hardware. I don’t really need anything, and even passed on adding RAM to my desktop. 32GB works great for me, and while I might squeeze a touch more performance out of another 16 or 32, I don’t really need it.

    I am tempted by a set of larger, high resolution monitors. The idea of a couple of large, 38″ monitors that would spread work across my desk is tempting. Probably overkill and silly, but enticing. If I were to really dream about some upgrade, I’d like to get some faster internal 3D storage for my system. I hate waiting for programs to start, and getting some faster internal NVMe storage to replace a legacy 1TB HDD and get my system moving quicker.

    At heart, I’m a practical computer user, but I can appreciate the effort that some people put into build their systems. I’m just not sure that I’d want any of the fancy RGB lighting or as many cores in my video card as I have in my CPU. I’m also quite pleased with my mobile devices, so I don’t want an upgrade there, though I wish I could get a device with a different (bigger) battery.

    You might feel different and perhaps you can change my mind today. What cool upgrades would you make if cost were not an issue?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Pressure to Perform

    I’ve worked as a technical person for most of career. I’ve often been responsible for building or managing systems, and ensuring they operated according to some set of requirements. If things didn’t work, I’d have to fix them, but often there wasn’t the threat of losing employment unless I was grossly incompetent. If I made a strong effort, I’d get other another chance, or maybe be shuffled to work on some other application or system.

    Throughout my career, I have worked with various salespeople in different companies. Either they were my customers/clients or they were just co-workers. However, they operated under a different mindset. Most salespeople are expected to make a certain amount of sales in a time period, and if they don’t, they may be let go. In fact, in quite a few companies, a salesperson couldn’t miss their target more than two time periods in a row without finding themselves out of a job.

    I realize that sales isn’t like the creative work that developers do, or the tedious work of managing systems, but what if we were treated like salespeople? Do you think you’d want to work in an environment where you had some quantifiable measurement, such as uptime for a quarter, plus tickets closed, and if you couldn’t meet the goals, you’d be let go from your DBA position?

    Imagine if you were a developer. You might have the demand that you meet the expectations for delivery and you are held accountable for estimates within some amount of variance. Perhaps as a developer there would be a requirement to complete xx amount of stories in some time period. I don’t this would work well. I think we’d try to make longer estimates and more granular stories to better define out work, but I’d also expect the business might press back and put more pressure on us.

    In plenty of organizations, there are these pressures. I certainly see developers working harder to get things done, not because of the threat of their job, but because many of us that choose technical work feel a sense of pride and responsibility in the work we do. We work long hours to meet deadlines, sometimes because we’re afraid not to, but usually we do want to complete our projects in the best way we can.

    I think we’d all be worse if we were treated like salespeople, and likely many of us would have more job changes. I also think our organizations would suffer more, with higher turnover and a lack of detailed institutional knowledge among those that build systems. Certainly some high performers might thrive, especially if we received large bonuses, like salespeople often do, but I also think we’d find shoddier work overall, with applications even more cobbled together with patches and prayers, with the hopes that our workload doesn’t overwhelm our systems.

    I know I wouldn’t like working in that type of environment, and why I’m not sure I’d ever want to go into any sales role.

    Steve Jones