Voice of the DBA

Tag: software development

  • Does Speed Compromise Quality?

    One of the parts of DevOps that is often hyped is the speed and frequency of releases. Starting with Flickr and their 10 deployments a day, to Etsy deploying 50 times a day, we’ve seen companies showcasing their deployment frequency. Amazon has reported they deploy code every 11.7 seconds on average. That seems crazy, but with so many applications and lots of developers, not to mention each change being smaller, perhaps it’s not completely crazy. With the forum upgrade here at SQLServerCentral, we had two developers (with occasional other sets of eyes reviewing code changes), and while we were bug fixing, we deployed multiple times per day.

    Is that a good idea? Does a rapid set of changes mean that quality is lower and more bugs are released? It certainly can. In fact, if you’re a development shop that struggles with releases and code quality, producing software faster is not going to help you. In fact, if management pressures you to adopt DevOps, and deliver code faster without culture change, without implementing automated testing, including for your database code, and using automated scripts, tools, or something to deploy software, then you are going to get more bugs out faster. You’ll still get to change direction quicker if you find you’re building the wrong software, but you’ll still end up becoming more inefficient because of bugs (And technical debt).

    There’s a fantastic video (long) about refactoring code in two minutes. A bit of an oxymoron since the presentation is nearly two hours long, but the video is from a real project. However, their approach is that good unit testing allows them to refactor code, to change things, without introducing bugs. That’s a big part of the #DevOps philosophy. I always note in my DevOps presentations that if you can’t implement unit testing, meaning you won’t bother, then you don’t get much benefit from CI, CD, or any DevOps ideas. Tests protect you from yourself (and others).

    In many of the DevOps reports, companies that release faster report fewer bugs and less downtime. Since Amazon has increased their speed, they have 75% fewer outages across the last decade, 90% fewer time down, and many, many fewer deployments causing issues. Turbotax made over 100 production changes during tax season and increased their conversion rates. The State of DevOps reports bear this out (2016 here). Thousands of responses show that speed doesn’t cause more bugs.

    Because they work differently.

    If your management won’t let you change the way you work, if you don’t implement automated unit tests (and other types of tests), if you don’t take advantage of version control, if you don’t ensure every change is scripted, then you won’t work differently, and speed will bring bugs.

    You can do better. Your company can do better. Will they?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.2MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Change Approvals

    Part of a developer’s or DBA’s career is getting their work released to some system where others can use the system. Some of us may do this more than others, but we all usually have some role, whether in packaging changes up or actually pushing the button that runs a script or copies files to some live server. This can be an exciting and stressful time, depending on how you feel about the quality of the work.

    Releasing software often isn’t something done in a vacuum. Even in the highly agile, DevOps companies like Amazon and Facebook where developers release code many times a day, there is often some sort of approval process, whether implicit or explicit, before code goes out. Even if it’s just a peer that code reviews something, or a business person that examines a test version and pronounces it correct, another person often weighs in on our changes.

    That’s not always the case, as I know in emergencies, some of us make quick decisions and change or run code that only we have examined. I’d hope that’s the exception, rather than the rule, with most database changes.

    Today I’m curious. Who approves your database changes? Is there a formal process? An informal one? Does the person making the decision even understand the code or do they depend on you to have written and tested solid T-SQL?

    It’s been said that the person closest to the work is often the best person to judge if it should be released, but that’s only partially true. Deploying code is often disruptive. It introduces change, which customers may or may not like. There may be good reasons to release at discrete intervals, rather than whenever the developer things things are working. This may change with heavy use of feature flags or feature toggles, but in general, code releases are interruptions and we want to limit them.

    Unless something is broken, in which case, we often want the change as quick as it can be released. Does that mean we want a different change process? Let us know today.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • I Hate To Send This Email

    I use TrueDelta to report my car status every month. It’s a nice service, allowing car owners to see what experiences others have, and think about which models and years might be a good fit for me. I’ve tracked multiple cars with their service for the last 3-4 years, with reminders from them every quarter to update any repairs I’ve made.

    Recently I got an email from them with this opening: ” We hoped never to have to send this email. A few days ago we learned that TrueDelta has joined the increasingly long list of organizations whose server security has been breached by hackers.” The email went on to note that names and passwords to taken, and that everyone needed to perform a password reset. I applaud them for including “Security breached” in the subject as well as immediately changing everyone’s passwords so old ones wouldn’t work.

    I’ve been hacked at SQLServerCentral, though to our knowledge no data was stolen, merely vandalized. We haven’t ever been able to track suspected data breaches back to SSC, and I hope we never do, but I’m not naive to think that we never will. I hope we don’t, but hackers make determined efforts to gain access to data. At least we are aware of security measures, have a small staff with administrative access, and try to not allow any simple attack vectors.

    Not every company does a great job at securing their data, especially from phishing attacks. There’s a spectrum of how carefully data is protected by organizations, and as we’ve seen from haveibeenpwned.com and plenty of media reports, more and more companies lose data all the time. Some of those companies notify customers (some have to), and I would guess more than a few people have had to send out emails they never expected to send. More of us will dsend those emails in the future, and we should think about that today. Is there something we can do to avoid having to send those notifications?

    There probably isn’t something to ensure it never happens, but we can certainly work towards improving our security. As developers, we shouldn’t have short limits or character choices for passwords. If you wonder why, there’s a great answer at security.stackexchange. We shouldn’t be writing our own authentication schemes, but incorporating code that’s been written, vetted, and reviewed. And make sure we apply patches. Most of the security holes in software are known and patched, but without being deployed. Certainly if new patches become available, we should be able to incorporate them quickly. Above all, learn what SQL Injection is and don’t allow unvetted user input in queries, including those in hidden form fields.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Launching Products

    Over the years I’ve seen a lot of products launched. I still remember the fanfare around Windows 95, and the efforts to make this an event of sorts, back when virtual events didn’t make sense. There have been a few Windows or SQL Server launches, some of which took place at large conferences, but quite a few were launched in different communities with small physical events. In Denver, there were a few times when Microsoft rented a hotel conference space and scheduled speakers, some from Microsoft, and some local. These would be all day events, showcasing new features, with customers talking about their experiences in previewing the product.

    Just a few weeks ago, Microsoft launched Visual Studio 2017, with some fanfare, and two days of virtual events. I watched one, and hope to find some time to watch a few of the second day’s events. While I don’t necessarily learn a lot, I go get a bit excited seeing new technology and how different people use it. It’s inspiring to see demos and solutions work well on stage. I think it’s a welcome respite from the drudgery of dealing with my own or other people’s code, and the various problems it presents, as well as the various bugs and limitations of the tools and IDEs I use. It seems there’s always some devil in the details that slows me down.

    However, for a short time, I get rejuvenated and excited by the new features. I want to dig in and experiment. It can be a struggle to find time, but watching someone demo a new way of working reminds me of the things I love about computers and technology. I think this is one of the reasons I love attending SQL Saturdays or other conferences and watching other presenters talk. I always see some new and interesting things that generate ideas and excitement. I’m really looking forward to SQL Bits in a few weeks (still time to register), and a few days of seeing what others have been accomplishing with SQL Server.

    I also have pride when I’m a part of the company launching a product. At Redgate, we’ve had a number of SQL in the City shows, which I’ve been honored to be a part of in many cities around the US and UK. Last year we tried a streamed version, which was yet another experience, but one I hope we repeat again. And, in fact, I get to be a part of another product launch next week. We’ve already release SQL Clone, but we have a live-streaming launch coming next week. I’ll be traveling over to Cambridge and getting ready for a few presentations that will hopefully inspire you to try the product. If nothing else, check out the stream and see if we show anything that sparks an idea in your mind.

    I know many of us work inside companies, and our product launches are small, to relatively few customers that are often our fellow employees. Some of you work with companies and the results of your work are visible to lots of others. No matter which environment you work in, I hope you feel the same pride and excitement I do when you release a new idea for your users to enjoy.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.8MB) podcast or subscribe to the feed at iTunes and Libsyn.