Tag: DevOps

  • A Problem with Cowboy Coding

    As we implement more automation and complexity in our systems, we increase the ability of every worker to accomplish more during their day. In many cases, this is a good thing, as many of us have no shortage of tasks to complete. Being able to do more is good. Some technology workers, and likely some managers, view these changes as opportunities to use less people to do the same work. That can be worrisome for many people, especially those concerned with job security. That’s a difficult situation to find one’s self in.

    I’m not sure if this worker was concerned about losing employment or just wanted to get more work, but a contractor put logic bombs in his project that would cause issues at various times. He would then be called to fix issues, getting paid for his efforts. Recently he was sentenced to prison time for his actions.

    Most workers make an effort to do good work and provide value for their time and knowledge. They may still make mistakes or cause problems, but it’s often unintentional, not malicious. We also don’t look to create future problems for others, though certainly we may accrue technical debt over time as we make trade-offs in our work.

    Whether intentional or accidental, we want to limit mistakes and issues in our code. This is one reason that modern development in a DevOps environment isn’t a wild west, cowboy style of work. We use code reviews and continuous improvement to help other developers learn to write better code. We use automated processes with instrumentation and logging to ensure we know what code is deployed when and by whom.

    We may move faster with DevOps, but it’s often safer, more secure, and more accountable than previous methods of building software. It doesn’t prevent mistakes, but every part of the process should be more transparent and auditable, which is good for everyone involved.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Quick Security Mistakes

    How many of you have gotten an urgent request from someone in your organization? Maybe it’s a new database, perhaps a quick, “simple” change to an application, possibly even a new server or share that someone can use to complete their work. It’s something new that you need to do. When that happens, how many of you ensure that you follow all the same steps and protocols to comply with the urgent request?

    A few of you do, but when we’re in a hurry, many of us don’t necessarily complete every step. We may shortcut something to get work done. We may have the best of intentions to go back and complete the work, but in a busy environment, it’s easy to forget to complete that last step, which might be configuring security or running all or unit tests or even decommissioning some resource that a user is done using. Some of you will realize these are big missteps, and some of you will still make them.

    Someone at the Department of Transportation in Coloroado (CDOT) made a mistake like this (thanks to DCAC and Joey D’Antoni for the story). They stood up a virtual machine, connecting it to their local network, but failing to properly secure it. In this case, the machine was exposed to the Internet and connected with a domain admin account. As you might guess, someone got into the machine and executed a ransomware attack on CDOT.

    As Joey points out, a few mistakes were made here, but these are the types of mistakes that anyone can make. Lots of us follow a process over, and over, and over again. Until we don’t. Until we’re distracted, busy, or in a hurry. They we take a shortcut. Most of the time nothing happens, but most of the time is becoming less acceptable. All of the time is the standard, which is why we try to use a DevOps, GitOps, or other process that ensures all the steps are completed. Not most of them.

    Do yourself a favor and build processes to handle your tasks with a script or the push of a button. Ensure the operations are logged and audited. Use these processes to be sure that setting up a new system, database, application, etc. is done in a consistent and secure manner. There’s still plenty of work to do for all of us. These processes will grow and change over time, and need to be maintained. Use your brain for the hard problem solving task of building a process and let the computer execute it, the same way, to completion, every time.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Product Platform Teams

    Is infrastructure a product? Should an Ops team start to feel like they are a team involved in building things rather than maintaining them? It’s an interesting idea that is put forth in this blog from Forrester. The piece postulates that the Ops group should be thinking like a product team, just like developers working on an application.

    When a development group moves to a DevOps style of building things, and starts to take some responsibility for the support of their work, what does the Ops group do? If we push infrastructure as code into our VCS, is there much for the Ops group to do?

    I’d argue there still is. From the DBA side, operational staff can help determine what infrastructure is needed, or what changes need to be made. It’s easy to say that devs should add RAM or CPUs, or even another replica, but do they really know when to add them? Or do they just scale everything up? The latter might be easy on people used to writing code, but it’s not necessarily a good idea for the organizational budget.

    Ops staff can still tune systems, add indexes and rewrite code, and give feedback and knowledge to developers to help them understand how to write better code, or infra-as-code code, the first time. There’s also the need to practice and be ready for various DR situations, including the “whoops, I deleted some data”. There are skills that help handle these situations, which are far different from developer skills.

    This is in addition to designing new systems and setting templates for that infrastructure as code that the developers keep in their VCS. There’s also the need to build self-service tools, which might be where Ops becomes a product team. I think that will be necessary, as tooling is important everywhere. I think Operations might even develop other things, like reports requirements that developers don’t have time to build. Ultimately, though, there is still a need for some Ops work, albeit work that ought to be closely aligned, and shared, with developers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Take the 2020 State of Database DevOps survey

    It’s still 2019, but we’re looking forward to 2020 and wonder how your world has changed since our survey last year. There are a number of questions and this will probably take 20 minutes to complete, but you have a couple incentives:

    • Completed surveys donate $1 to UNICEF
    • You could win a 64GB iPod Air

    You can take the survey here: https://www.surveymonkey.co.uk/r/FWCVCBJ

    Feel free to pass this along to your friends as well.