Voice of the DBA

Category: Editorial

  • Improving DevOps Automation

    We use a lot of automation at Redgate. As a company that builds software, we want to ensure that all the changes from our teams get integrated and tested in a timely manner. We have multiple teams on some products, and while we do have regular meetings between them, it can be easy for a developer to miss some update on a change others are making and break something with their own work. Worse, they could cause a regression or security issue, which we work hard to avoid.

    Recently I saw a post from one of our lead software engineers about the build process. While we have lots of pipelines, the general process hasn’t changed in many years. In some sense that’s good, because we focus on building better software, not the process. We don’t change just for the sake of change, or because a new team or division lead likes one tool over another. I’ve seen customers where they move from Jenkins to Azure DevOps because someone in charge “likes one better.” Not a good use of time.

    However, you do need to evaluate whether your process is meeting your needs. In our case, we surveyed lots of developers to get their thoughts on pain points and issues. The build and release process was consistently listed as a pain point, with releases often requiring a dev to manage it from their workstation, preventing them from spending time building software for customers.

    That might be the big lesson I saw in the write-up. We realized that a non-negligible amount of time was being spent by developers on the process of moving bits rather than building the software. This led us to re-examine how things are done. In particular, we looked at the agent OS (Linux v Windows) and containers. Both of these are more viable technologies now than in years past, and they can reduce costs while smoothing the process. As a result, we are developing a general contract that helps us decide how to change our process. This is similar to an API, that doesn’t specify the tool to use, but rather the inputs, outputs, and what the effect should be. From this, we’ll start to help teams move forward is changing their process as we have time.

    The big takeaway right now is that this is pushing us to use containers, which simplify the steps and allow us to easily move from Azure to AWS to on-premises, or really any environment. We can switch builds across different platforms and more easily scale up or down as needed. It will take some time, and some of our software will be more challenging in containers. However, we are also seeing benefits from customers, and at some point, I expect we’ll provide containers for certain functions that make it easier for the end-users of our software to also deploy and upgrade their tools.

    DevOps is an ongoing process. Not a set of tools that you change just because, or a way of building software that matches what another organization does. Instead, it’s learning, experimenting, and evaluating how you can be more effective. Then adopting what you’ve learned and repeating the process. Keep improving, and you’ll find that you can produce software quicker and at a higher level of quality while improving the skills of your engineers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Enterprise Software

    “I’ll go in the store; it’s built for that.”

    That is a quote from Jessica Kerr, who is a developer. She has a great blog on software development, called The Enterprise Eats Software. It starts with a poor experience for an online order for Lowe’s. Interestingly enough, as I read this, I was starting a home renovation project, but I didn’t have a bad experience, I had a great one. Read Jessica’s blog, and see what you think.

    For my short story, we had contracted for a bathroom project, but we forgot to get some of the fixtures. The contractor came a day early to talk about things, and we realized we needed a shower diverter valve and trim kit. We looked for the trim, which is what was important to my wife, and found a valve to fit it. While the contractor called two local supply houses, I looked at Amazon. The suppliers didn’t have the valve, but Amazon did, noting delivery in two days. I ordered it around 10:30 am that day. It arrived around 11:00 am the next day, a day early and before the existing shower had been demoed.

    That was fantastic. It made me think why would I want to use any service that wasn’t that amazing and quick. I got an email at my desk with a picture, telling me the package had been delivered and showed me where it was. I walked it upstairs thinking that calling multiple suppliers and then driving around town would have been a pain,  not to mention a time sink.

    I don’t often find a lot of large companies do a good job with their software integrating into real world operations. A few do, and apart from Amazon, I’ll say build.com was incredible for us to get a tub and Wayfair got us a vanity very quickly (too quickly, actually). Both have built great systems that not only took the order but updated us and handled the complexities of shipping large items. Not to mention they both had an incredible selection available to peruse easily and quickly. Just finding a place to look at something like a tub in person is incredibly difficult.

    Apart from the retail challenges, just the general experience from many enterprises leaves something to be desired. They just aren’t good at being agile, flexible, and maybe more importantly, constantly improving. They get caught up in some of the hassles, like bureaucracy, power struggles, too many rules, etc. They don’t know how to operate in a flexible, agile, constant change environment.

    I’m reading Project to Product, which in many ways sees a lot of the same problems. Software is disconnected from the goals of the business, and too often there are individuals and processes that get in the way of becoming more effective across teams and partners. BMW is one of the success stories here, and I still think they’re behind Tesla in their industry. They might catch up, and if they do, it’s because they are learning to be a better software company, not a better manufacturer.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Do We Need to Learn Linux?

    This week we got an announcement about SQL Server in Containers. Microsoft put out a blog noting that the beta program for SQL Server on Windows containers is suspended. They didn’t give any details, citing “ecosystem challenges and usage patterns” as a reason so suspend the program for the foreseeable future.

    Not only that, they noted that they were deleting docker hub images for express and developer, which makes sense from their perspective. They don’t want to support these. From the developer perspective, however, this will break and flow, pipelines, or even local scripts from developers. I understand the desire to delete images, but doing so without any notice doesn’t seem like fair treatment for people using them.

    This means that if you want to run SQL Server in a container in production, you likely have to use Linux. Windocks has an alternative with Windows, but Microsoft not supporting this is disappointing. I think that containers will be the future of database development, as they standardize and simplify how we can get an environment up and running. They also ensure consistent environments between developers, branches, automated systems and more. I think it’s going to take years, but that’s the direction where I think many of us will go over time. Those days of the Developer Edition will be gone, and instead we’ll just run a container.

    I’ve been using Linux containers for a few years, and they make it easy to quickly set up a clean environment with a new instance and a database or two. In that time, I’ve had to use a little Linux at times, but really, I start a container from a command line and then it acts like an instance I’ve installed. There isn’t a reason to actually do much in Linux. At least not for the core database stuff. Most of the ways that you’ll work with a Linux instance as a developer stay the same.

    For system administrators and DBAs, however, you might need to learn more. While setting up an instance on Linux isn’t hard, and easier in a container, if you are called on to work in bash, execute sudo, or some other Linux command, you might want some basic familiarity. Fortunately, you don’t need another workstation as Windows 10 includes a Windows Subsystem for Linux (WSL) that let’s you play around in Linux inside Windows 10.

    I think Linux is fascinating, and it’s evolved in many ways since I first saw it in 1991. Even if it’s not something you expect to use daily, spending a few hours learning how to navigate around and get things done is good for your mind, forcing you to learn a bit. The San Diego TIG did a series a few years ago, and you might go through those meetings or pick up a book and spend some time playing around with something new. You never know when this might be required in your career and a little familiarity could give you a jump start in the future.

    Do we need to learn Linux? No, but it’s not a bad idea to build a few skills in this area.

    Steve Jones

  • Fixing Report Visuals

    I’m not a visual designer, and I find that as I learn a bit more, many of my previous reports contain lots of silly mistakes. They’re often full of data, which I like, but which can be hard for others to follow and understand.

    I need to do better, and I do ask for help at times from friends. I also need to get a better process for evaluating if I’m doing a good job. I was thinking about that as I saw a makeover article from storytellingwithdata. It gives a short process of things to consider when building reports.

    With most of our software, we go through iterative approaches, either by ourselves, or with others, to try and build something that meets the needs or a client. Reports involve the same process, but I know that often the report process is my showing them something very raw and them asking for changes.

    What can be more time consuming is that the numbers are often the thing a client is concerned with early on. Are we calculating things correctly and including enough data. We might go back and forth quite a bit on this part of the report process.

    Once we agree, and they are happy, I think I’m done and can move on. However, often this is where they start to now care about the visual appear. The layout, colors, etc., and I am drawn into another report writing session, spending more time on something that is far, far less interesting than writing queries or other software code.

    I can see why tools like Power BI, which let the client mess with the visual design, have a lot of appeal. As a software developer, I’m out of the loop here, or more out of the loop. Let them decide they want a vertical or horizontal bar chart, or a stacked line chart. What do I care?

    I know visualization, layout, colors, etc. are important. They’re not enough of my job to care, but I do think someone needs to. I’m lucky in that my company has artists, so I can send over slides, reports, etc. and get hints about what I should change. It’s certainly something I wish I’d have had in the past at other jobs, since anything beyond a simple graph or table is well outside my wheelhouse.

    Steve Jones