Voice of the DBA

Tag: software development

  • The Test Mistake

    Last year a company was testing their software in a dev/test environment and accidentally sent out an “Integration email” to their customers.  They blamed this on an intern, though a little jokingly, and plenty of people shared their own stories about making mistakes.

    I did this once, and only once. I was doing some software testing on a copy of the production database and accidentally sent out thousands of emails to customers. I hadn’t scrubbed the email addresses in the copy, but since I also hadn’t scrubbed the data, customers got an “extra” email for their account and few complained. My boss, however, complained quite a bit and with good reason.

    These days using production data in dev/test environments isn’t really an acceptable practice. Plenty of organizations do it, and hopefully, they are taking precautions. Kendra Little wrote a little about what happened and what should be done, including scrubbing out sensitive data. Not enough people do this, though I am seeing more and more companies looking for solutions that prevent data release.

    This isn’t an intern’s issue. I appreciate someone making a bit of a joke here, and hopefully, there were engineers that taught the intern something. The bigger issue is that we make mistakes as humans and our protocols and processes should expect that. This is one reason why adopting automation in our software process is important. Whether that’s deploying code to the right production server or refreshing the non-production databases, we want to be sure that we limit the number of human mistakes.

    There are always places where we will use humans to perform a process. Whether that’s typing code or clicking something. It’s entirely possible someone clicks the wrong button or types the wrong item. As much as possible, we can use automation to provide a safety net to prevent these issues from reaching customers. We can’t prevent every mistake, but we can continue to improve our process but adjusting it over time. At the very least, we ought to prevent the same mistake from happening over and over.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Think Marathon, not Sprint

    I saw an article about automating responses to security issues being a marathon not a sprint. The articles gives a few examples of different levels of automation response to situations, noting that each of this is a different level of maturity in the organization. At early stages, the response is mostly alerting a human to take action. Later, the automation will make some changes, but still defer to a human for more actions. The final example is automation handling most of the issue itself. Note, none of this means humans are unaware of what responses are being made.

    The idea is that improving security with automation is something that takes place across time, as the organization matures and becomes more comfortable and trusting of automation. It’s a marathon, where we push, but we know this will take some time to get to the end. It’s not a sprint where we make a quick fix and get a result.

    Actually, I think a lot of things are marathons in the technology area. That’s if we are looking to improve how we work with automation. If we like just firefighting issues and building quick patches, then we’re constantly sprinting.

    A lot of the work I do in advising clients about DevOps is to get them to think marathon. There isn’t really a finish line, but we are racing our competitors and trying to improve how we work. We just need to recognize that this is a process that takes months, not minutes. We want to mature and evolve processes, making them better over time. We also start small with our scope, hoping that we expand things to the entire organization over time, but again, that time is months.

    This was my same approach as both a developer and DBA. Find something that I can automate to make better, start to improve it, learn from success and failure, and repeat. At some point, I usually found something was working well enough to move on to a new area to improve. If I needed to come back and continue to improve something, I could do that as well.

    I don’t like sprinting. In real life, or in technology. I prefer to think marathon. We are pushing to achieve something, but with the further away future in mind, not the next few minutes.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • It Isn’t Worth Automating

    Every year I’m responsible for a portion of the Database Weekly newsletters. I used to do half, but right now Kathi, Grant, and I are on a rotation. Our boss puts out a schedule each year, which tells me the weeks for which I am responsible. Each January I then set up a recurring appointment for Mon/Wed/Fri on my weeks to remind me to curate articles and write an editorial.

    Each year I also think about whether or not I can automate this. Last year I spent a few hours looking into the Exchange APIs for creating appointments, at the C# code, and PoSh (which is a nightmare). I spent a few hours, got frustrated, and gave up, just manually creating appointments.

    This year I took a slightly better approach and tried to figure out how long it takes to create an appointment. I got a stopwatch and went through the process. Each recurring appointment takes about 30 sec to create and (a repetitive) 15 clicks. I have 17 weeks, so this is about 9 minutes (rounding) to get this done.

    That doesn’t seem worth automating. In a decade, I’d spent 90 minutes doing this. I’m not sure I could automate it in 90 minutes. I might learn a new skill, but I don’t do much calendaring, and honestly, I’m not sure I could do something I’d think would be reliable, portable to others, or even useful in other ways.

    Throughout my career I’ve had a lot of tasks like this as a DBA or developer. They are often short tasks, things I do fairly rarely, and are not tightly defined. Often I just do a little manual work rather than spend too much time trying to automate things. In my life, I’ve found similar tasks, where I’m not sure the time spent building automation or the maintenance requirements over time, would be worth the effort.

    I am, however, a big proponent of automation, and I think there are many places where the time spent is worth the effort. Periodically cleaning log files from servers, running data loads of know files, DevOps CI and CD processes, and more are worth the effort. Many people need these processes running often, and the work to get automation running becomes a time saver.

    Not everything needs to be automated, or even should be. Certainly, when the manual time or effort requirements are small, it pays to do a little analysis. If you disrupt the way humans do their work, then be sure your solution saves time or effort. As technologists, we often reach for the scripting/programming tools to solve problems, but be sure that you include a little thought before you start coding to be sure the process is worth automating.

    Steve Jones

     

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • A New Language

    In my life, I’ve mostly dealt with people that speak English. I tried Latin in high school and Japanese in university, but I wouldn’t say I learned a lot. This past year I’ve been working on Japanese (konichi wa) and French (Bonjour) on a regular basis, but I’m still fairly uni-lingual. At least compared to many of the people I’ve met in my travels that speak multiple languages. I’m mostly just trying to read a bit, and that’s quite hard.

    In my career, I haven’t met many technologists that only know one programming language. Even the SQL professionals know a little C# or a scripting language. Quite a few people know a bit about many languages, and some are very competent. I used to think it would be good to master one language before moving on, but I find that trying different languages can give you an appreciation for others, or even an appreciation for your primary one. Comparing and contrasting code might even help you learn faster.

    These days many of us are being asked to handle more tasks, and we may be required to support or work with code that others have written. As a result, I think it makes some sense to have a basic knowledge of other languages that you might encounter. I can often look at code in different languages and get an idea of what’s happening. I might even think about fixing simple bugs if needed.

    I saw a Data Exposed edition recently with Hamish Watson that was titled KQL: The Next Query Language You Need to Learn. I had heard of it, but never used it. After watching the episode I played with it a little. While I don’t know how important it is, I do appreciate I may run into it and will likely spend a small amount of time writing some queries and familiarizing myself with some of the common aggregates and other types of structures.

    Today I’m curious, which do you think you would learn as a next language? Not that you’d give up your current work in T-SQL or something else, but what else might be good for your career, or perhaps be exciting to you personally?

    I’ve enjoyed doing some work in Python and PowerShell over the last year, and time spent in R before that helped me appreciate those languages. There are places I’d choose to work in those over T-SQL, especially for data not already in a SQL Server.

    Take a moment and leave a comment: what new language would you like to learn and why?

    Steve Jones

     

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.