Voice of the DBA

Tag: software development

  • Trunk Based Database Development

    Early in my career we had a folder that contained our source code. Each of us developers would copy the folder to our workstation and work on our code. To prevent issues, we had read only access to the main folder.  When we thought work was finished, we had to set up a meeting with two other developers, print our code out, and go through a review. If we got two sign offs, we would send our code to a manager to copy into the main folder. Since we were assigned work in different areas, conflicts were rare and this mostly worked, although it was slow.

    Over time I evolved to use various VCSes (Version Control Systems) for development work, with different paradigms for working in teams. These at times included code reviews, but like comprehensive testing, these were optional (and often ignored) when time was short. As an industry, I think we’ve gotten better with code reviews in some sense. The popularity of git and the integration of pull requests (PR) has made it more likely that others will review your code as a habit rather than a rare activity. I certainly see constant calls for someone to examine a PR in the Redgate Slack channels.

    I think this is a good move, but I was listening to a few people talk about trunk based development recently, which has less of a chance for code reviews. In this model, developers commit their changes directly to the copy of code everyone is using. In a PR model, developers work in branches, and only have a PR is reveiwed, is the changed code merged with existing files.

    Trunk based development is a more trustworthy model. We trust developers to make changes to the code everyone is using. This is often what many of us do in database development when we have shared development databases. A change made by one developer is visible and affects everyone else. While some organizations struggle with conflicts, this does ensure that problems are visible early and that your team must learn to communicate with each other.

    One way isn’t better than the other, and if you aren’t familiar with one of them, give it a try. Work on a small PoC with someone else and experiment with how well you can make changes and what conflicts come up. Many of us get stuck developing code in a singular way unless someone forces us to change. The software development process has evolved a bit over the years and you might find that a different style of putting code together may suit your team better.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Giving Computers Ethics

    I was reading a fascinating paper recently about autonomous cars. I’m actually excited about having a car that can drive itself, though I think this is likely quite a few years away, despite the hype. Ever since I read Red Thunder, I’ve thought that we would first get full time autonomous cars that would either be limited in where they were in use, or part-time autonomous cars that could only be self-driving in certain places. Dense inner cities, or maybe isolated highways might be good places to try this, in my mind.

    While we want to do some programming of these cars, we also have a lot of AI/ML systems in place that run models trained to react in certain ways. They identify things that are moving and stationary, trying to determine how the car should navigate and react. The systems aren’t quite as tightly programmed as many of us expect, with if this then that logic. Instead they have guidelines that are decided upon by the designers and then reactions to data inputs and analysis are a little more fuzzy.

    What are the goals? Well, in most cases they are just moving the car safely down a road. In crisis situations, it’s a little more murky. What happens when collisions are unavoidable? How should the car react? Humans often panic and do strange things, but we don’t want erratic behavior from automated systems, so what should we set as goals? There’s a bit of research that was done to ask humans what they would do when they can consider the situation a little more slowly.

    In short, humans make different decisions in different cultures. There are clusters and tendencies in different parts of the world, which is interesting. While people are people and behave similarly in many cases, we tend to value different things, depending on our views of the world. That can be problematic when we start to expect computer systems to be more consistent or predictable. After all, we should decide how computers react and be able to trust our decisions are followed. It is up to humans to imprint our ethical desires as a society on computer systems.

    This is an area where I feel AI and ML systems are moving faster than our ability to comprehend the implications. I would want to have a framework built for automated systems, certainly cars, and then expect all vendors of systems would implement that framework in their vehicles. However, this goes beyond cars, and in any places where we are using software, AI/ML based or not, we ought to publish a comprehensive outline of the way in which our system works.

    Computers have the capability to improve our world and reduce chaos, but only if we agree on the way in which these systems work, and disclose in a transparent way what data they handle and what decisions they make based on that data. I hope that we start to get better about informing the world the goals and operation of our systems. I’m not sure that will happen anytime soon.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Dealing with Technical Debt

    Stop for a minute and think. How much technical debt do you have in your code? Let’s make this easy, take a break from this, close your eyes, and take thirty seconds or so and think about just your code. How much would you rewrite or redo if you could?

    Now, stop and think for another minute. How do you measure the amount of debt? What’s your metric? Leave a note in the discussion after you finish this and let us know.

    There’s an article from a project manager about technical debt and how his team decided to measure this in their organization. They actually set a series of metrics for things they cared about and assigned an actually dollar, well, pound sterling, value to each. They then totaled these for all projects and came up with a total debt number. They then track this, incurring new debt or paying off older debt over time.

    I think that’s an interesting idea. I don’t know that the debt needs to be money, but I do think money is a little better than some abstract metric like tickets or requests. Money is something many of us are familiar with and we hold debt. Assigning a concrete value does let us measure where we stand in a way that we can better relate.

    Just like financial debt, not all technical debt is bad. Sometimes debt lets us get more things done sooner than we might otherwise be able to do. We do need to take shortcuts at time in development to get a prototype working, or deliver a feature or meet some other goal.

    It is easy to overdo things, and just like with finances, overextend ourselves. Paying down that debt can prove to be a struggle in both cases, perhaps even crippling our ability to tackle any other projects.

    If you think about technical debt now, do you view things differently? Do you have a measurement system? Can you tell if your incurring or paying down debt? It might be valuable to implement something and ensure you don’t keep building up more than you can handle. It might even be something to show your manager and get some time to fix a few bugs.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Pulling the Plug

    I’ve worked on some large scale software projects. For the purposes of this discussion, I’ll define a large scale project as one that takes more than a two man-years to develop. I’ve never spent two years on a project, but I certainly have been involved with some that took more than two people, all more than a year to complete to some level. Even while working in a DevOps style environment where we released software every week, I would count a project that followed this process across a year for multiple developers as a large scale project.

    When working on a large project, it’s hard to sometimes keep perspective on whether to keep going or stop and change directions. We often try to continue to improve and fix a project, even when it is not going well. There’s a term for this: Concorde fallacy. I’ve seen this similar situation occur pre-Concorde in other industries, who often have made an investment and continue to do so because of the weight of that previous investment on them. If I’ve spent $20 or $20mm on a project and I am evaluating whether to spent an equivalent amount moving forward, I can’t continue to worry about the money I’ve already spent.

    That money is gone whether I stop now or continue on. What I ought to do is look forward and decide if future spending is worth the investment. Certainly my reputation, and often some pain for switching or decommissioning existing work is to be considered, but that’s part of the value and too often we become afraid of abandoning something we ought to get rid of for a newer, better something else.

    This is discussed a little in the case of legacy technology in this piece about pulling the plug. If you start to avoid maintenance, especially in this day and age of additional regulation, you need to upgrade. If your staff or users are starting to pick other software because you can’t maintain your application, you ought to abandon it and move on. Does some application take too many resources to keep going? Look for something else. In fact, some portion of your staff ought to be making evaluations on alternatives. Not as a full time job, but with some framework to help them perform a cost benefit analysis.

    This doesn’t mean pick the shiny new thing or move to the cloud or make everything self-service, but it should be an evaluation of how well you can get software to your users. One of the best lessons of The Phoenix Project, to me, is that continuing to bet on a bad software project isn’t as good as building a plan to get replace it with something better, often something that is built with a more agile, DevOps process. This may not be the cheapest or quickest way to move, but if can often be the best way to change.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.3MB) podcast or subscribe to the feed at iTunes and Libsyn.