Voice of the DBA

Tag: software development

  • Giving Computers Ethics

    I was reading a fascinating paper recently about autonomous cars. I’m actually excited about having a car that can drive itself, though I think this is likely quite a few years away, despite the hype. Ever since I read Red Thunder, I’ve thought that we would first get full time autonomous cars that would either be limited in where they were in use, or part-time autonomous cars that could only be self-driving in certain places. Dense inner cities, or maybe isolated highways might be good places to try this, in my mind.

    While we want to do some programming of these cars, we also have a lot of AI/ML systems in place that run models trained to react in certain ways. They identify things that are moving and stationary, trying to determine how the car should navigate and react. The systems aren’t quite as tightly programmed as many of us expect, with if this then that logic. Instead they have guidelines that are decided upon by the designers and then reactions to data inputs and analysis are a little more fuzzy.

    What are the goals? Well, in most cases they are just moving the car safely down a road. In crisis situations, it’s a little more murky. What happens when collisions are unavoidable? How should the car react? Humans often panic and do strange things, but we don’t want erratic behavior from automated systems, so what should we set as goals? There’s a bit of research that was done to ask humans what they would do when they can consider the situation a little more slowly.

    In short, humans make different decisions in different cultures. There are clusters and tendencies in different parts of the world, which is interesting. While people are people and behave similarly in many cases, we tend to value different things, depending on our views of the world. That can be problematic when we start to expect computer systems to be more consistent or predictable. After all, we should decide how computers react and be able to trust our decisions are followed. It is up to humans to imprint our ethical desires as a society on computer systems.

    This is an area where I feel AI and ML systems are moving faster than our ability to comprehend the implications. I would want to have a framework built for automated systems, certainly cars, and then expect all vendors of systems would implement that framework in their vehicles. However, this goes beyond cars, and in any places where we are using software, AI/ML based or not, we ought to publish a comprehensive outline of the way in which our system works.

    Computers have the capability to improve our world and reduce chaos, but only if we agree on the way in which these systems work, and disclose in a transparent way what data they handle and what decisions they make based on that data. I hope that we start to get better about informing the world the goals and operation of our systems. I’m not sure that will happen anytime soon.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Dealing with Technical Debt

    Stop for a minute and think. How much technical debt do you have in your code? Let’s make this easy, take a break from this, close your eyes, and take thirty seconds or so and think about just your code. How much would you rewrite or redo if you could?

    Now, stop and think for another minute. How do you measure the amount of debt? What’s your metric? Leave a note in the discussion after you finish this and let us know.

    There’s an article from a project manager about technical debt and how his team decided to measure this in their organization. They actually set a series of metrics for things they cared about and assigned an actually dollar, well, pound sterling, value to each. They then totaled these for all projects and came up with a total debt number. They then track this, incurring new debt or paying off older debt over time.

    I think that’s an interesting idea. I don’t know that the debt needs to be money, but I do think money is a little better than some abstract metric like tickets or requests. Money is something many of us are familiar with and we hold debt. Assigning a concrete value does let us measure where we stand in a way that we can better relate.

    Just like financial debt, not all technical debt is bad. Sometimes debt lets us get more things done sooner than we might otherwise be able to do. We do need to take shortcuts at time in development to get a prototype working, or deliver a feature or meet some other goal.

    It is easy to overdo things, and just like with finances, overextend ourselves. Paying down that debt can prove to be a struggle in both cases, perhaps even crippling our ability to tackle any other projects.

    If you think about technical debt now, do you view things differently? Do you have a measurement system? Can you tell if your incurring or paying down debt? It might be valuable to implement something and ensure you don’t keep building up more than you can handle. It might even be something to show your manager and get some time to fix a few bugs.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Pulling the Plug

    I’ve worked on some large scale software projects. For the purposes of this discussion, I’ll define a large scale project as one that takes more than a two man-years to develop. I’ve never spent two years on a project, but I certainly have been involved with some that took more than two people, all more than a year to complete to some level. Even while working in a DevOps style environment where we released software every week, I would count a project that followed this process across a year for multiple developers as a large scale project.

    When working on a large project, it’s hard to sometimes keep perspective on whether to keep going or stop and change directions. We often try to continue to improve and fix a project, even when it is not going well. There’s a term for this: Concorde fallacy. I’ve seen this similar situation occur pre-Concorde in other industries, who often have made an investment and continue to do so because of the weight of that previous investment on them. If I’ve spent $20 or $20mm on a project and I am evaluating whether to spent an equivalent amount moving forward, I can’t continue to worry about the money I’ve already spent.

    That money is gone whether I stop now or continue on. What I ought to do is look forward and decide if future spending is worth the investment. Certainly my reputation, and often some pain for switching or decommissioning existing work is to be considered, but that’s part of the value and too often we become afraid of abandoning something we ought to get rid of for a newer, better something else.

    This is discussed a little in the case of legacy technology in this piece about pulling the plug. If you start to avoid maintenance, especially in this day and age of additional regulation, you need to upgrade. If your staff or users are starting to pick other software because you can’t maintain your application, you ought to abandon it and move on. Does some application take too many resources to keep going? Look for something else. In fact, some portion of your staff ought to be making evaluations on alternatives. Not as a full time job, but with some framework to help them perform a cost benefit analysis.

    This doesn’t mean pick the shiny new thing or move to the cloud or make everything self-service, but it should be an evaluation of how well you can get software to your users. One of the best lessons of The Phoenix Project, to me, is that continuing to bet on a bad software project isn’t as good as building a plan to get replace it with something better, often something that is built with a more agile, DevOps process. This may not be the cheapest or quickest way to move, but if can often be the best way to change.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.3MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • How Perfect Should Our Software Be?

    I’ve a fan of Basecamp (formerly 37Signals). I think that Jason and David run their company, business, design, and software in a fascinating way. If I didn’t work for an amazing company (Redgate Software), I’d might see if they want a DBA/evangelist.

    Over the years I’ve read a number of interesting posts from them, including their ReWork book, and I’ve learned a lot. They make me think, give me ideas, and a perspective I might otherwise miss about work, life, or something else. They tend to focus on software and since I’m a part of that business, I like reading those posts. I ran across a good one from Jonas Downey called “The right amount of perfect“.

    The piece talks about the tendency to want our work to be perfect. We want our new software, class, table, method, stored procedure to be right. We craft systems to work well and meet requirements. Sometimes, perhaps often, we over engineer things. At least, the very talented people I know tend to over engineer. The less talented more often than not just barely get things working, meeting only one particular case, and leave behind a load of technical debt, often in a steaming pile.

    Finding a balance for how good our software should be can be hard. I do think that often it doesn’t take longer to write good code than bad code, but it does take time to learn to write better code and practice using patterns that perform well, both in terms of solving a problem and not using too many resources. Early on, when experimenting, it does seem more important to try things, even if they don’t work well. Later, when we are more confident that this particular feature/view/method/etc. is highly likely to be used, we ought to ensure we refactor code or adjust our design to become more “perfect”. We still have to balance how robust we make code given the fact that we always have more work than time, but it should be better than a the first, quick cut at getting a working feature.

    The worry I have had, and likely many of you, is that our quick design, our hasty prototype, will be used in production without any time given to rebuild the underlying code properly. That is certainly a concern, and I don’t have a good solution if your management insists on releasing poor code and living with poor performance. The one piece of advice I can give is to learn patterns that work well for queries, for database design, for error handling, or any other task. That won’t allow you to build an extremely robust system quickly, but it can help you minimize the amount of poor coding you do when under time constraints. You’ll write more good code the first time.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.