Voice of the DBA

Tag: software development

  • Building Recommended Software Practices

    Many of us work inside an organization that has a process for building and deploying software. We may find our org doing this well, or we may feel our process is poor with lots of room for improvement.

    A lot of the discussion around how to be better at building software in the last ten years has been around the philosophy of DevOps. This concept doesn’t really prescribe how to build software, but give you goals to aim for. That means you still need to take the ideas of flow, feedback, and learning and decide how you implement them with your staff. What practices do you follow to ensure you can deploy quickly anytime your code is done? These can include ensuring you’ve tested code, getting feedback from customers, and more.

    I ran across a post on recommended software engineering practices for an organization. The list includes seven things you should do:

    1. Keep documentation in the code repo
    2. Have a mechanism for test data creation
    3. Use rock solid database migrations
    4. Create templates for new projects
    5. Automate code formatting
    6. Automate a process for new dev environments
    7. Automate preview environments

    This is a set of things I often preach to customers as well, especially 2, 3, 6, and 7. I often focus on the database and having curated test data, migrations you can count on and easy setup is important. And, of course, with SQL Prompt, you don’t need formatting ;). Just kidding, that’s important, too.

    These are solid practices, and none of them are that hard to set up, but they do require some discipline and willingness to work as a team and maintain your process across time. Each of these items needs some care and feeding across time to remain relevant and helpful to your staff.

    Do you have good software engineering practices? Are you proud of them and would you bring them to a new position? Or perhaps you wish your team would adopt better habits and a different mentality towards building software.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Coding Magic Values

    It’s 2023. I would hope all software developers would learn that hard coding specific values in your system is more likely to cause issues than not. Across the years, we’ve learned not everything is installed on the c: drive, or that not everyone wants to put all data in a Documents folder (or in OneDrive). We’ve learned that any sort of magic number is poor practice, and we ought to know that hard-coded names are problematic as well.

    Yet, we still see it happening.

    This week I was reading about an admin issue in the Microsoft TechCommunity. This is related to Azure Managed Instance, but it’s really an In-Memory OLTP issue. That was introduced in SQL Server 2014, so I know the code for this was likely written in the 2011-2013 timeframe, but how can this type of issue get through code review and be released?

    In this case, the name of a filegroup is set specifically to XTP. It’s a logical name, and I’m sure that some developer thought that things might be faster with a known location. That doesn’t make sense, and while this might not be an issue for most customers, I’m sure there have been some databases built with a filegroup called XTP. After all, there are companies named XTP. What about if this feature evolves to allow a second filegroup, maybe because of some distributed architecture need in the future? Are there then code paths looking for XTP or XTP2?

    As much as possible, avoid coding values in your code that a user might enter as data. Names, paths, etc. Just don’t do it. Use variables, which are in every language, and let those values be read from the environment. This ensures that you don’t end up with weird support requests from customers because they chose the same value you did.

    Steve Jones

  • Forgoing Tech Investments

    The US was hit with a number of storms over the Christmas holiday weekend. This disrupted air travel for many airlines and their customers, but one of the worst hit was Southwest Airlines. They accounted for most of the cancellations, over half of their scheduled flights at one point.

    A number of places reported talking with Southwest employees who blamed the lack of tech investment by Southwest over time, noting this caught up with them. The Chief Operating Officer disagreed, saying that their scheduling system is the best in the world, even as the CEO noted that their scheduling software couldn’t keep up and they fell back to manual operations.

    Most of us likely have no idea of how Southwest software works or the scope of the problem. This airline does tend to operate differently than many others in that they mostly fly point to point, rather than using hubs. Possibly they have the best point-to-point scheduling software in the world, but it still couldn’t keep up with the storms covering much of the US.

    There’s an interesting perspective on Facebook, supposedly from a pilot with 35 years of experience with SouthWest. If you don’t want to click, his view is the hands-on CEO retired years ago and accountants were appointed as CEO and COO. They improved the money flow, but neglected investments in tech and weren’t aware of how the business really runs day to day. The infrastructure and software deteriorated, and they’ve had many small issues, but issues that were bigger than other airlines. They’ve started turning around with a CEO that is more hands-on, but they’re digging out of a hole.

    Like many of you, I’ve built and operated software over the years. I sometimes realize just how hard it can be to keep up with the demands of customers for adjusting how our systems work. I also know that it’s easy to slow your investment in a system that appears to works and limit your efforts to just maintenance work. Allan Hirt wrote about this.

    This does bring up the issue of investing in systems and maintaining them over time. I see why many companies would prefer to purchase software and let someone else manage the investment in ongoing development. I also know that for companies that see software as strategic, likely there needs to be regular investment, upgrading and refactoring code, as well as finding ways to scale higher and use resources more efficiently. Especially for databases.

    The battle between enhancing software and reducing technical debt is a constant one. I see this struggle being one that project managers and developers never agree on, but in the companies that seem to thrive, there is a balance. Perhaps it’s splitting the sprints, perhaps it’s allocating regular time during each development period, or maybe there’s another way.

    One thing is certain. We need to find a balance. Otherwise we might get into the situation where a complete rewrite or replacement of software is warranted; a situation that is almost always very costly.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Self-Service Databases

    When I first started work as a software developer, I knew that getting an environment set up where I could compile a project might take a few hours or a few days. The complexities of how people built software projects, the dependencies, and more were handled in a very immature manner. These days I can often download a project from GitHub and compile it in minutes, with many of the issues handled automatically with various tools such as complex config files and package managers.

    In the world of database development, I often find people struggle to get a database set up with a project. Whether you are coding in SQL, C#, or Java, many organizations haven’t got a standard way of bringing up a database environment for developers. Some have a shared database for a team, which can be better, but can also lead to collisions of code as the pressure to get software built grows and team size increases.

    This leaves out the challenges of data, which can be changed by others, or lack the size/shape/skew to properly reflect what the production code has to handle. There are no shortage of bugs that come from incomplete data sets used in development that are unrepresentative of the live database.

    Do you have a way for developers to self-service the database assets they need? Whether this is a new dev database, a refreshed one from prod, or even a point in time from development.  Is there a way developers can get a database into the state that matters for them. When developers are ready to work on a problem, the last thing we want is a significant delay to them getting started.

    One of the things DevOps stresses is that we want to reduce friction and tedious, manual work were possible. Let the computer manage the tedious things. This should be how we manage those database environments. We also want some consistency among all developers, QA, CI, and any other environment, to ensure we can reproduce problems to troubleshoot issues.

    The customers I work with that have the smoothest development processes, and best ROI from the resources spent on development, have a way to hydrate a consistent database environment for their developers. More importantly, the developers can self-service this. When a single developer, or a team, decides to reset their environment, they can do so, applying in-flight code changes from version control to bring back the database to the state it was previously in.

    Application developers do this all the time, as they restart their application. Removing their abliity to change code, restart the app, test it, and observe results easily would be crippling. Yet, we rarely reset dev databases, and often, don’t have consistency between environments.

    Self-service is one of the key indicators of a mature software development organization. This is easily achievable for many app projects, but not simple for databases. Relatively few groups have figured out how to do this well, but those that do are much more productive and efficient than everyone else. Traits that are becoming more important in this highly competitive world with increasingly fickle customers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.