Voice of the DBA

Tag: software development

  • Reducing the Cycle Time

    There are lots of software development methodologies. This page lists a few, among them waterfall, agile, iterative, rapid, and more. What’s been interesting to me is that the process of deciding what to code and then whether it works doesn’t change much between different ways of building software.

    Instead, the cycle time between when we ask a client what to do and when we deliver it changes. The more agile/lean we are, the lower the cycle time. The more waterfall-ish, the larger the cycle time. I guess that analysis and breakdown of problems into work also changes, as the scope in modern DevOps styles of development is smaller (more contained) than in waterfall.

    However, we seem to follow the same steps. In the database world, we might do similar things if we think about how we build data models and code systems. We could get all the requirements and build the entire model, or we could get some requirements in an area, build that, and then ask for more. The former is more of a waterfall approach and the latter more agile/DevOps-y.

    Is one better? Not really. I would say they are both situational. In some domains, waterfall might work better. When deciding to build a system to launch rockets, most of the problem domain is known and not changing often, so waterfall type approaches likely work well. Certainly we still went some level of decoupling to take advantage of changes that do occur, primarily in the hardware, but the overall problem remains the same.

    However, in many of the business or software tooling places I’ve worked, no one has a good grasp of the entire domain. Heck, I think in most businesses, people roughly know how business runs, but they forget the myriad of exceptions that ensure our environments look chaotic to software, and they often constantly refine (or re-direct) the way the business works in pursuit of their latest goals. At times I’m amazed business runs smoothly, though I think this just shows how much we tolerate variance in business processes that we think are more set and defined than they are in reality.

    I am a big believe in loose coupling and accepting uncertainty. I hope for the best, but plan for the worst, or at least, plan for things to change. I like NULLS in databases, not everywhere, but in places because have unknown values. I like agile/DevOps approaches to software because we rarely know all the information about a problem. Heck, sometimes clients don’t know the entire problem or don’t spend time thinking about the entire problem when they create requirements or requests for changes. Therefore, I like short cycle times, with the flexibility to change directions as necessary.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Simple Talks Episode 9–Data masking and subsetting

    The episode on data masking and subetting is out. You can see it here:

    Watch and check this out. This is especially close to my heart as I’ve been pushing for subsetting from Redgate for a long time. I think subsetting is incredibly important for development agility.

    Some interesting thoughts on the problem space, which isn’t a simple thing to solve. It’s not that complex (hard to understand), but it is complicated (lots of moving parts). As I’ve talked with others and worked on the problem, it’s not something that you can just knock out quickly.

    What’s funny to me is that Andy Warren and I asked Redgate for this in the early 2000s and they didn’t build it.

    We have a subsetter and masker at Redgate in our Protect/TDM area. Check those out if you need a solution for your org.

  • Tech Debt Perils

    My wife and I have been thinking about some new audio equipment. We’ve been a little unhappy with our Bose soundbar because of the software flakiness and sporadic network connectivity issues. In looking around, I saw a Sonos product, but after reading a bit about the company’s recent history, I decided to look elsewhere.

    Sidebar: if any of you have recommendations that aren’t high-end $$$$ audio, let me know.

    I saw this article about some of the problems Sonos has had, and it resonated with me. I’ve been in a place where I’ve worked on software that had a lot of technical debt and needed to be changed/improved to help grow the company. Management was pressing everyone to get the software ready quickly, without more concerns about the customer experience or the quality.

    That seems to be what happened with Sonos, where they released new products and new software, with the software missing functionality that their customers needed. There were bug reports, bad press, lower sales, and canceled raises/bonuses. Those last ones, to me, ought to be completely focused on executives and managers, but I’m sure that’s not the case. Management rarely takes the blame, but ultimately they are responsible for hiring, training, steering the employees, and deciding to launch.

    The story reads like a summary of The Phoenix Project. Technical debt, poor project management, and pressure to increase sales combine to create a disastrous launch. It’s always hard to know where to assign blame, but clearly, good software engineering wasn’t a priority, nor was reducing technical debt. I know it can be hard to balance the need to alter software with the need to keep it maintainable, but in this case, they made poor decisions.

    The article says there were yelling and screaming in meetings, and concerns from developers about pushing back on senior leadership on the timelines and demands. Some of us might have been in those situations, and my experience has been if there is management that doesn’t value software engineering, any particular developer is vulnerable to a layoff or termination if they complain. If management doesn’t value software, they don’t value developers and think they can be easily replaced. I’ve seen quite a few companies start to fall quickly with this attitude when developers are not valued. I also see constant job openings and employees constantly looking for other opportunities at these organizations. People only stay long enough to find another job.

    Software is complex. It’s going to have some bugs. There are always tough decisions about which things to work on now and which to delay. I hear those conversations, and I find myself trying to balance the needs of sales and engineering. We need both, and we also need to ensure we can change directions in the future. Paying down technical debt should be a regular occurrence to ensure the software is maintainable and adaptable. I know that when we look to quickly take advantage of new opportunities, this can mean we’re adding new technical debt. Walking that line is the key to success.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Everything is Code

    I posted a note on Twitter/X with this quote: “The content updates had not previously been treated as code because they were strictly configuration information.” This is from testimony given by Crowdstrike to a US Congressional committee in trying to explain how they grounded much of the airline industry a few months ago. That was a mess of a situation, and apparently, the vendor didn’t think their configuration was part of their code.

    That’s an amazing viewpoint to me. The fact that any developer or manager thinks that their configuration data isn’t a part of their code is worth testing. Yet, I see this attitude all the time, where developers, QA, managers, and more think that the code is the only thing that changes or doesn’t change, ignoring the fact that there are configuration items that affect the code and need to be managed appropriately. Certainly, if the config data were in enums rather than in a file or database they’d feel differently.

    I think part of the reason that people try to ignore config data is that it is hard to manage. Often config data might change between dev, test, and prod. Dealing with that, and testing appropriately is hard. I haven’t ever seen a good solution for getting data into an environment the first time. That’s the hard part. Once the data is there, you can use it as a token where it is needed, and hopefully, the value has already been tested. At the very least, you can test how that data affects that environment.

    I am glad to see Crowdstrike publicly recognize that they need to dogfood not only their code changes but also their config changes. However, for a company that hasn’t shown a rigorous engineering approach, I suspect they’ll test very simple and basic config changes and not necessarily do a good job of carefully testing a variety of potential problem vectors. That takes work, and excluding config data from testing is a sign (to me) of a technology group trying to avoid doing too much work. It’s likely more management and leadership than technology workers, but the entire organization is showing signs of shortcutting good engineering.

    My view is that developers should be free to experiment and try lots of things, and have a lot of freedom on how they build software. I think the same thing for infrastructure people as well. However, as we start to move our changes towards production, everything should be in code, version-controlled, and promoted through PRs. In other words, get everything stored as code and nothing gets changed outside of development. It only gets approved to move forward or rejected, after it’s well tested.

    That’s a tough process to implement, and one many companies don’t spend the time doing, but for those that do, they end up deploying many  fewer bugs.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.