Voice of the DBA

Tag: software development

  • Common Algorithm Concerns

    When we build software, many of us use the same algorithms to solve problems. We might choose a similar method for a quicksort or a lambda validation or a regular expression. For database work, your code for a running total (or other common challenge) is likely very similar to many other people. At least on the same platform. You might solve this differently in SQL Server and Oracle, but for the same type of database, many of us write very similar code.

    Actually, many developers might copy and paste an answer from SQL Server Central, Stack Overflow, or another site. I’m not sure if I think this is good or bad, as it’s a good idea to reuse code if it solves the same problem. If you copy it and don’t test it, that’s bad. After all, the code might not solve your slightly different problem if you don’t check it.

    In the modern world, if we build software for our business using an AI assistant, could our company be liable if we knew our competitors were using the same AI service? Is this any different than a human developer performing a copy/paste from SQL Server Central? I don’t think it is in many cases, though the same concerns about intellectual property might be present in either case.

    The concerns over AI seem murky in some sense, especially as the AI might “generate” code that isn’t directly available on some public resource. I do think that this is more of a collusion using a service than an algorithm. Still, in the hyper-connected world, where many of our applications might look to take advantage of some service instead of implementing it ourselves, this could be an issue.

    I ran across a piece that discusses a lawsuit about a common pricing algorithm being used by different hotels. In this case, it’s not that the developers at different hotels used the same code, but rather that the hotels used the same service from a company, which of course, used the same code for all their customers. Whether you think this is a valid lawsuit or not, this is the type of legal action that others might bring if two competitors ended up using the same AI service and developed very similar code that might behave the same way.

    I don’t think that AIs (at present) can actually develop new algorithms or solve problems in a new way. Instead, they predict the likely solution based on how they’ve been trained by similar scenarios. In that case, how concerned are we about how getting common solutions in disparate pieces of software? For most of us, I think we’ve be pleased that we have well-tested (hopefully) code that runs efficiently (again, hopefully) being re-used in many places. That would be better for most systems in the world.

    What isn’t better is if humans become more adept at specifying prompts and producing software without lots of specialized expertise. For many developers that might be average, or even slightly below average, this might cause them concern for the security of their position. With good reason, as labor is one of the most expensive parts of building software.

    Ultimately, just as with any other position, the best way to build a safe, secure career, is to continue to build your skills and produce value for your employer. That way it’s unlikely any AI will ever outperform you.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • The Code Freeze

    One of the ideas behind DevOps is that we minimize the time between code commit and deployment to production. We want to avoid work-in-progress and bottlenecks to getting our software into the hands of customers. This has led a lot of companies to release more often, albeit with smaller sets of features. The total number of things delivered under DevOps might not be greater, but it often is more targeted to those things our customers want/need/use.

    However, the idea of releasing often means that we try not to stack up too much work before deploying it. What does that mean for holidays and the code freezes or no-deploy periods that many companies have? How do you implement a code freeze under DevOps?

    I read an interesting series on code freezes (partially paid content: part 1, part 2, part 3), and it looks at some of the implementations, data from surveys, and pros/cons of implementing a code freeze.

    Two things here. First, I like code freezes as it gives staff a break. Two, doing this in a way that doesn’t just shift work (and stress) is hard. Do you have code freezes or deployment restrictions at your company? Do you like them?

    The post has lots of metrics from various companies, some known, some anonymous. It seems that for those companies using code freezes, they help employees take a break, but not everyone takes a break, which means some work is done and merges after the code freeze are problematic. For those who don’t mandate a code freeze, less work is done when people are gone, but some work continues and is released.

    I think the natural flow of work is that when fewer people are available, less work gets done. Poor management (or a hero complex) might overload the staff still working, but that doesn’t work long term. It might work now, given there is an oversupply of tech people and fewer jobs, but when things change, people will leave those positions, and lower quality, or at least less knowledgeable people, will cause problems for those firms.

    I’m not a fan of code freezes, but I am a fan of understanding that staff needs a break. Lots of people want to take time off during holidays, especially those with children who want to celebrate or travel. Others might be fine working and want to catch up on work, refactor things, or maybe address tech debt. I’ve been in both situations, and I hope organizations can adapt to both situations, though without sacrificing quality. We still need good code review and testing, and if there isn’t the staff to do those things, then delay merging code.

    The one thing that stood out to me was the learning that changing the pace is good for humans. If you’re in the habit of releasing every week (or day or whatever), moving to a different cadence for some time is good for you. Just as high-pressure work can’t be sustained for a long time, very slow periods aren’t good. However, both have their places as a change of pace. A high-pressure situation might be necessary to meet a business goal (or solve a problem). A low-pressure time might be a good time to think, experiment, and perhaps innovate.

    Or just fix and refactor some poorly designed technical debt.

    Do you want a time when you aren’t trying to get code to production, either as a dev or operations person? Or do you like to have your job predictable and work in the same pattern year-round? Let me know today.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • Re-platforming

    Re-platforming is the process of moving a system to a new platform. Imagine taking an ASP.NET/SQL Server website and moving to Azure Functions on Azure SQL Database. Or maybe taking a Java client/server app with Oracle and moving it to a series of microservices against MongoDB. Those changes could be a net benefit to your organization in the end, but they aren’t quick or easy. They’re often fraught with various challenges that can cause a lot of stress while creeping over budget.

    There’s a post that talks about some of the things you might think about if you embark upon a re-platform. Often this takes place when an organization is looking to modernize their tech stack. Quite a few of the technology DevOps success stories take place when the older structures are not maintainable, but also not able to handle increased workloads or performance requirements.

    It’s often said that a complete rewrite rarely makes sense. I think that’s likely true, but that doesn’t mean it isn’t something you should do. It says that you should rarely do it, and you need to pick a good time when your old application, database, network, or some combination of these can no longer meet your anticipated future growth demands. This is likely a debate that your organization should have every year or two and ensure that continued investment in the current platform makes more sense than a new one.

    Once you decide to move forward, it will be a journey, which means understanding how new and old systems will interact becomes important. Compatibility, at least for some time, is very important. I’d say this is especially critical for how your data systems will work together and what options you have to replicate/ETL/etc. the data between systems.

    Even if you plan to make a cutover from one system to another, data migration is fraught with issues. Anyone trying to ensure data moves cleanly from one place to the next should have strong checks, usually with automated tests, that evaluate if your data is moving correctly. Row counts aren’t enough, or not the only metric. Random checks of individual rows, as well as checks for outliers, should be a regular part of any project that will migrate data. Learn where you struggle to correctly transform data because there will be places where this happens.

    I don’t believe that wholesale re-platforming often makes sense. Leaving SQL Server to go to another database platform, or even when you might try to change the paradigm with NoSQL or a Data Lakehouse, often costs a lot. Getting a return after staff training, migration costs, and even delays from inefficiencies during the process often equals many years of licensing. I think slow migrations are a better idea, perhaps adding something like ElasticSearch or Redis, experimenting with a small data lakehouse for a particular purpose, and replicating some data to Neo4J for complex hierarchy queries. These evolving mechanisms give you the chance to experiment and learn about the process. It will help you understand if there is a positive ROI if you continue.

    In many ways the world of technology changes slowly. We have lots of new shiny toys, but many of them don’t magically solve your problems. There are always tradeoffs and without strong domain knowledge, you may have more unknown unknowns that reduce your chances of success. Re-platforming can work, but make the decision carefully and proceed methodically, not with haste.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • The Art of the Code Review

    The inspiration for this was a piece about the art of the good code review. Throughout my career, I’ve seen code reviews grow and change. From formal meetings to automated notifications and asynchronous discussions to complete lip service to the process. I’d like to think that most organizations are beyond the latter and there is some sort of review beyond the developer, but I still see a lack of other eyes looking at code before it’s deployed, especially database code.

    The article above opens with the idea of why we review code. The main reason is to create ownership, or more specifically, shared ownership. I had never thought of it in these terms, even though I think the ideas of standards and patterns are certainly shared items. Having everyone take ownership not only keeps quality high but could help you share knowledge and also ensure everyone feels a responsibility to safeguard all the code. This also helps everyone keep an eye on the larger picture of the entire codebase.

    I know lots of modern application developers are very familiar with pull requests, though I think these are still somewhat rare in the database world. This is a notification that someone would like their code to be put together with everyone else’s code. A good code review does start with a good pull request, as the idea is to have enough information to let the reviewer decide if they should approve things. PRs should also be focused, so if you are making a major change in one object and refactoring another, make those two different PRs. That way I can reject one without the other.

    Overall I like the suggestions in the post, but I worry about one of them. The author notes that if there are minor changes requested in a comment, you should pre-approve the PR and trust the author to address the issues. In today’s very busy world, and with the challenge of changing code once it’s deployed to a database, I don’t know if I’d follow this for SQL code. Maybe for C#, but if I have to live with your code for a decade, I don’t want mistakes deployed that could be prevented.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.