Voice of the DBA

Tag: software development

  • Effective Engineering

    I’m not the smartest developer or DBA. I find myself mystified at times by Itzik’s posts on T-SQL queries and I’m amazed at times by the complex systems that I see the DCAC people put together. I can usually figure things out (sometimes by asking the authors a question), but it’s not always easy to do. We have some truly gifted, incredibly intelligent people in this business.

    I am, however, effective. I have been very successful in my career at getting things done well enough, things that work well, meet the needs of my client/employer, and meeting deadlines. I don’t just slap things together, but think about them, build them, test them (don’t forget this), and then make sure they’re working when they’re deployed.

    Sometimes this might take a few PRs or patches for patches, but I get things done. And my customers/clients are happy.

    I saw this post on Linked In noting Platform Engineering is Dead, which is a great title, but not really true, and not quite reflected in the piece. The author worked on the Software Delivery Enablement team, which is what the platform engineering team is supposed to do.

    I see similar complaints about DevOps, and previously saw complaints about Cloud computing or Agile or Scrum or SOLID. There have been similar complaints about how some new methodology or idea isn’t working and should be abandoned in favor of this other new thing.

    Ultimately, near the end of the piece on Linked In, there is this:“we also knew how to help them use solutions to deliver software better, and we partnered with them instead of inflicting things upon them.”

    This is what Software Engineering should be. In waterfall, we want to have customers tell us what they want and build that. Often customers don’t know what they want, so we decided Agile would help. DevOps is a way of talking about a partnership between developers and operations that still delivers what the customer wants, quickly.

    Platform Engineering or Software Delivery Enablement or whatever name you give it is still partnering with customers to deliver what they need. Not what you think they need or what you want to build, but what they need.

    Whenever Agile or Scrum or DevOps or Platform Engineering doesn’t work, it’s because you’ve forgotten that this is a partnership. That’s what effective engineering is, and it’s what I’ve practiced. Partnering with others to achieve our aims.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • When Do We Worry About Scalability?

    “We need to get the code written for feature X. Can you finish this query today?”

    We’ve all heard some variation of that request. We have a request or demand, and we need to get it done. We need to get code out so our business can advance, sell more things, get more customers, etc. There’s always some reason to get new code pushed to production quickly.

    However, many technical people want to ensure their code works well. At least, I believe most do. While most people can write code that works and meets the requirement, some don’t know how to write code that performs well or don’t know how to test their code to check. Often there isn’t a large workload in dev or test environments to verify things.

    There may not be a large workload in production either, at least not at first.

    So, what do you worry about first: your code being used or performing well? That’s a similar question to this one: Worry about Scalability or Popularity First? While most of us don’t work for a startup and our organizations have some sort of financial stability, does popularity matter?

    I’d say that for any feature you build, whether a startup mobile app or a legacy ERP system, you’re still looking at this type of question. You want to know if it’s used, and how often. That might determine if you spend more time on this feature or area. Maybe you have some idea of popularity, or just plan old use of the feature. In that case, certainly make sure it will scale to not only meet your data size now, but plan for some level of growth across the next 6-12 months.

    If it’s a new area of functionality for your application, then maybe you have no idea. In that case, the DevOps approach is get something working, a minimally viable version of your code or query, and then tune it later if it becomes a problem. Many technical people approach the endless number of tickets and requests they get like this.

    The problem is management often doesn’t budget in time to clean up the technical debt (to care about scalability).

    My view for database code is that we should always be leveling up our database code knowledge. If we deploy bad code in production, and can’t fix it, then at least we can avoid adding to the problem by writing the same poorly performing code again. Learn a better way to write that type of query. Whether you’re splitting strings, finding islands and gaps, calculating running totals, or anything else. Learn what works well and write that code next time.

    That helps your team balance the scalability and popularity-chase by producing good code the first time. Or at least, the next time.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • A Simple Solution

    In general, I like simple solutions to problems (and simple songs). They can take work, but a simple solution means it’s easy to explain to others, easy for others to refactor/modify, and we limit the amount of effort required to ensure others can use the solution. Sometimes I see very clever engineering solutions, but they become cumbersome to implement and maintain, or worse, no one other than the author understands how they work.

    In any size organization, that’s a piece of technical debt that can cause problems over time.

    When I see a simple solution, I feel a bit of joy. I got that recently with a blog post from Microsoft on how to transfer your CLR code to Azure SQL Managed Instance. While moving CLR code can be a pain, as you are moving a binary object, this post suggested using MI Link. This is a feature designed to replicate between a local SQL Server and MI, which isn’t something I think a lot of people want, but some do. I don’t know if I’d want this permanently set up, but as a way to move code around, this is very simple and easy way to move this code in a one-time shot. Perhaps as a way to set up a test from a local instance to the cloud.

    This isn’t my ideal way to move code around, as I prefer to have a specific pipeline dedicated to deploying code. However, I also think that this is a nice simple solution that might be easier in some situations. I like ease, and I like effectiveness, as long as I’m not sacrificing something more important, like performance.

    I see lots of creative solutions built by customers, often built by some smart people. However, they become quickly complex, and they are hard to explain to others in the organization. It can be difficult to trust anyone other than the author(s) to maintain the solution across time. It can be even more challenging when new staff onboard and never get an explanation of how or why something works.

    I aim for simple solutions, and those that I can easily explain, and others can easily pick up and understand. I’m curious today if you’ve seen complex solutions that were overkill, or ingeniously simple ones you admired.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • A Lack of Architecture and Planning

    A few weeks ago, I was sitting in the audience, waiting for my turn to speak at DevOps Days in Minneapolis. Just before me, Xe Iaso delivered a funny and thought-provoking talk on building a social network on a whiteboard. It was very well done and had me feeling nervous about following that session.

    The talk is a bit of a satirical look at an interview Xe had for a company that tried to get them to derive an architecture for a large distributed system. It was interesting to hear Xe note that often we have architecture diagrams of what we’d like to have, but never an explanation of how we implement a large system, especially one that has to grow as our workload grows.

    This talk was a nice analogy of how often we get into situations where many of us can’t believe our system was structured this way. We often wish we could completely redesign things from scratch, and we’d do it better. Why didn’t the previous engineers think things through?

    Watch the talk. It shows how a lot of software is built. We build what we think we need, but when we get overwhelmed, or often when we get unexpected pressures from others, we make decisions that seem to be the best ones in the moment. However, a few years later, with a few of these decisions behind us, we realize that each of those choices was too short-sighted. We have a mess of software the seems cobbled together rather than well-engineered.

    I don’t believe we can engineer everything well from the beginning. I also don’t believe in early optimization, mostly because I think we are asked to build a lot of things that are never (or lightly) used. Who uses the paintbrush in Word or the FACTDOUBLE() function in Excel? However, I do believe that we ought to write code that performs well the first time, following patterns we (should) know well. We certainly should assume that whatever code we write will see at least 10X more data in production than in dev/test, so prepare for processing more data.

    I’m sure many of you know of a codebase and system that doesn’t run well. I hope most of you have another application that does run well and is fun to work on. Hopefully, the goal of your organization is to turn out more software like the latter than the former.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.