Voice of the DBA

Tag: software development

  • Coding for the Future

    I posted a note on Twitter about some code. In this case, it was the start of a for loop, but with two things in it that I didn’t like. The code is shown here:

    for ($i = 2011; $i <= 2021; $i++) {

    Both the starting point and the ending condition were problematic for me. This code is used in a scheduling application, to populate a dropdown that I used regularly. It’s always worked, since about 2017 or 2018, and I had no reason to even think about the algorithm. I assumed the code was something like this:

    for ($i = 2011; $i <= year(); $i++) {

    In fact, when I had issues on 27 Dec, I assumed that things were coded to the current year and I’d be able to schedule things on Sunday night when I got home. Instead, things still didn’t work the first few days of 2022 and I had to get someone to come in from holiday to fix the code. Now it looks back a few years (likely unnecessary) and looks forward one year, as it should.

    I don’t know why a developer would bother coding anything like this. Get the need to perhaps test something expediently and limit choices, but hard-coding most anything is a poor idea. There ought to be some logic in how or why you choose values, and that logic often can be expressed in code that adapts to situations. I certainly could have lived with only the current year being available, though that still doesn’t think through the problem space well.

    I’ve seen lots of simple bugs in code over the years that are similar to this one. These often occur because the developers haven’t thought through how code might need to adapt to slightly changing situations. Something like a change in the year, or maybe the addition of a time slot for scheduling or a new location added, even a new tax rate. Often I find developers think they have considered the problem given to them, but a little too tightly. They haven’t thought about how their code might change if the user requesting the feature has made a mistake.

    Writing code that solves a problem and allows for adaptation to likely scenarios is a bit of an art. It can be hard to do, but great developers assume users will need some changes and plan for this in their solution. They also assume users make mistakes in their specifications and they don’t hard code anything.

    Steve Jones

     

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • No-so-smart Contracts

    Perhaps the best quote I’ve seen in a long time: “These kinds of attacks are common in smart contracts because many developers do not put in the legwork to define security properties for their code…” I’m sure that this would apply to many kinds of software, not just smart contracts.

    This is from an article on a hacker that stole money by altering a smart contract. In this case, tokens used to replace parts of the contract overwrote other tokens, which allowed a smart hacker to change prices and make more money. Or steal it, with a contract change, I don’t know that theft is actually the correct term.

    The wider issue here is poor developer practices, and really, not listening to the results of security audits and making changes in code. Maybe they listened to the audits and hadn’t completed the work. There were some critical issues, and some remediation, but not enough in this case.

    Building security into software is hard. The threat landscape changes and hackers are incredibly creative. It is hard for developers to keep up, but it is important, especially where there are finances involved. There are tools to perform security assessments and automated pen-testing. Everyone ought to use these, and more importantly, management should take security more seriously. If they don’t, they deserve some sort of penalty.

    The problem for many of us is that we can raise issues, but we are powerless to do anything. We can change jobs, but that’s not practical all the time. We can continue to raise awareness, but that can be detrimental to our careers. After all, management will get tired of us repeating ourselves at some point.

    Mostly what we get to do is worry. We worry that the company will get penalized, which can affect our employment. We can worry that management will blame us for an issue they didn’t allow us to fix or give us the tools to detect. We can worry management will blame us for not knowing about an issue as well.

    I believe we ought to have more focus on security, but I’m not sure what that means or how to achieve this in a practical sense. I don’t even know how we’re set up regulations and penalties for such a complex situation.

    Mostly I’m just sad for the state of software security.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The Complexity of Modern Systems

    I am a big advocate of version control and testing for your software. The main reason is that modern systems are incredibly complex, and capturing what has been done (and what is happening), along with looking for potential issues, is important. It might be critical for ensuring you have success in building software.

    Part of the reason is this post on software complexity. Now I don’t think the complexity is killing software developers. I do think it adds to developers’ stress, and in poorly managed environments, the complexity makes for a poor job experience for many.  While I do think most people are smart, and they find ways to overcome this, they just don’t enjoy the process.

    This is part of the reason, IMHO, that we are starting to see lots of people move to a new role, often in a new company. With remote work, there are more options, and for people with talent, they can easily leave. My recent job searches seem to show more remote job options and more demand for mid and senior level people. I think some of this is looking to get better talent in your company, and some are because experienced people are tired of poor work environments and poor code.

    Modern software is big, and I don’t think any system of consequence is built by individuals anymore. There are still good projects and applications being built by a person, but once they start to grow, a team of developers is needed. This means that we have to learn how to better work together. We need to adhere to standards. We need to communicate and collaborate both in code as developers and together as humans.

    While someone may understand how the entire system works, it’s unlikely that many of our staff does. If the system is large, it’s unlikely anyone completely understands, and then our ability to function as a team is incredibly important to be sure that we are building something that works well together.

    To me, the complexity of modern systems emphasizes the need for teamwork and collaboration, the thing that many companies adopting DevOps struggle to emphasize.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The API Bottleneck

    While talking with a client recently about their performance challenges, I was relieved to find that the database wasn’t the problem. Instead, their API server was overloaded by the number of calls taking place in their application. While the database did provide the backing for the API calls, there was a fair amount of caching. However, as they’d moved to microservices, more and more of the interaction between modules was taking place as a network call to a single server, which became overloaded.

    Certainly, they could add more API servers, or break out different types of API servers, but each of these changes potentially has downsides. Data can become out of sync across two systems, which is why we often want a single database storing all data. Distributed systems are hard to architect and implement, and I suspect as more developers try to build them, we’ll find more issues like this. Not that developers are doing anything wrong, but they often don’t realize the challenges of building a distributed system when the workload scales.

    They’re relearning the knowledge that others have gained through their own experiments and failures.

    One of the things we try to do here at SQL Server Central is share our knowledge with others. We hope to help each of you learn from others, and hopefully not need to make the same mistakes in your applications.

    As I listened to the client talk, I was reminded of this post from Troy Hunt. In it he talks about the HaveIBeenPwned site, which he built a scaled to meet demands. The site doesn’t cost a lot, but there is a great quote about the API server he set up. He wrote: “I’ve tested it with 380k queries a minute! Infrastructure will scale out and magic will happen, so long as you’re not maxing it out for perpetuity, just go for it.”

    However, in your company, you might be scaling in perpetuity in an app, especially as a workload grows and more parts of your application need to make queries against an API. Good design, good coding, and adhering to good distributed system design, will help ensure you can handle the load. This is especially true in the cloud, where scaling up is just provisioning another resource, but there is a cost.

    You have to know what to do, so I urge anyone looking to build a distributed system to read about the experiences of others. Learn from their knowledge, rather than make your own mistakes and try to correct them. There is a lot of great information out there, but you need to be looking for resources that provide technical details, like Troy’s posts on how he built his site or Jeff Moden’s posts on writing high performing queries. That’s the type of knowledge that is invaluable if you want to be an efficient and effective developer.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.