Tag: software development

  • Database APIs

    Creating an API and using an API server of some sort has become a very common way of working with modern software, especially distributed systems. Whether developers move to microservices or something else, they often make more use of APIs than ever before. At Redgate, I’ve watched us work on new products, often beginning with a basic API at the command line and evolving from there.

    However, does the database need an API? This article thinks so, though I don’t love the piece. It seems to wander a bit amongst a few topics and doesn’t always fit together. It’s also trying to promote the Stargate API gateway for Cassandra. I don’t know if this is a good idea, or if it really helps make developers more productive, but the idea of having an API for developers makes sense.

    In fact, when I work with clients I often try to get them to think of their tables as an API of sorts. With APIs, we want to rev them, but carefully. After all, we know there are dependencies from various other pieces of software, including other items inside the database. We want to be sure we don’t break anything. At least not for long, which is why DevOps and being able to release work quickly matters.

    Communicating changes and coordinating work is a challenge. I was surprised by a client who used SQL Doc for this reason, to ensure everyone had up-to-date knowledge of what their schema meant. It’s also a reason I’ve taken some of the work I do with clients and built my Architecting Zero Downtime talk. Many developers think narrowly about the requirement in front of them and not about all the other dependencies and impacts. I think a bit more unit testing in databases would help here. In fact, I demonstrate sometimes how to write a test that treats your table like an API to prevent breakage.

    I do agree that databases, especially relational ones can be complex. If we can simplify things for developers, especially developers new to our system, we can get more work done and hopefully, raise code quality. I don’t know that Stargate or GraphQL or anything else is better, though I have started to think LINQ is a better implementation than SQL for working with relational databases. That’s not going to change anytime soon, and plus, I need SQL to remain popular until I retire 😉

    Whether you formally build an API or not, it is helpful to treat your schema like an API and make changes in the same way that you would if there were other teams depending on your interface. That’s because there likely are others who depend on what you build.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Technical Debt

    We all experience technical debt when building software, especially software that has been around for years. Most of us have likely encountered code that we are loathe to touch and modify. At least, we don’t touch it twice. We might change things once, but when it doesn’t work as expected, most of us put it back in it’s previous state and move on to something else.

    At the same time, it’s hard to define what technical debt is or point it out to managers, who often can’t grasp the concept of why technical debt matters to a development team. They can’t understand the reasons why new code built on older code takes longer and longer to produce. It’s a nebulous concept that can baffle both people new to software development and those with years of experience.

    I ran across a few articles that talk about technical debt. This one notes it’s commonly used as a phrase and also, it’s inevitable. There’s a short video from Ward Cunningham, who is said to be the inventor of the term. It’s interesting because the explanation isn’t bad code, it’s more that we don’t always understand the problem when we write code, especially at the beginning. As we have a disagreement between what’s coded and what’s needed, we accumulate debt. Ward talks about refactoring regularly as we gain understanding.

    There is another piece that uses the Chernobyl disaster as a comparison with software development. This one looks at cost-cutting in both situations. The emphasis in software development is something Jeff Moden will appreciate: developers not writing robust code. There is also a section on designing to meet only 95% (or less) of perceived use cases, which I think is just reality. We can’t write software, in any practical sense, to cover 100% of use cases.

    The final section talks about incomplete testing. While software development has gotten better here, there is still plenty of work to be done, especially with database software. I find far too few database engineers wrap tests around their code and often have problems with new data or when code is refactored. A few more tests added early often prevent issues later, or at least, save time when debugging. This is one place I think AI might really help with labor savings by writing these tests for us.

    Technical debt is a reality of life. We have imperfect engineers, we have the pressures to get things done, the increasing complexity of software systems, and perhaps most of all, the pressure to keep moving forward with something new instead of refactoring something old. I don’t have any great solutions, but I also do see software developers working better than they did in the past. That gives me hope for a future where software is embedded in more places and used more every day.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Design Lessons for Software

    I play guitar as a hobby. Not great, but I enjoy it and find it relaxing. It’s a good break from my day periodically, killing a few minutes before a meeting. It’s also a nice way to unwind at night. I find it better than playing games or my phone or streaming more Netflix shows. I enjoy those as well, but there is something different about music.

    I tend to use acoustic guitars, meaning no electronics. However, I have had electric guitars in the past, and have even used pedals to alter sounds. When I saw this article about design lessons from guitar pedals, I was intrigued. It has 5 lessons from these devices, which are for other digital gear. However, I think they could apply to software as well.

    The first one is that these pedals are rugged. While I’m not stomping on pedals with my feed, I do think that we could ensure software is more robust and not susceptible to small mistakes by users, especially in the order of their touches/clicks that might cause problems.

    The second is about using more than our hands, which I hope doesn’t apply. You can add voice or gestures but don’t require those. I HATE those features. The fourth is about physical UIs, including physical buttons, which I think is important for cars, but not necessarily for all software. However, if you can give someone a button or knob instead of a touch, it can be helpful.

    The third is to have bold, visual cues. I had a designer once say we ought to build more Fisher-Price software, meaning something obvious and usable by a child. I know some of our processes are complex, but we ought to work to keep things as simple as we can. For databases, I think clear, consistent names help here, especially for indexes, FKs, and triggers.

    The last one is to make things beautiful. I have to admit I didn’t think about this much before coming to work for Redgate Software. Across the last 15 years, I’ve learned to appreciate the value of design, UX, and the people that make things look good. I can’t do that; I have no skill in this area, but I know that having someone come behind me to do this is worth the effort.

    Of course, on top of all this, your software has to work and perform well. If the software doesn’t work correctly or is very slow, none of your clients are happy. Learn to write better code, improve your skills, and listen carefully to those asking for features. If you do that, these design lessons will make sure all your efforts shine through.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • A Case for Document Storage

    One of the challenges for both database developers and administrators is doing more, often with less. Many companies continue to grow their database estate, both in width with more platforms, and in depth with more instances of the platforms they have.  Some companies will look to shrink their staff, especially when adopting a cloud platform, while others may add more databases, but not increase staffing to match the additional load.

    In either case, what many have found over the years is that the cost of labor is high. Both for developers that write code against databases, and administrators that manage those platforms. While licensing can seem to be a large number, compared to the cost of labor, it isn’t usually a significant number.

    Often it seems administrators would prefer more of the same database platform. Developers often seem to ask for new types of database platforms, often some type of NoSQL data store. I ran across an article that makes a case for adding in document storage data stores to your environment, instead of just choosing am RDBMS. Labor is one of the big reasons for doing this. The other one is that for a given workload, the hardware cost is lower.

    The article opens talking about the object/relational mapping problems. There is some truth to the time and effort to map an object in an application to a table (or set of tables) in an RDBMS. There is some knowledge required to do this, but I also think it’s an important skill for many developers. The same type of object mapping to a serialized JSON document is shown as being easier, and it is.

    However, if you add or change your object, the application code to handle the document from the data store gets complex. Over time, you will have lots of “new” fields that don’t exist in older documents. How do you handle those? It’s not hard, but labor is required to write this code. And this code has to be maintained over time.

    The other argument is that less hardware is needed, made by noting all the data you may need can be co-located with your object. This is what we would call denormalization in an RDBMS and leads to data duplication? Whether that is a problem or not depends on the amount of duplication. Certainly the structure of an application that often works to send or retrieve singleton rows is easier in a document database.

    However, non trivial queries, which the author postulates are hard to write for developers, are likely hard to run for a document database. The load of querying across lots of rows, or updating them, is much higher in a document database. Depending on how often you update data, this can be an issue, and require more hardware.

    Which is better? The classic “it depends” applies here. Database modeling is important in both cases. As I’ve worked with people that move to NoSQL databases, I find they struggle to model in that world as much as many of us struggle to model in the RDBMS world. I also find that a NoSQL database often is going to require some sort of data warehouse or other structure that is built for reporting across documents.

    I’m not against the various types of NoSQL databases, but I also don’t think they are a panacea of any sort that magically makes building and operating an application easier.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.