Tag: software development

When we design software or databases, we have to make decisions right now. If you work in as many environments as I have, this means I’m often picking a tool, platform, version, technology, or something else without a complete set of information. In some cases, I’m actually writing code (SQL or C#/Java/etc.) without completely knowing the business problem I’m solving. I do know what someone described, but as many of us find out, those descriptions often prove to be lacking.

Over time, we refine the code with feedback from users, which is why DevOps has become so popular. We write the best code we can and then quickly alter code if we haven’t hit the mark. We constantly adjust by making small changes and improvements.

However, changing the code is one thing. Changing more fundamental technology choices is harder. There is an interesting post on the disproportionate impact of your early tech decisions from one of the developers at Stripe. One of the examples given is for the database platform, which was Mongo. The piece notes that once the database platform is chosen, most groups never switch platforms. I think that’s been true in many places I’ve worked. The same thing for languages and cloud providers as well, as once a choice is made, rarely does a team, much less an organization, switch.

Why is this? Is the cost of switching that high? I think it is. I’ve seen a few organizations want to migrate from a licensed relational platform to one that is free and OSS. However, the time it takes to rewrite code, the time to build new expertise, learn the tips and tricks for a new database (or other platform technology) is significant. A team can make no shortage of mistakes during this process. Is that worth the cost of licensing?  Perhaps. Perhaps not. It’s a hard thing to decide. With the high cost of labor, I think it’s hard to make a good argument to rewrite.

In many cases, I’d argue that core changes to your technology stack ought to be considered, but understanding that no tech will solve all your problems and you will make lots of mistakes in the process of changing. Does this mean you need to make a great decision when you start working on a project? In my mind, no. Make the decision to use what your team is most familiar with and then live with it.

And be open to using targeted technology for specific needs. Lots of transient data in a busy workload? Add Redis to your database stack. If you need full-text search, consider ElasticSearch. Maybe add a graph database (not SQL Server) if you need queries in this problem domain. Use what works, and learn to use it well.

That last sentence might be the most important. Any technology can likely support most workloads, but you need to write good code and learn to use the platform well.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Developers, in general, are very optimistic about the code they write. This is likely one cause of their estimates of the time required being low, as well as the various bugs that slip through because of corner cases that appear for the problem being solved. Often developers think they’ve considered the various ways this code ought to work and covered all the possibilities. Usually we find they’ve not thought about the problem from other perspectives and need to adjust their code.

They often also feel that their code is superior to others, and that they can examine a problem in a new way. One of the reasons that I think many developers want to rewrite systems in some new technology or new way, embracing the Not-Invented-Here way of looking at other people’s code. They want to write their own solution.

Of course, many developers these days don’t eschew all code from others. Usually, it’s just other people in their organization, as a developer will embrace some random open source project they’ve discovered, sure that using this library/language/whatever will make their code better. While I do think there are examples of some new tech that is better than what’s around, I don’t know that any of them are a panacea. While F# is praised in many circles as better than C#, it’s not widely used, at least according to surveys of the top programming languages in use (IEEE, SO). It must not be better enough to get more projects to use it.

What is widely used is SQL. In fact, if we look t0 other database languages, nothing else is close. While I’m sure plenty of those C# and Java programmers use LINQ or some ORM to produce the SQL code, I don’t know many competent or highly regarded developers that can’t write some SQL code or don’t regularly write SQL.

There are some interesting ruminations on the optimism of developers in this piece. It made me smile, and I like the practical ending. While you can debate and discuss things, ultimately we need to ship code. And we are almost always better off using a technology we know rather than searching for a perfect new one to solve our problems.

When working with databases, relational or even most of the popular NoSQL ones, this means knowing SQL. And when trying to solve our database problems, in many cases, this means learning to better write SQL and build relational entities, not abandoning our platform for some new shiny one that we think will make everything run smoother.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Every once in a while I find a fun post that makes fun of programming. I think because the author is struggling with some piece of code, they write about the craziness of software development, often with an analogy that compares building something in the real world to how software programs get written. Here’s one, with a bridge construction analogy, that degenerates into a rant on the disparate ways in which different programmers work together and how the entire world of the Internet is held together in a constant patch-fix loop.

I do think that most large software applications have a decent amount of inconsistency. I’d suspect most large databases (in terms of object count), also have a variety of techniques, tools, and standards in use. That’s because the tooling that we use to build things change. We don’t have the constraints of the real world, which is why we could have a stored procedure using a subquery to calculate a running total and another one that uses the OVER() clause.

We can be more consistent in SQL because the language doesn’t change as quickly, but in application code, frameworks, techniques, and even libraries come and go constantly. Different developers have favorites, and without strong controls that limit the experimentation of your staff, likely your code base has a variety of styles and algorithms that do the same thing. Static code analysis, linters, and unit tests can help ensure your code runs effectively, but it still likely has sections that no one wants to touch.

This flexibility can be maddening to developers, especially when they start a new position. At the same time, each of them probably thinks there are a few places where they think they can write the code better, and they will attempt to do so, even if they are creating more inconsistency in the project. Multiply this by the 40% of programmers that are willing to make changes without trying to mimic how others write code and pretty soon the architect (if you have one) has no idea how anything works.

In the real world, we can’t easily get things to work together if they are disparate. We don’t suddenly change from framing one wall with a 2×4 to another with a 2×6. We might use different size nails or thinner brake pads or any of a number of other changes from one place to another, but often those differences don’t matter. At least not enough that we find houses and bridges constantly falling down. Of course, the workers building those structures follow a set of plans and standards, using similar tools and techniques.

That doesn’t happen in software, where you are often not limited by external constraints, but often your imagination. We can construct things in software that no one ever thought of before, we can grab materials, i.e. code from Stack Overflow, and use them in our system. Often builders don’t get to source materials from wherever they want, which inherently limits the variation between parts of the structure. That being said, lots of little mistakes get hidden in construction projects, just like they do in code.

I don’t think programming sucks. It’s always an interesting challenge, and I do think that when you have a group of developers that care about their craft and are trying to work together to solve a problem, we get some amazing software built and deployed. It might not be consistent or pretty, but often it works very well. If it takes regular maintenance, that’s fine. It’s an ongoing challenge that keeps us employed.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.