Voice of the DBA

Tag: software development

  • Customize Software or Process

    Many of us work with software and often wish that we something was designed differently. I’m sure many of us feel the same way about our database schemas, which is usually even harder to change. In any case, we often want to mold software to fit our thought process. We’re often asked to alter software to meet the expectations of our customers as well, trying to alter visuals or workflows to match some method of working that our customer desires.

    We often try to do this with COTS (Commercial off the shelf) software, notably ERP systems. SAP and Oracle applications, and similar software packages, have made many consultants rich and required millions (maybe billions) of dollars of investment by companies that look to ensure their logos, their workflow, the specific process is implemented in software. Even many of us that purchase some infrastructure software, or even get open source applications, may spend lots of resources on getting the software to fit our organization.

    The thought that most people have had is that it’s cheaper to make the software fit the system, or the people, than vice versa. However, is that the best way to proceed? Do we want to customize software to work in a way different from the way in which is was designed? Or should we learn to alter some of our processes to better fit with how the tools work? Do we think that we really have some special secret in our process that makes us more efficient? Or are we resistant to change?

    For some software, like SAP, the system is designed to morph and fit an organization. Certainly when there are hundreds, or thousands of users, it might be worth the cost of customizing the software to fit the users. However when we have more specific software, such as that used to monitor infrastructure or deploy software or track features, with dozens of users, do we want to spend a lot of time changing the way things work? I sometimes wonder if we should instead focus on getting our people to learn a new way of working that flows with the software. After all, upgrades, patches, and other minor changes are less disruptive.

    I don’t think there’s just one answer here, and certainly there are valid reasons to spend time and money on changing a system. I just think it’s worth some thought to be sure that changing software is a better decision that adapting our process to work with the application.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.9MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • It’s the Engineers

    There are all sorts of articles and blogs on the Internet that try and teach you how to use a particular technique or technology to solve a problem. Plenty of this information is well written, and showcases the knowledge of that individual (or team) and how they’ve built a fantastic system. Every vendor building software platforms has reference case studies that show how well their system has worked for some customer. These are all good models that might give you confidence why SQL Server or MongoDB or Entity Framework or GoLang are good choices for your company.

    In reality, it’s not as simple as just choosing a platform or framework or language to make your application perform better. You really need the staff that understands how to use your choice of X and has some skill in building a system in that manner. It’s why I think that the cost of the software is a pittance compared to the cost of your people. If you have to train them, or they need to learn how technology X works, then you’re going to pay more in salary than you’d save in software licenses.

    There’s an article that I think illustrates this well, and might be worth passing along to your developers. It’s called Why Amazon DynamoDB isn’t for everyone and it’s worth a few minutes of your time to read. The gist of the article is that a NoSQL system, like DynamoDB, isn’t as simple and easy as you expect. It also says that for many applications, a relational database is a better choice because it’s often better understood and will solve most problems at small scale. For most of us, we don’t really get past what I’d consider small to medium scale, and so we should stick with relational systems.

    This isn’t to say that DynamoDB will be a problem for you, but there will be a learning curve for your developers if they haven’t used it in production and at your scale. The same thing occurs if a development team decided to switch from MongoDB to SQL Server because they like automatic tuning. They won’t necessarily configure and use SQL Server efficiently, and might not have their application perform well. Switching your development paradigm or technology is hard, and it will take time for your staff to get up to speed.

    Ultimately it really comes down to having good engineers that can write good code and understands the ins and outs of administering that particular platform. The database is extremely important here as it is the central location of your information, but the same arguments apply to frameworks and languages. As much as we want to learn and try new technologies and incorporate them into our work, we need to realize that it takes time to learn to use them well. Make small experiments, try POCs and slowly build up skills before you decide that your mission critical system needs some new technology that you read about and are excited to try working with.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.1MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • A Christmas Bug

    It’s not really a big deal for me, despite the fact that I travel to Europe in a few days for two weeks. At least, I’m fairly confident this won’t be an issue for me. A few of my fellow #sqlfamily might not be so lucky, as they use a different airline.

    Apparently American Airlines has a bug in their scheduling system. It’s not a safety bug, don’t worry about that, but it’s potentially a huge bug for the company in terms of profit and cost. The bug is a part of the system that lets pilots request holiday time off. The system allowed too many pilots to take vacation, and as a result, American Airlines (AA) doesn’t have enough pilots to fly their planes.

    In case you wonder about the scope of this, it’s 15,000+ flights that are affected. AA flies about 6,700 flights a day and are the world’s largest airline by some measure. They are offering pilots 150% of their normal rate, which is apparently the max allowed by contract. I’m sure some pilots will take the offer, but I’m also sure there may be some flight cancellations. After all, pilots have likely made their own plans, which they may not want to change. Apparently some have as the news as of this writing is a few hundred flights.

    The complexity of airline scheduling software has to be quite high, and while this won’t really endanger lives, it will affect the profitability of AA. This is definitely a development mistake, and one that should have been caught in testing. I don’t know how often this software is updated, nor what their process is. Someone suggested this is outsourced software, but that doesn’t matter. We clearly have a software development failure here.

    I certainly think developers are responsible for testing their code and meeting requirements. This includes not assuming the happy path is followed. However, this is also a place where we need (as an industry) to learn to better test the bounds and limits of our software. We need to be sure that we account for not only security, but pushing the limits of use by legitimate users. Software mistakes cost money, and as we continue to use computing more and more, we need to become better at testing our code. Our systems are very complex these days, more complex than a single person can handle. We need better testing everywhere, including the database.

    Please, make an effort to test better next year, in both your application and database. Use larger data sets at some point (both numbers and dates), add automated testing as a part of your development routine, and assume that users will do stupid, silly, and malicious things with inputs.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Avoiding Development Taxes

    When I first worked in an environment where multiple developers needed to release software, I found too many rules and constraints for smooth development. The process was developed by one person, or for one application, and somehow expanded to encompass all work being done in the organization. It was if we weren’t trying to think about our creative work as developers and DBAs, and instead viewed our systems composed of widgets built in a factory, each one a copy of the others, perhaps just a bit bigger or smaller. We used the same set of rules for the mission critical finance program as we did for the department vacation scheduling application.

    While working inside an overreaching command-and-control mentality, the idea of moving faster or releasing software more often was seen as really dangerous. Two decades of Agile methodologies, and the rise of DevOps have started to change this for many companies. Each year the State of DevOps report seems to show more and more organizations finding ways to build better software, often by having their operations people embrace the concepts and processes used by software developers as a way of building and managing their environments.

    Databases still lag behind, and as Donovan Brown says in this piece, if you’re not automating the back-end work of code deployments to the database, you’re still “faking” some of your DevOps process. Databases must maintain the state of data, even as transactions take place around code changes. Ensuring that we properly handle those data changes is a challenge. However, by treating the database changes as similarly as possible to application changes, we can minimize risk and learn database development techniques that help us push forward without being reckless.

    Tools have gotten better for database development. Microsoft has some tools for SQL Server, and various other vendors such as Redgate have others. Good tools are essential, but each new one adds some complexity to the environment. That’s one reason why having a minimal number of tools and platforms reduces the friction of getting things done. I see this as a big reason SQL Server on Linux might take off. Plenty of companies don’t want Windows in their infrastructure when most of their systems are on Linux. Not that Windows is hard, but consistency makes everything easier for a staff.

    This is the same reason why I’d say that it’s worth sticking with a database platform or two and not experimenting with each new type of system that comes out. If your staff knows SQL Server, then adding in MongoDB or Cassandra means there’s a learning curve, or a “tax” as the piece notes, to getting things done. This same tax gets paid with each new vendor, platform, language, or technology you take on.

    Database work is hard, and once your staff gets good at building and deploying changes, you want to take advantage of their knowledge. For companies that have solid development and deployment practices, stick with the things that work well. However, if you have staff that turns over regularly and don’t have a mature process, you can reduce the “tax” you pay for database development. There are companies that have paid some of that integration tax, that learning effort, and they’ve got tooling to help you build code better and faster. Just as I wouldn’t want to code my own build server, I don’t want to code my own database deployment tools. I could, but I’d rather spend my time solving problems and use the tooling that someone else has built to make my job easier, and provide a consistent coding experience for my developers.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5MB) podcast or subscribe to the feed at iTunes and Libsyn.