Voice of the DBA

Tag: software development

  • The Rename Game

    As I’ve been talking about database development, Continuous Integration, and Continuous Deployment, I keep hearing complaints and concerns about the problems that occur when we make database changes. In particular, I often hear about breaking changes made to databases, that require some manual scripting work.

    It seems that the most common breaking change is adding a not null column to a table with existing data, but certainly there are other problematic issues. The other common change is renaming objects. In the past, I’ve rarely renamed anything once I’ve deployed tables to production, but I’ve run into quite a few developers that note they regularly change names. This week I’m wondering how common this is for most of the data professionals out there.

    How often do you rename objects?

    It could be tables, views, columns, constraints, whatever. If you rename anything in your database, I’m curious if it’s a regular event. Is this an event that causes you issues with your software development process, let us know. I’d also be curious with how you manage tracking and deploying the code from development through to production when you are making these types of breaking changes. Are things still scripted and able to be automated or do you force manual steps.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 1.9MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • When Should We Be Cleaning Data?

    I was reading Grant Fritchey’s update from the Business Analyst Conference  and noticed this quote: ” There were lots of sessions on the need to clean data, which implies that we need to do a better job at data collection in order to support this wild new world.” Grant was talking about the fact that many of the sessions seemed to imply that it’s mostly the processes around data that cause issues with data, not the people.

    However is that really what we should do? Do we want to do more and more cleansing at the point where people enter data? I’m not so sure that’s the case. The more that I deal with various applications and forms, the less I want to see too many fields and values required. Certainly there are pieces of data that are necessary for action, but I often find there are lots of additional fields that analysts want, but are more of a burden than a necessity.

    Most of us as data professionals design tables to handle the needs of an application. We put fields in tables that we expect to fill with data, based on a specification. However the real world is messy, and often the data we want to collect and store isn’t always available. The question is, do we force failures in the application or do we handle missing data?

    I don’t want to eliminate DRI, or go to an EAV models for lots of data. However I do think that we need to ensure we allow defaults and work with developers to allow data in our systems that might not be complete now, but perhaps will be in the future. We should work with analysts to help them build reports that can handle defaults or missing fields. We can work with developers to allow applications to request updates to data later and then design ETL that can efficiently fill in the updated information.

    Applications and software need to be flexible to work with problematic data. We have the ability, as data professionals, to help our clients still find meaning in data that might not be as complete as we’d prefer, or they’d prefer. However we can still find valuable information in what data they have..

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Your Job

    This quote is a great:  “I don’t really care if your code runs locally. If your code just runs locally, then my only option is to sell your computer so that our users can use our software, and that really doesn’t scale.” That quote is from an open letter to engineers and talks about what the job of a software engineer really is. It’s not just to write code, but to make sure our code and the entire system works for users.

    That’s important. It doesn’t matter how well you code if the application only works on your machine. We traditionally haven’t been great at getting code from one system to another and ensuring it works, at least not in a smooth fashion. We haven’t really followed an “engineering” process when we write, and deploy, software.

    The DevOps movement is changing this a big, bringing automation, scripting, and repeatability to both software development and operational deployment. All of these things should help us adhere to a more repeatable engineering practice. The commoditization of powerful hardware has also helped as many client environments might be nearly as powerful as development systems these days. It also means that our test systems can be good substitutes for production systems, at least if we can quickly build up and tear down large virtual environments.

    However we still need to be sure that we always remember that our code needs to work on other systems. Whether it has to scale down to meet the needs mobile devices or slightly older browsers and desktops, or our code needs scale up to handle thousands of times more transactions on a production database server than it processes on our laptops. We have to always consider the needs of the end user environment as we develop software. We need to test it in those situations, and broaden our skills to ensure that our applications satisfy the needs of the clients. And, of course, we should be able to quickly and easily deploy our changes to any other environment.

    Steve Jones

     

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Documenting with Tests

    Documenting code isn’t a task that many people enjoy. Those of you that have been through a computer science curriculum probably added lots of verbose, and obvious, comments to code. It’s probably unlikely that you ever found much value in the comments in a paying job.

    There are all sorts of ideas on how to document your software, writing good comments, and even refactoring code to remove unnecessary comments. I’m sure that if we put five developers in a room, we’d end up with eight different ways to comment code that would be debated, with no agreement on how to proceed.

    I ran across an interesting approach from Ed Elliot on documenting your code with unit tests. I hadn’t thought about the tests providing some documentation, but it’s an interesting idea. I’d have to work with the concept a bit, but I’m skeptical I’d get enough information from unit tests to ensure I understood what a stored procedure was doing, especially if I had 10 tests for a long procedure.

    I do think unit tests are important, and perhaps in conjunction with some type of code header that gives the requirements the procedure fulfills, I’ll get enough information to understand the code.

    I don’t have a universal solution that I think will work in most situations, but I do think that having a tool like intellisense or SQL Prompt helps you self document code with expressive names for columns, variables, and aliases. If nothing else, those phrases for variables can clue the next programmer in to what is happening better than single letter names.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn.