Voice of the DBA

Tag: software development

  • It Starts with Version Control

    As a software developer, I learned about the value of version control over time. My first forays into development were as a child, where I made a single backup of my code on a separate floppy disk in the event I had an issue. This was my one-man-wolf-pack version control system, and it served me well, although it wasn’t efficient. In a few of my early development positions, we zipped up all our code each night to a separate folder, giving us the pentagram-version-control system. This worked as long as we didn’t need to go back over a week, but it still wasn’t very efficient. I think that because we worked on fairly small projects and could hold most of the code in our minds, we muddled through issues.

    Later I was introduced to a real version control system (VCS), Visual SourceSafe. Some of you cringe, but that system worked well. We could see the differences in code, and over time I learned to cut loose with my development, just trying new things, knowing that I could easily revert code backwards without much effort. Since then I’ve worked a bit with Subversion and Git, both of which seem to be very capable, mature systems. I still plan on getting some work done with Mercurial and TFS at some point, just to compare the features and functions.

    I started source controlling my databases after reading an article by Bill Wunder. This was over a decade ago, and it seemed to make sense to me. All of the things I had learned about VCS worked with databases as well, and I learned that having a database under VCS provided me with some stability in development, including the ability to roll back to previous versions of code when we found bugs. Trust me, checking out a previous version of a stored procedure is much, much less stressful than restoring an old backup just to look at stored procedure code.

    These days I see more and more people putting their databases under version control, which is comforting. It makes me think that we, as data professionals, are maturing in our industry. However there are still lots of people out there that don’t use a VCS for their database (DDL) code. Some don’t even use a VCS for their application code.

    There’s no excuse these days. Subversion and Git are free and open source. Download them, spend a few days, and get control of your code. All of the ways in which you can improve the efficiency and quality of your software start with more knowledge, and a bit of version control on which to apply your craft.

    Steve Jones

    Video and Audio versions

    Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

    Follow Steve Jones on Twitter to find links and database related items and announcements.
    Steve Jones Windows Media Video ( 23.4MB) feed

    MP4 iPod Video ( 27.2MB) feed

    MP3 Audio ( 5.7MB) feed

    Feeds are available at iTunes and Mevio

    To submit an article, rant or editorial,
    log in to the Contribution Center

  • Runtime Code

    I was talking with someone recently who was building a dynamic process that generated its own code at runtime to process ETL files based on a set of rules. It was an impressive system, and reminded me of how BIML can dynamically produce code and build packages.

    However then I started to wonder. If you produce code at runtime that then runs, how well is that code tested? I suspect most people would test the code generation process. Is this producing the code I expect, but how many would then write additional tests for the generated code as it executes? I suppose people might also just test the generated code, but if that’s the case, are they really testing the generation process extensively?

    I’m not sure there’s an easy answer, and not sure if there’s actually a strong testing framework that exists to help here. As BIML becomes more popular and even ETL frameworks are used more and more in our software processes, I hope that we develop new testing methods to better ensure we are building quality code. I know some SSIS test frameworks exist, and there has been some writing on the topic, but I wonder how many SSIS developers bother to perform anything more than cursory tests? My suspicion is very, very few.

    Ultimately I think we as an industry need to learn to build better software by improving our techniques, samples, skills, and testing.

    Steve Jones

    Video and Audio versions

    Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

    Follow Steve Jones on Twitter to find links and database related items and announcements.
    Steve Jones Windows Media Video ( 15.2MB) feed

    MP4 iPod Video ( 17.9MB) feed

    MP3 Audio ( 3.7MB) feed

    Feeds are available at iTunes and Mevio

    To submit an article, rant or editorial,
    log in to the Contribution Center

  • Test Coverage

    I’ve never had to work full time in a QA group, but I have had to perform software testing of an application and it wasn’t fun. Even as I worked through the various features, looking for edge cases, common mistakes, etc., I realized that running tests was something that really required more discipline and process than I was giving it at the time. Too often I ‘d realize that my regressions weren’t 100% duplicates of previous executions because I’d allowed too much “human effort” into my process.
    These days most testing of software is automated. I know QA groups still exist, and they need to handle some of the manual checks that are very hard to automate. However more and more testing is being pushed back onto developers to handle, with frameworks like NUnit and JUnit. In the database world, we haven’t done a great job of including testing into the code we write, but there’s a great testing framework we can use.
    TSQLT is a framework written by Sebastian Meine and Dennis Lloyd and it’s free. It’s been developed to help you write tests that can exercise your T-SQL code to determine if it’s doing what you expect. I’ve used it a little, and while I see the potential, I also realize that this will take some practice to learn how to more easily write tests that can cover the various potential places where bugs can be introduced.
    However, if you use version control, and you should, then you can easily spread the load of writing tests to all of your developers. Since the tests are stored procedures, they can be included in your VCS project and shared by all your developers. With a little effort from each member of your team, you might be surprised at the code coverage you can achieve with this framework.
    Testing is important, as we see over and over again as software is released and doesn’t work as expected. I think much of that is our fault, as software developers, for not improving our testing skills and discipline.
    Steve Jones
    If you want more automated ways to easily share tests between developers and execute them, take a look at SQL Source Control and SQL Test from Red Gate Software.

    Video and Audio versions

    Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

    Follow Steve Jones on Twitter to find links and database related items and announcements.
    Steve Jones Windows Media Video ( 20.9MB) feed

    MP4 iPod Video ( 24.8MB) feed

    MP3 Audio ( 4.9MB) feed

    Feeds are available at iTunes and Mevio

    To submit an article, rant or editorial,
    log in to the Contribution Center

  • Cyberwar

    There are no shortage of books about cyberattacks and hackers. Some of these fictional stories even postulate that countries will build armies and engage in digital war with other countries. Some of this has come true, with the US and Israel attacking Iranian targets, suspected Chinese hackers targeting the US, and the announcement that Britain is building their own cyber army.

    While there are certainly no shortage of attacks made on corporations every day, how long before countries make determined efforts to disrupt their enemies’ economies with digital war? It’s a scary thought, and given the poor security habits of so many developers, it’s possible that many companies might find themselves struggling to conduct businesses while under attack. It might not be any different than if conventional weapons were being used near our facilities.

    The state of coding by so many “developers” today is somewhat scary. It’s not even old applications that are vulnerable to SQL Injection, but even new systems that have poor security practices being used that are vulnerable.

    I think that secure coding practices like these should be implemented by anyone writing software. I think examples, frameworks, and presentations about coding shouldn’t use simple passwords and bypass checks. Yes, it’s a pain for those that teach, but it also means that shoddy coding practices aren’t proliferated as people borrow your code and alter your examples.

    Security is a problem, but I think a lot of the issues would be minimized if we, as an industry and professionals, learned to write more secure code as a matter of habit, not as an additional feature to be added later.

    Steve Jones

    Video and Audio versions

    Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

    Follow Steve Jones on Twitter to find links and database related items and announcements.
    Steve Jones Windows Media Video ( 17.0MB) feed

    MP4 iPod Video ( 19.9MB) feed

    MP3 Audio ( 4.1MB) feed

    Feeds are available at iTunes and Mevio

    To submit an article, rant or editorial,
    log in to the Contribution Center