Voice of the DBA

Tag: version control

  • The Control Poll

    This editorial was originally published on May 17, 2013. It is being re-run as Steve is out of the office.

    I was reading about version control systems (VCS) recently, brushing up on some skills, and saw this quote in a thread:

    “There is no excuse for not using version control, even for a small project developed by single developer. Setting up local version control is beyond trivial, benefits huge. Any developer not knowing that cannot be considered good nor experienced.”

    That’s quite a pronouncement, and one that I believe is very true. No matter what type of development you engage in, I’d expect that you’d understand the benefits of using version control, and the dangers of not using it. It’s just like never backing up your system. I’d think that any developer that cares about their craft and is a professional has used version control. The really good ones will insist upon it.

    However I know that the decision to use a VCS is not always made by a developer. The company building the software might feel differently, and while I’ve always asked for a VCS, I have ended up with a series of folders on a share, named for dates, each containing a zip file of all our code at the end of that day. It was the bare minimum of version control I could live with, and fortunately we got by with just two people coordinating work. Any more than that and I’d insist on some type of VCS.

    This week, I wanted to ask how many of you voluntarily or involuntarily might be forced to do something similar.

    How many of you skip source control for certain apps?

    Even if you have source control for those large, multi-person teams, are there apps that you avoid putting into a VCS? What about your database code? I think it’s important that you keep all your code, whether for the front end application or database objects, in some type of Version Control system. If your boss won’t buy one, then check out Git or Subversion, both of which are open source and free.

    Let us know this week how you feel about source control and whether or not you decide the effort isn’t worthwhile for your projects.

  • Bringing DevOps to the Database

    Abstract:

    DevOps is a hot topic in today’s software development world. However most of the knowledge and experience with DevOps is based around application software and ignores the database. We will examine how the concepts and principles of DevOps can be applied to database development by looking at both automated comparison analysis as well as migration script management. Automated building, testing, and deployment of database changes will be shown.

    Level: 200 – Not to technical, but you should have some idea of how software development works to get something out of this talk.

    Goals:

    1. Conceptually understand how DevOps principles can easily be applied to the database in either state based or migrations based changes.
    2. Learn how databases can be automatically built and tested in a CI/CD environment
    3. Understand how database releases can be automated, but are different than those deployment/releases made for other software.

    Demos:

    This presentation has a few demos where I show how I stored database code in a VCS, perform an automated build, and use a scripted release process to move code from one database to the next. The demos are pre-built, so I just show how they work. I can do this with:

    • git, VSTS Build and VSTS Release
    • git or SVN, Team City, and Octopus Deploy

    Downloads:

    There are a few versions, as this talk has evolved.

    GroupBy – Bringing DevOps to the Database Groupby.pptx

    Live!360/VSLive – VSLive-Orlando 2016.pptx

    SQL Saturday Cambridge – Bringing DevOps to the Database.pptx

  • Ignoring Database Version Control

    I find myself surprised that so many people resist keeping their code in version control. I think this is less people all the time, but I don’t have good percentages because at this point I think a fair number of people embarassed enough to not answer the questions. However, in quick conversations, I still find people that don’t think a VCS is necessary. I can’t conceive of this for application code, but more and more I think this also makes sense for database code.

    I ran across a blog that asks you to find a way to keep your database code in a VCS. While I’m glad that the author provides some ideas on how you might script this yourself in a pinch, I think that’s silly. There are tools to help, whether paid for ones, like SQL Source Control or free utilities. In either case, it makes sense to work with something that has been tested a bit rather than inventing your own. It seems that running scripts is simple and easy. In practice, these tools add value because of their maturity. Some of the mistakes you’ll make with your own software have already been made, and solved, with many of these frameworks and paid for tools. Don’t spend your time maintaining software scripts when you need to be developing other software.

    I also think the author makes the process sound a bit simpler and easier than it is. In practice, moving between schema versions in a database is typically a one way process, and rollback scripts are of dubious benefit. A discussion of this post on Hacker News shows some of the shortcomings in trying to version database schemas and easily move between versions. In practice many database schema changes can be rolled forward and back, and the corner cases that are very difficult for tools to solve, aren’t items that occur too often. Really, it’s table changes that are hard, and having a good process helps here, but these are usually one way changes.

    However, the reason I think version control is important is that database development is often disconnected between the database and application. Changes might occur in the database long before they are used in code, or they may need to be undone in development, perhaps in a destructive way that isn’t possible in production. The history of changes, and understanding of what needs to be changed, is much easier with a record in a VCS.

    These days there are a variety of ways to capture your schema changes and record them. I would encourage you to investigate some method and give it a try. Even building the habit of manually recording changes by all developers will help you to move faster and faster in your development process without worrying about losing track of what’s happening.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.8MB) podcast or subscribe to the feed at iTunes and Mevio .

  • The Complexity of Branches

    Branching code is hard.  Well, branching isn’t hard. Just right click in some GUI or or type “git branch” and off you go. The actual branching of code is pretty simple and painless. In some sense, branching might be too simple in today’s tools as developers are almost encouraged to branch in a VCS and “test something” quickly without a lot of thought.

    Even merging has become easier, with tools to visual code changes, and automatic merging in many cases. Despite the hassles of merging, it often isn’t too difficult, which can mask the complexity of branching.

    However branching can easily devolve into a nightmare of multiple versions of code, with various developers unsure of the impact of their changes on a merged code base. This is especially true in a large code base where many developers each have their own branch where they work. With pressure from multiple individuals to patch bugs, add features, and streamline performance, I’ve met developers that end up building changes in three or four branches at the same time.

    Merging all the work together, into the correct main branch, becomes tricky.  We’re all human, and I’ve seen developers merge a change into the wrong branch. Tools try to limit this, but it still happens. The more branches that are in use, the more time that ends up getting spent on the administrative action of reconciling code differences rather than building new code (or fixing old code).

    Understanding our code is hard enough, without adding the additional complexity of versioning multiple files and objects. My philosophy has always been to limit branches as much as possible, usually only allowing one, or at most, two, per team. We have also endeavored to merge back changes every few days, just to limit the amount of time spent performing merge reconciliations. I’d rather have developers have the pain of merging daily on a shared branch than dealing with branch merges less frequently (usually more complex merges).

    We’ll always have mistakes made, but using automated testing and continuous integration builds on all branches can help limit the number of obvious mistakes that break code. This won’t prevent bugs, but our tests can help reduce the number of bugs, especially regression bugs, over time. At least, they can if we use testing on all branches, all the time.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.2MB) podcast or subscribe to the feed at iTunes and LibSyn.