Tag: continuous integration

  • DLM Automation–Making a Database Connection

    One of the things I’ve been wanting to do is dig more into the command line automation cmdlets from the Redgate DLM Automation suite. While there are plugins for many build and release systems, I should be able to customize, or fall back, to the command line if necessary.

    Plus, this allows me to get a better feel for what’s happening at each step.

    This post looks at the basics of making a connection to a database. Note that this is documented on the DLM Automation 2 Cmdlet Reference page.

    New-Database Connection

    When I started working with the connections, I used the New-DatabaseConnection cmdlet. However, when I run this now, I get this:

    2016-10-14 15_32_04-DLM Automation–Making a Database Connection - Open Live Writer

    That’s fine, as my call to the new cmdlet works.

    2016-10-14 15_32_34-cmd - powershell

    Do I know what’s happening here? I should investigate further.

    First, if I’ve run this, what does it do? Not much, apparently. If I run an Extended Events session looking for a connection, I don’t get one. Let’s try a new command, this time using a name and password.

    2016-10-14 15_37_53-cmd - powershell

    That’s an invalid password for my instance (as it should be on every instance in the world). However, I don’t get an error. No connection was made. In this case, the cmdlet is storing off the connection credentials that will be used. If I don’t provide the user/password, then a trusted (Windows auth), connection is assumed.

    Can I now test this? Sure, I’ll use the Test-DLMDatabaseConnection to make a connection.

    2016-10-14 15_39_37-cmd - powershell

    This fails. That’s good to know. Now I can use this to actually determine if I target database is alive before I continue processing. My preferred method here would be to wrap this up in some error handling. For example, I can use a try..catch structure in PoSh. I’ll switch to the ISE to make this cleaner. Here’s the code

    try
    {
    Test-DlmDatabaseConnection -$test -ErrorAction Stop
    } 
    catch
    {
    Write-Host "error"
    }

    In the ISE:

    2016-10-14 16_22_45-Windows PowerShell ISE

    I can also get the error details from the current object ($_). Note, I’m not a PoSh expert, so if there are better ways, let me know.

    2016-10-14 16_36_48-Windows PowerShell ISE

    I can also pass in some common parameters, like –ErrorAction and –ErrorVariable. Then I can use these to decide whether or not to continue. In my case, I’ll just write a message. Here’s my code:

    $testconn = New-DlmDatabaseConnection -ServerInstance ".\SQL2014" -Database "st_integration" -U "sa" -Password "test"
    try
    {
    Test-DlmDatabaseConnection $testconn -ErrorAction SilentlyContinue -ErrorVariable ErrorHolder;
    } 
    catch
    {
    Write-Host $_.Exception.Message
    };
    
    if ($ErrorHolder)
    {
    Write-Host($ErrorHolder)
    }
    else
    {
    Write-Host("Everything works");
    }

    That will give me this:

    2016-10-14 16_41_08-Windows PowerShell ISE

    If I remove the user and password parameters, I get this:

    2016-10-14 16_41_29-Windows PowerShell ISE

    You didn’t think I’d publish the real sa password, did you? Of course not. If I change the code to the right password, things connect.

    Checking the Object

    What if I just want to check the object? Well, there are properties associated with the object. For example, if I use a simple variable to assign to the object, I can see some properties. If I just type the variable name, $testconn, I get these properties:

    • ServerInstance
    • Database
    • SQLServerCredential
    • ConnectionString
    • Description

    You can see these returned in a test session.

    2016-11-14 16_58_29-cmd - powershell

    Other Options

    I have other parameters I can use with my connection, or rather, in place of my connection object. I can use a connection string, with all of the parameters that are valid there. For example:

    2016-10-14 16_53_17-Windows PowerShell ISE

    If I enter the correct password, I get this:

    2016-10-14 16_54_09-Windows PowerShell ISE

    My connection string can contain all of the items I might normally use: ApplicationIntent, Encrypt, NetworkLibrary, etc. A full list of parameters is at: https://www.connectionstrings.com/all-sql-server-connection-string-keywords/

    The New-DlmDatabaseConnection object outputs a RedGate.DLMAutomation.Compare.SchemaSources.DatabaseConnection object, which is the input to quite a few other of the cmdlets. While not complex, it is worth understanding how to use this cmdlet, which is the basis for determining from which source or two which target you will be moving database changes.

    In another post, I’ll start to bind this cmdlet with others and actually perform some database deployment automation work.

  • Held Hostage by the Database

    Your database platform will constrain and limit the flexibility you have in evolving your software. It doesn’t matter which platform you choose, which type of database, or even the format for your data. At some point, you will be dealing with legacy data in some legacy schema or structure, and your development, and certainly deployment, will be slowed or impacted. Face it, the need to maintain state for your data is an impediment in a relational system. In a NoSQL or other store, the need to maintain code in your application that can interpret your data might be the issue over time.

    That’s not to imply this need to maintain state has to slow your development. On the contrary, there are many companies that find themselves moving quickly, able to make database changes on a weekly, or even daily, basis. There are multiple tricks and techniques for managing change, but ultimately the real secret is having a process that computers can follow and your developers adhere to.

    In other words, having some automation.

    Certainly you need to program the automation, or use tools like those from my employer, Redgate Software, but it doesn’t matter which what process you use. The process does need to be flexible because it will change over time. I can almost guarantee that the way in which you need to deploy code to the database in your environment will change over time. It has to as the needs and requirements of your business change.

    This means the way in which your developers need to build, test, and package changes will need to grow and evolve as well. While every developer and sysadmin needs to work within the process, the process does also need to be flexible as needs change. That isn’t to imply that your process should change every week, or for every deployment, but it will need to do so periodically, and hopefully, rarely.

    There are techniques to make deploying database changes easier on the developer and system administrator, but there are no magic techniques. All the tools I’ve encountered, including those from Redgate, do the same types of things you’d do manually. They just save you time and stress by helping you get set up and easily maintain your deployment tasks over time.

    Whether you use tools or not, please don’t allow the database to hold back your software development. Learn new ways to alter your database. Learn the ways to make changes to large tables. Learn how to avoid downtime. Just learn to design database changes in a better way and then automate the deployment of those changes.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • DB CI: Getting Test Results in VSTS

    One of the things I struggled with a bit was getting test results to appear in VSTS from my builds.

    Here’s my original Build flow:

    2016-10-07 10_50_32-Microsoft Visual Studio Team Services

    However, when I’d run a build, I got this in the Build summary:

    2016-10-07 10_51_43-Microsoft Visual Studio Team Services

    That’s not great, and I knew this could work better, so I started to look at where my test results were. Here’s what I did to get the results to appear.

    I added a Publish Test Results task to the end. This is in the “Test” section of the Task catalog.

    2016-10-07 10_39_11-Microsoft Visual Studio Team Services

    When I looked in the output of the build tasks, I found these files:

    2016-10-07 10_31_41-s

    I kept thinking I needed to get the .trx file to load, but that wasn’t correct. I really wanted the “*reports.junit.xml” file. After a little experimenting, I configured the task like this:

    2016-10-07 10_40_33-Microsoft Visual Studio Team Services

    Once I saved this (with a comment), and ran a new build, I could see test results.

    2016-10-07 10_31_50-Microsoft Visual Studio Team Services

    I could have just read the XML file in my build result folder, but that’s a pain. Typically we don’t care unless there is a failed test, but when there is, it’s nice to see what actually failed.

    Plus, having the results in the console means I can show the other developers and management that tests are being run and are checking code.

  • Webinar: Database Deployment with Git, Jenkins, and Octopus Deploy

    I’ve got a webinar later today that looks at how Redgate Software’s DLM Automation and other SQL Toolbelt tools can be used to smoothly move development changes from one database to another.

    Sign up and spend an hour with me this morning.

    This is part of a monthly series that Redgate is running, where we’re changing the platform out every month. In this case, I’m going to use a git as my VCS, storing changes in there as I make them to a database.

    From there, I will push the changes to a remote git repo and have Jenkins automatically pick up the changes and build, test, and deploy the changes to another database. I’ll make table changes, data changes, proc changes, anything you want. Just ask.

    However, that’s not all. From there, we’ll see Octopus Deploy take a package of changes and deploy them out to other environments, letting my manage and track the process with the click of a button.

    I’ve been doing this demos for a few years, across multiple platforms, but never on Jenkins. However, I’m amazed that things work as smoothly here as they do with TeamCity or VSTS.

    Building software is building software, including database software.

    If you’ve got some time today, join me and see how easy this can be.

    Sign up and see database changes made live.