Tag: DevOps

  • Moving Lookup Data with ReadyRoll

    I’ve been using ReadyRoll for a small project and wanted to move some lookup data. There are some tables where I want to keep data in sync across environments, so if my build and release pipeline can do this, great. ReadyRoll makes this easy, and this is a short post to show how.

    The Current Environment

    I’ve got a table in my dev database that has SQL Server versions. I can see the list here:

    2018-01-31 12_15_05-SQLQuery6.sql - (LocalDB)_Projectsv13.SSBuilds_1Dev (PLATO_Steve (51))_ - Micros

    In my QA and staging systems, I see this (only one is shown, but it’s the same in both).

    2018-01-31 12_13_41-SQLQuery5.sql - dkranchstaging.database.windows.net.ssbuildsRRstaging (vstsdeplo

    The goal is to have this data migrate with other changes in a build and release pipeline.

    Adding Data to the Project

    I’ve already built this as a ReadyRoll project and set up a build and release pipeline in VSTS. Now I want to include my data.

    In the ReadyRoll tool window, I have refreshed the project and I see everything in sync. That’s good, and that’s how I want to be starting a small section of development.

    2018-01-31 12_11_09-builds_azure - Microsoft Visual Studio

    I want to find my table, so I need to expand the “Identical Objects”. When I do that, I see a list of all objects in the database.

    2018-01-31 12_17_22-builds_azure - Microsoft Visual Studio

    I can scroll down to find my table. When I do, I’ll right click it to get a few options. One of these is “Include Table Data”. We want to pick this one.

    2018-01-31 12_20_01-

    Once I do that, I get a note to refresh again. This is so RR can determine what table data needs to be added to a migration script.

    2018-01-31 12_21_07-builds_azure - Microsoft Visual Studio

    My changes show that I need to get 11 rows of data into a new script.

    2018-01-31 12_21_44-builds_azure - Microsoft Visual Studio

    I click Import and generate script, which will build my migration script and add it to the project. In my case, this is script 4.

    2018-01-31 12_23_00-builds_azure - Microsoft Visual Studio

    Note that this script has a few things in it, based on a call to SQL Data Compare in the background. First, it sets a dateformat. Next, it does the insert for an empty table, since that is the situation I’m in. Last, this uses the SET IDENTITY_INSERT option.

    If you don’t like these options, change them here. You can alter this script to suit your environment. Remove the IF, let the identities be what they are, make the changes that matter to you.

    The only thing I’ll do is click the script name twice to edit it.

    2018-01-31 12_25_08-builds_azure - Microsoft Visual Studio

    The numeric sequencing is important. The rest, not so much. I’ll choose something simple here.

    2018-01-31 12_25_21-builds_azure - Microsoft Visual Studio

    Build the Project

    I can build locally, and I always should to be sure things work. Once I see this and I’ve tested a few things, I’ll let the “system” do more work.

    2018-01-31 12_28_32-builds_azure - Microsoft Visual Studio

    The nice thing about ReadyRoll is that I can include my database project alongside application projects, if I want. For application developers or hybrid developers, I can use a consistent interface for saving changes to version control.

    In Team Explorer, I’ll check changes, add a comment, and then Commit and Push.

    2018-01-31 12_30_36-builds_azure - Microsoft Visual Studio

    If I go to VSTS, I’ll see the build in progress. I’m using a local build agent and usually within 15 seconds, my build will start. Sometimes it’s really fast.

    2018-01-31 12_31_04-builds_azure-CI summary

    If I click the build number, I can see I’ve barely caught this before it finished.

    2018-01-31 12_31_19-Build 36

    The build completes, so now I need to check releases. I have a CD trigger that will deploy to a local QA instance when the build succeeds. If I go look at the release, I’ll see that’s occurred.

    2018-01-31 12_35_49-Release-14 - Visual Studio Team Services

    This worked, so let’s check the QA instance. As you can see, my data was deployed.

    2018-01-31 12_36_43-SQLQuery7.sql - (local)_SQL2016.ssbuilds_rr_sjones_2_integration (PLATO_Steve (7

    If you were watching closely, you see I have a small data issue. I need to correct that, but that’s for another post.

    ReadyRoll is a part of the SQL Toolbelt, and if you’re a customer, I’d urge you to start a PoC and see what you think of the tool. ReadyRoll Core, with limited features, is included with Visual Studio Enterprise, and if you’re a customer of Microsoft, you can try that. We also have 14 day trials if you’d like to do this on your own.

  • No Magical DevOps

    I saw this comment in a post a short while back. A reader said “I’m still trying to wrap my head around “devops” as it pertains to database deployment. … how do you deal with those potentially dangerous/destructive changes without manually coding? … what tools are there that help in that regard? That’s what database developers really need. The only tools I’ve found that’ [are] useful so far are experience and care.”

    We have some tools over at Redgate software to help with database DevOps, but there aren’t any magic solutions that can alter your database in some way that doesn’t impact users and preserves all data. In fact, if anyone claimed to have a tool, I wouldn’t trust it because there are just too many variables in database deployments. I certainly wouldn’t trust a tool with automated rollbacks. Ultimately, experience and care are needed to build better software.

    And, by the way, DevOps doesn’t solve deployment issues. All DevOps does is help you automate the tasks you’d run manually in a reliable, repeatable fashion. DevOps teaches you to learn from your actions (gaining experience) and experimenting with small changes so that your mistakes aren’t as impactful (developing with care). DevOps isn’t magic, it’s good, solid, learning from what works and what doesn’t, altering our habits, and striving to be better. DevOps is often just providing a reliable, repeatable way of automating the things you do manually now, not inventing new processes.

    I do believe in DevOps and I think that it can work well, but it requires balance, commitment, and huge culture changes. Automating code deployment is easy with some tools. Getting people to write better code is hard. I can help with the former, and I try to help with the latter here at SQLServerCentral, but it’s really up to most of you to make an effort to become better at your craft. Getting that to work in any size organization requires that your management supports teams learning to be better and tolerating a few mistakes here and there.

    Many successful developers and DBAs have been practicing for many years (even decades) what the media and high profile companies have been calling DevOps for mere months. You can join them by looking to improve all parts of your software development pipeline through collaboration and automation. Just don’t forget to also learn to write better code along the way.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.5MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Live on Channel 9 with ReadyRoll

    I was interviewed in Redmond a couple months ago and the video was just published. I show and demo some database migration work with ReadyRoll in this video:

    https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/Database-DevOps-with-RedGate-ReadyRoll/player

  • Automating SQL Clone Creation with PoSh

    I think SQL Clone is one of the game changing products from Redgate. This product really fits into a DevOps mindset, allowing me to quickly and easily build (and rebuild) a dev database.

    While the agent web pages make this easy, they’re slightly cumbersome and the PowerShell cmdlets fit better with a DevOps flow. In setting up a Query Store demo, I found myself changing some data and needing to reset my database rapidly, so I built a quick PoSh function to do this for me.

    Here’s my function:

    function Add-ADWClone {

    param([Parameter(Mandatory=$true)][string] $CloneName)

    $mycredential = Get-Credential

    Connect-SqlClone -ServerUrl ‘http://socrates:14145’ -Credential $mycredential

    # remove the image if it exists

    $SqlServerInstance = Get-SqlCloneSqlServerInstance -MachineName Plato -InstanceName SQL2016

    $Clones = Get-SqlClone -Location $SqlServerInstance

    if ($Clones.Name -contains $CloneName) {

    $CloneToDelete = Get-SqlClone -Location $SqlServerInstance –Name $CloneName

    Remove-SqlClone -Clone $CloneToDelete | Wait-SqlCloneOperation

    }

    # Create the new image

    $image = Get-SqlCloneImage -Name ‘Adw2014Base’

    $sqlServerInstance = Get-SqlCloneSqlServerInstance -MachineName Plato -InstanceName SQL2016

    $image | New-SqlClone -Name $CloneName -Location $sqlServerInstance | Wait-SqlCloneOperation

    }

    In this function, I’m taking the name of a cloned database. I’ll use that to check if the clone exists, and if so, remove it. I do this by using Get-SqlClone. Once that’s done, I get the image, which is static here (this is a function for a specific project) and then I’ll create the new clone.

    This works great. If I run the command, I’ll get a clone being created. You can see my client in the back creating the clone database.

    2017-11-28 14_46_08-SQL Clone

    If I re-run the command, I’ll see the delete.

    2017-11-28 14_48_22-SQL Clone

    And then the clone create again.

    Changing Development

    When I first saw a prototype a few years ago, I could immediately think back to being a full time developer and the hassles of manipulating my development database, making data changes to test code, trying to reset them, writing scripts to undo changes and more. At some point I tried to perform backups and restores of a standard database, but I’d keep forgetting to update things.

    SQL Clone makes this easier, and together with some way to push/pull code from a VCS to your database, it means that I can quickly reset a database back to a known state. One of the things that I’d like to easily do is whack my development database, recreate a new one, and then move on with writing code. If I make a mistake, I repeat the process.

    Moving On

    This is a basic Proof of Concept, something I just whacked together for a project, so I’ve coded in the image name, and I get the credentials from the user. I could clean this up, and have it as a simple up-arrow, enter from the  PoSh command line that I use as I need to reset my system. Or code in a saved, secure credential that lets me double click some batch file to run this for me.

    Automation speeds up development by removing simple tasks from the developer. SQL Clone makes it easy to reset my database to a known state (the image) and create new databases as needed.

    If you haven’t tried SQL Clone, give it a try today by downloading an evaluation.