Voice of the DBA

Tag: Azure Devops

VSTS–Changing the Default Build Queue

I wrote recently about the Release agent using the hosted queue as the default, which is sometimes a problem. The Build process has the same issue, though the way you change things is different.

In a build, you see a set of tasks with various menu items across the top, as shown here.

If you click on Options, which will then be underlined, on the right side the Agents options appear. The default, as with releases, is the hosted agent. Change this to another pool if you want local builds.

Microsoft makes this the default for a couple reasons. One, it’s easier. No need to install an agent, which might be an issue for some people. I think it’s simple, but for some teams, you might not want this.

The other reason is that while you get 240 minutes per month, if you’re like me and built often, you can blow by this. Also, I have machines available, so I’d rather build locally, and I wish I could make this a default.

Microsoft wants to make money, and that’s fine. I like the service, and in a commercial environment I’d weight the cost here v purchasing something like TeamCity + add-ons, and then make a decisions. Since I dislike uncertainty, and not knowing the costs, I lean towards local builds always. YMMV.

June 12, 2017
VSTS– Hosted Agent is the Default for Releases

In keeping with the idea I don’t want this to get out of date, this is VSTS as of Jun 1, 2017. If your screen looks different, find a different article.

I was setting up a test the other day using VSTS (Visual Studio Team Services) and once had something bite me (again). This has happened a few times this year as the service has evolved, but it is one of those things that I keep having to change.

Here’s a new Release definition, where I start with an empty process. I clicked “New Release Definition” and then selected an Empty task, hooking this up to my current CI process.

Usually I add tasks, and then create a release and execute it. Since I mostly work with databases, I’m usually releasing to a local SQL Server instance and want to use my local agents. I have separate ones running on my desktop and laptops, so I can easily demo releases in different environments.

However, when I’ve done this, usually things fail at first. If I’m not paying attention, I won’t notice my build agent looking like this:

That’s the screen showing nothing happening with my agent.

Grouping Tasks in VSTS

Earlier this year, the release process in VSTS changed to allow us to group tasks. I can now add different groups of phases, which I stumbled upon by clicking the arrow next to “Add Tasks”. This is a UI Fail because the arrow in other parts of VSTS is linked to the text to the left (like the Add environment area).

I stumbled on this one day, thinking the arrow let me add tasks, but I accidently added a new agent phase. By default, we have a single agent phase, but clicking the arrow allows more.

What’s also interesting, and not intuitive to me, is that if I click on the “Run on agent” area, I get settings to the right. This is different than the old release section, which had the properties as a global item at the top.

This is where I need to change to the agent queue, in this case, the default for my desktop.

This might not be a big deal, and I’m sure lots of people are happy to use the hosted agent. I’m always wary of costs, so I try to avoid spending compute dollars in Azure when I don’t need to, and this is a place I want to avoid spending $$.

Hopefully, after writing this, I’ll remember to set the queue for my future demos the first time. Now if I can just get the VSTS people to give me a “default” setting I can change…

June 5, 2017
Agent Phases in VSTS Deployment

VSTS (Visual Studio Team Services) continues to grow and change over time. It seems every few months I see some new features and changes, which are good and bad. Good in that usually things improve. Bad in that I get confused and lost at times, and sometimes have to re-acquaint myself with the system.

One of the changes I noticed recently was the Deployment phases, which come as Server and Agent phases. I won’t cover server here, but I wanted to make a quick note on Agent Phases because I think they’re handy and a nice packaging concept.

Agents

There are agents that run for VSTS to do work. I’ve written about how to download an agent so tasks run inside of your network (on a server or client). This post will look at the other side, how do I control an agent from the release process.

Once I get a new release definition, I’ll see something like this:

This is a blank deployment definition. To the left I see the environment I’m working with, in this case, just the generic Environment 1 name. For each environment, I can set up tasks. On the right I have the Add Tasks button, but also the “Run on Agent” item. This is the container where I will add tasks.

I used to just add tasks with VSTS 2015 here, expecting that everything would run on an agent. VSTS has hosted agents that contain a set list of software, and also downloadable agents that you run on your servers or workstations. Things changed recently to include phases.

Phases

The phases are place of execution, which allow a series of tasks to be grouped together. VSTS and TFS 2017 give you two choices. We now have Agent or Server phases. The difference between these is that I can choose where items execute, on an agent or on the server (VSTS or TFS). In addition, I can set these to execute in parallel if need be.

If I click the down arrow next to the Add Tasks, I’ll get this dialog. This allows me to set up a process that can run on the VSTS server, or one that can run on the Agent.

I haven’t played with server phases, but when I add an agent phase, this gives me a section under which I add tasks. Note that when I click the “Agent phase” item, I get a new blade to the right. Note that the pool for these agents is the default.

This threw me early on because I have a separate pool of agents for mobile development. As I upgraded some deployment definitions, I couldn’t figure out why I didn’t have any agents available to execute my tasks. It turns out I needed to change the agent pool here.

There are options here, but I haven’t really played with parallelism or demands since I typically deploy to one system at a time.

Once I’ve selected an agent, I can click Add Tasks and then add a series of tasks to that agent. If you look below, I have added a couple tasks to each of the two agent phases. These will run one phase, then the other.

Each of these phases is independent, and as of the time I wrote this, there’s no way to copy a phase and duplicate it.

I’ve used this to upgrade my release definitions by adding new tasks, and then copying settings between one phase and the other. This is a manual, copy/paste operation, but I can easily see if I’ve duplicated everything by going back and forth quickly.

Once I’m done, I can just click the “x” to the right of the agent phase and delete the entire phase. I have to confirm this action, and then I’ve removed a complete phase.

Right now I haven’t see any benefit outside of the upgrade scenario, but that’s useful. I will duplicate my tasks, disable all the ones in one phase and then see if the other one runs. If it doesn’t, I can always enable the old tasks and disable the new ones.

I expect as I look to perform more complex deployments I will use these to logically separate things, and also perhaps use the server to do some things when I don’t need an agent.

April 17, 2017
Making VSTS Deployment Changes to Databases without Breaking Your Application

One of the things that I do a lot is demo changes to databases with CI/CD in a DevOps fashion. However, I also want to make some application changes to my sample app without breaking things. As a result, I’ve built a few ideas that work well for both situations. I found recently these ideas can help me when I need to actually upgrade or change my CI/CD pipeline.

Note: DevOps isn’t a thing, it’s a set of principles, ideas, and culture that produces results. I will endeavor to ensure I call out the specific principles I use to adhere to DevOps ideas. In this case, automation and CD.

Release On Your Schedule

One of the principles of DevOps is that we use feedback loops to ensure information moves from right (operations) to left (development). One way of doing this is to release more often, though this isn’t required. What we really mean is release on your schedule, when you want, and allow developers to get feedback on their work quickly.

If a developer takes a month to build a feature, they don’t need hourly or daily releases. They need a release after the month is over (assuming testing, review, etc. has taken place). If releases occur on the 5th of the month, and the developer finishes on the 6th, they must wait a month before they get feedback. What I want to ensure is that we can release on the 5th, and also on the 6th if I need to.

Upgrading my VSTS Pipeline

I wrote in another post that I planned on upgrading my Redgate DLM Automation tasks in VSTS. I was doing this on a trip to a conference, and I didn’t want to start making the demo changes I’d make as I’d have to undo them, and it’s possible that I’d forget to clean something up. I hate making that silly mistakes, so I needed a way of testing my build and release without breaking demos.

I decided to use a technique that I use in presentations when I’m talking process and not code. In those cases, what I deploy doesn’t matter, but if I change random objects, sometimes I break the application using the database. As you can see here, I needed to do a bunch of tests, and repeat some.

Deploy Often

When I talk with Redgate clients, and they are starting to get comfortable with the idea of deploying on their own schedule, they will sometimes make innocuous changes that trigger a deployment they can test, without breaking things. For example, a user might take this stored procedure (partially shown):

ALTER PROCEDURE [dbo].[GetEmailErrorsByDay]
@date DATE = null
/*
Description:

Changes:
Date       Who         Notes
———- —         —————————————————
2/14/2017 WAY0UTWESTVAIO\way0u
*/
AS
BEGIN

IF @date IS NULL
SELECT @date = DATEADD(DAY, -1, GETDATE())

SELECT Errcount = COUNT(*)
   FROM dbo.EmailErrorLog
   WHERE EmailDate = @date

and make a small change. Perhaps they add SET NOCOUNT ON, or maybe they’ll add a comment. I’ve even see someone add this code:

…

AS

BEGIN

SELECT 2 = 2

…

These aren’t big changes, and certainly choose which stored procedure to change (one that isn’t used a lot or is critical). These are changes to test your process, gates, approvals, deployments, etc.

I decided to try something else. I tend to write this code when I want to test changes, since I can be additive with a procedure like this:

CREATE PROCEDURE Get7

as

SELECT 7

Or I can modify things with this:

ALTER PROCEDURE Get7

@plus int

as

SELECT 7 + @plus

In either case, I can see if my changes go through. Since I often deploy to multiple environments (QA, Staging, UAT, production, etc.), and I don’t always have deployments go all the way through (see my image above), I will usually end up creating Get7, Get8, Get9 as subsequent procedures. This way I can continue to commit new changes to the VCS, get new builds, and get new releases.

I can also do this with tables. My favorite is MyTable, MyTable2, etc. I usually just have an integer MyID column, but I can add other ones to test the ALTER process. I can even use this to test data movement, static (reference/lookup) data, or anything else to do with tables.

Eventually (hopefully) I get clean deployments all the way through to all environments.

Cleanup

I sometimes get collissions, where a test will return the error that “Get10 exists”, and I’ll move on to Get11. However, I don’t want to leave those objects in all environments. After all, likely I’ll find a way to improve things in the future and I’ll want to repeat this testing.

This usually is one last deployment for me. I’ll delete these objects in dev, and then deploy the changes all the way through to production. This allows me to test my checks to prevent data loss, if I have any. Including if approvers actually read scripts

April 17, 2017