What is a Failed Deployment?

When talking about DevOps, the goal is to produce better software over time. Both better quality as well as a smoother process of getting bits to your clients. There are a number of metrics typically used to measure how well a software team is performing, and one of the things is Change fail percentage. This is the percentage of deployments that causes a failure in production, which means a hotfix or rollback is needed. Essentially we need to fail forward or roll back to get things working.

For most people, a failed deployment means downtime. I’ve caused a service to be down (or a page or an app) because of a code change I made. This includes the database, as a schema change could cause the application to fail. Maybe we’ve renamed something (always a bad idea) and the app hasn’t updated. Maybe we added a new column to a table and some other code has an insert statement without a column list that won’t run. There are any number of database changes that might require a hotfix or rollback and could be considered a failure.

However, some people see an expanded definition. If a service is degraded (slower), is that a failure? Some people think so. If we change code in a database (or indexes) and see performance slow down. In that case, is this a failed deployment? Customers would think so. Developers might not like this idea, at least not without some sort of SLA that might allow for some things to be a little slower. After all, slow is still working, right?

What if I don’t notice a problem? Imagine I add a new table/column, and the app starts accepting data and storing it. What if we are supposed to use this data downstream, and we don’t notice it is being aggregated incorrectly by a process until many days later. Perhaps we’ve performed some manipulation or calculation on our data and the result isn’t what we wanted. It might not be incorrect, but maybe it’s ignoring NULLs when we want NULLs treated as 0s.

Is that a failure? If I deploy today and Bob or Sue notices next week that the data isn’t correct, that’s a failure. I don’t know I’d count downtime from today until next week, but from when Bob/Sue files a ticket, the clock starts on calculating the MTTR (mean time to recovery).

I don’t often see database deployments failing from the “will it compile on the production server” standpoint. Most code gets tested on at least one other system, and with any sort of process, we catch those simple errors. More often than not, we find performance slowdowns or misunderstood requirements/specifications. In those cases, some of you might consider this a failure and some may not. I suppose it depends on whether these issues get triaged as important enough to fix.

While I might have a wide definition of deployment failures for most coding problems, I don’t for a performance slowdown. Far too few people really pay attention to code performance and are happy to let bad code live in their production systems for years.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Posted in Editorial | Tagged , | Comments Off on What is a Failed Deployment?

Republish: What Keeps You Employed?

I’m traveling this week to various customer sites. This is a hectic week, with trips to Norfolk (VA), Nashville, and St Louis. Three customers in three days. This is  a rough week, with meetings during the day, flying in the late afternoon/evening to a new city, and a repeat again.

As a result, you get What Keeps You Employed? to reread.

Posted in Editorial | Tagged | Comments Off on Republish: What Keeps You Employed?

Modifying a Trigger to Capture More Info: #SQLNewBlogger

I had someone ask me about using triggers to detect changes in their tables. This is a second post looking at triggers, in this case, modifying my trigger to detect more changes and using that information.

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

The Setup

We’re using the same table from the last post.  This is the dbo.Customer table with a PK and 5 other fields.

In this case we want to track the changes to an email and capture what those changes are. In other words, if I update the email from ‘sjones@sqlservercentral.com’ to ‘steve.jones@red-gate.com’, I want to capture

To do this, let’s modify our trigger. We can still test if the field is updated with UPDATE(). We did this in the last post.

This will let us know that the column changes by returning a boolean. If this is true, we want to insert the new values into our logger. We also want to capture the old value. These values are stored in the inserted and deleted tables, which are available in a trigger. I’ll use a join between these on the PK to get the same data from both.

I’m also using a query with these tables because more than one row can be updated and we want to capture all the changes.

Here is my new trigger, with the OR ALTER added to the code.

CREATE OR ALTER TRIGGER Customer_tru ON dbo.Customer FOR UPDATE
AS
BEGIN
     IF UPDATE(CustomerName)
         INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerName changed')
     IF UPDATE(AddressKey)
         INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.AddressKey changed')
     IF UPDATE(CustomerStatus)
         INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerStatus changed')
     IF UPDATE(CustomerContact)
         INSERT dbo.logger (logdate, logmsg) VALUES (DEFAULT, 'dbo.Customer.CustomerContact changed')
     IF UPDATE(ContactEmail)
     BEGIN
         INSERT dbo.logger (logdate, logmsg) 
         SELECT GETDATE(), 'ContactEmail updated from ' + d.ContactEmail + ' to ' + i.ContactEmail 
          FROM inserted i
          INNER JOIN Deleted d ON i.CustomerID = d.CustomerID
     END
END

This is mostly the same code, but now I’ve changed the last conditional test. If the email is updated, I want to query the inserted and deleted tables and create a log message. This is the type of thing I’ve seen often in systems, and while it’s not a great pattern, it does let me capture some information. There are some problems with this, I’ll discuss in the next post.

We can see below that when I run this code, I get the updated captured and logged.

2025-05_0230

This post has introduced a few new things, the inserted and deleted tables, which I didn’t discuss in the last post. However, they are useful when you want to capture information affected in triggers, which can be more than one row. Using these tables helps you set up triggers that handle multiple changes.

There are problems with this trigger, mainly with NULLs, potential performance, and architecture in what is captured, but we’ll address those in the future.

SQL New Blogger

This post looks at enhancing a previous post and providing more information. You (hopefully) learn from your work, from both feedback  and experiments, and you should modify your thinking and work. This shows how I’ve adapted something I did previously, which is a skill we all need.

This was a 20-30 minute post for me. You could likely do it in a similar amount of time.

Posted in Blog | Tagged , , | Comments Off on Modifying a Trigger to Capture More Info: #SQLNewBlogger

T-SQL Tuesday #187–Solving Problems

This month we have a great invite from Joe Fleming, a first time host of T-SQL Tuesday. Joe reached out when I requested some hosts and I’m glad he did. He’s got a great challenge for people and I’ll answer in two ways, for work and non-work.

I manage the T-SQL Tuesday site and I’m always looking for hosts, so if you want to host one month from your blog, send me a note on Twitter, Blue Sky, or at SQL Server Central.

Troubleshooting SQL Server

In my career, I’ve had all sorts of issues come up. Here’s a description of one of the stranger issues. This isn’t the exact issue, but it was similar to this.

A user reported they couldn’t log into the server. They were logged on earlier, but can’t connect now through an application. What could be the problem?

When looking at this type of issue, I usually think there is some sort of network issue here, or perhaps a service issue. My  thoughts are:

  • Can I connect? Or can others? (trying to determine if it’s this user)
  • Can this user connect with another app that might give an error message?
  • Is the server instance up? (check the basics and isolate if this is networking)
  • Has this login had a change to permissions/password/etc. Perhaps an AD change.
  • Has something else changed on the server?
  • Has something changed on this user’s machine?

At some point, I’ll isolate where the issue is and determine how to fix it. In this case, I managed to determine the user was a DBA that was playing with a logon trigger and locked themselves out.

The UTV Won’t Move

On the ranch we have to learn to handle lots of things ourselves. YouTube has been a boon, but it’s also a bit of common sense and problem solving that we need to get things done. It can be hard to get people out to the ranch, especially for small things.

Sometimes big things.

Awhile back I came back home from a trip to find the spare UTV we have in one of the fields. I asked what had happened and a kid said that it died.

That’s not a good description, so I asked how it died. What was going on? what was happening? This kid said they’d stopped the vehicle to do something and put it park. When they went to shift into drive, it wouldn’t move.

Did the engine run?

Yes, but the UTV didn’t move.

Now that I had a better story, I could debug further. I knew that it ran and went out to start it myself. Sure enough, it starts and the shifter didn’t do anything, but it felt loose.

Sometimes hands-on helps. I knew immediately a cable had broken.

First thing, get this out of the field. I knew there was a way to shift it without the cable, I just had to figure out. (YouTube to the rescue). Once I did that, it’s research to figure out how hard this is to replace and where I can get the part.

I logically move through the steps of how can I practically get things done.

From here, I saw someone talk about this on YT and show me this isn’t hard. I learned how to shift into Drive with a pair of vice grips, got it up to the house, and I found the part online. I ordered it, send the YT link to the kids, and tasked them with fixing the cable.

They did, after some problem solving from me.

Posted in Blog | Tagged , , | Comments Off on T-SQL Tuesday #187–Solving Problems