Practical Problems with CI

I agree. Continuous Integration has issues in the real world, at least it probably does in many companies. The more tests you have, the more likely you refactoring or changes will break something. The more things break, the more people will look to leave them alone, deferring the "fix" into the future.

This defeats the purpose of CI. The whole idea is to keep your software in releasable condition. That way you can push it out if you need to.

How do you deal with the constant build breaks? The first thing I’d do is see if I mad a mistake and can fix my code. That should be the first reaction.

The next thing is stop for a moment and see what tests are failing. It’s entirely possible that the tests aren’t valid or aren’t needed. Maintenance of tests is an issue. If this is a test that regularly breaks, I might call a quick meeting and see if we should remove it.

The last thing is that I’d rollback my code and then go back to work on it. Perhaps there’s another way to solve the issue and not break the build.

This isn’t magic. It’s also not easy. However if you are keeping up with your CI process and it’s healthy, I believe it will pay off over time with better software.

Unknown's avatar

About way0utwest

Editor, SQLServerCentral
This entry was posted in Blog and tagged , . Bookmark the permalink.

2 Responses to Practical Problems with CI

  1. Dave Wentzel's avatar Dave Wentzel says:

    Here’s one way to avoid this…stop having developers write hundreds of tests. That sounds counterproductive and stupid but it works. Most tests aren’t good tests and are testing commutative and idempotent stuff. This is especially prevalent in TSQLT. I’ve seen stored procs with tests like “Ensure that if I pass a BIGINT to my INT parameter that I get an error.” Really? That’s pointless. All you are testing is that the behavior of the TSQL language works as you expect it to work. It tells nothing about the stored proc and it’s logic. Not every line of code needs a test. IMHO. Almost everyone disagrees with that however, which is why you have written your post.

    Like

    • way0utwest's avatar way0utwest says:

      Dave, completely agree, but writing tests is a journey. It’s like software and you learn what works and doesn’t work for your app over time. We have certainly learned at Red Gate that some tests we think were good early on, weren’t good later.

      The other thing is that patterns of tests, like “checking the bigint”, should be automated. You shouldn’t write those. Use an automated process and a naming standard for the test. That way if you decide they aren’t good, then whack the whole series of tests and remove that automated section.

      Like

Comments are closed.