Deploying software is hard. Despite decades of experience working to update software smoothly, it seems that many of us tasked with that responsibility find ourselves nervous about making changes. This is especially true for database changes, where we must not only update the schema and code, but we must preserve the data as well. If only we could drop and recreate the database during every deployment.
One of the ways in which you can make changes in a live environment is to have a second system already in place and running. With a second system available, you can stop the secondary system from handling user requests, make changes, test them, and if things are working well, you can have users connect to the updated system while you take the one one offline and perform your updates there.
This is the same process that many people use to patch clustered database servers, though the “rolling upgrades” are usually done for patches from Microsoft. It would be great if we could easily apply this same system for our schema upgrades. Deploy our changes to the passive node, have the application able to connect to this node through some switch, test the changes, and if things are working, switch users over. The problem is that our databases on the passive node aren’t live.
Really the ability to deploy database changes to a live system and perform checks requires a strong architecture in both the database and application to support this. Your code must be able to handle additions to the database objects without breaking. The use of feature flags, turning functionality on and off with switches, allows certain users to test new features without most clients being aware of the changes.
It’s a pattern that more and more software companies are using when they want to deploy changes rapidly to systems for limited numbers of users. However it’s not a design pattern that is widely used. I hope we get more sample applications and reference architectures in the future that will help developers and DBAs code more robust and resilient applications, allowing changes to be deployed live without affecting every user.