Voice of the DBA

Tag: DevOps

  • The Complexity of Modern Systems

    I am a big advocate of version control and testing for your software. The main reason is that modern systems are incredibly complex, and capturing what has been done (and what is happening), along with looking for potential issues, is important. It might be critical for ensuring you have success in building software.

    Part of the reason is this post on software complexity. Now I don’t think the complexity is killing software developers. I do think it adds to developers’ stress, and in poorly managed environments, the complexity makes for a poor job experience for many.  While I do think most people are smart, and they find ways to overcome this, they just don’t enjoy the process.

    This is part of the reason, IMHO, that we are starting to see lots of people move to a new role, often in a new company. With remote work, there are more options, and for people with talent, they can easily leave. My recent job searches seem to show more remote job options and more demand for mid and senior level people. I think some of this is looking to get better talent in your company, and some are because experienced people are tired of poor work environments and poor code.

    Modern software is big, and I don’t think any system of consequence is built by individuals anymore. There are still good projects and applications being built by a person, but once they start to grow, a team of developers is needed. This means that we have to learn how to better work together. We need to adhere to standards. We need to communicate and collaborate both in code as developers and together as humans.

    While someone may understand how the entire system works, it’s unlikely that many of our staff does. If the system is large, it’s unlikely anyone completely understands, and then our ability to function as a team is incredibly important to be sure that we are building something that works well together.

    To me, the complexity of modern systems emphasizes the need for teamwork and collaboration, the thing that many companies adopting DevOps struggle to emphasize.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • A Version Control Primer for Databases

    This is a talk for database people that might not be familiar with version control and how it can be used with database code.

    Abstract:

    With modern DevOps software development, there is a need for the database code to be tightly integrated with application code, and this requires the use of a version control system (VCS).

    This session will examine the basics of version control, the different types of systems, and how we can use them for database code. There is an emphasis on Git, as this is the most popular VCS in use today.

    Level: 100

    Slides: vcs_primer.zip

  • Going Cloud Native

    I am constantly surprised at the migration of workloads to the cloud. It seems that many of the customers and clients that I speak with are moving some of their assets to the cloud. Some are moving the majority of their systems, and they are pushing to do this sooner than later, including database assets.

    I ran across an article that noted that while many companies love the on-demand provisioning of cloud resources, they don’t always get improvement in their software performance. In fact, quite a few might experience worse performance, though often not horribly worse. Instead, what they often get is a bit more unpredictable performance, and sometimes a bigger bill, which makes perfect sense. After all, the cloud is really just someone else’s computer, and the one you’re using might have other people doing the same thing.

    To truly embrace the cloud and get the power of the cloud, you do need to consider being cloud-native, which is often a change in how your software is architected. It is also a change in your entire software lifecycle, meaning not just buying DevOps tools or renaming a team, but also embracing the entire way that DevOps works. Not just fast releases, but experimentation, learning, adapting,  testing, and changing how you build and release software. This is also changing how you manage the teams that build software.

    If you think this is just for the Netflixes and Spotifys of the world, that’s not true. Stripe and Capital One are financial companies completely in the cloud. There are plenty of other examples as well, and the successful ones, in all industries, are embracing a new way of building and operating software. Docusign has had a lot of success, and quite a bit with a lift-and-shift to IaaS resources as they continue to build and rebuild their software to embrace the cloud and adopt more PaaS services. They didn’t move systems and expect everything to just get better. They started a journey that they continue to travel every week. The move is a journey, not a quick move, another item that the article points out and many managers forget.

    That being said, not everyone is moving their systems to the cloud. Even organizations that adopt some cloud services, implementing Salesforce as a CRM, might continue to run plenty of systems on-premises. Lots of companies use the cloud for IaaS, but with the architectures and software that used to run in their own data centers. They just moved their  VMs.

    The cloud can be successful in many ways, but you need to understand how you adopt the technology, what capabilities can help, and how limitations can hinder your software. There is no magic bullet, but there are a lot of opportunities if your organization takes advantage of them.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Choosing State Over Migrations

    I ran across an article about state v migration based development for Snowflake. While I think that this article is a bit slanted because the author is building a tool, it does present some good reasons why people worry about migration-based approaches. This article also references a work from Alex Yates on the two approaches.

    Everything we do in relational databases, and in some other platforms like Snowflake, is a migration script. If you run SQL Compare and get the differences, you are generating a migration script. It might contain lots of changes, but it’s still a migration script. I wish I could write CREATE OR ALTER TABLE scripts, but that’s not a part of the SQL specification (yet).

    The concerns in the article are valid, though some are a bit outdated. I know that I am biased a bit as my employer, Redgate Software, sells tools in this area. However, over the years, we’ve supported and worked on both approaches. We do a lot of research, and I get to provide feedback on the good and bad of how we approach the problem. Ultimately, I think the world is very chaotic and we should be as flexible as possible with developers.

    I find that most developers like the state approach. Even if they want to write code themselves, having an easy way to capture the state of objects and keep track of them is useful. When we see a table difference, we want to see a view of the old table and new table in a CREATE TABLE statement: the state approach.

    For deployments, we always need migration scripts, but we want to build them as appropriate for the situation. Cherry-picking out specific changes is often what we need when urgent hot fixes need to be deployed, or a piece of work isn’t tested. In those cases, we might need to deploy some changes and not others, which often require us to put a subset of changes into a migration script(s). That’s a hassle, and these situations are often where we find humans making mistakes.

    Ultimately I prefer state sometimes and migrations others. If I had to pick only one, it would be migrations, but I’d ensure I had a state view of the schema using a tool like SQL Compare. I want to be able to see history and capture changes. in a human-readable format.

    One approach isn’t better, and both have advantages and disadvantages. We are often under pressure to be more like DevOps software developers, adaptive and agile. We can use either state or migrations to do this. The important thing is that your whole team understands the approach we are using and the potential limitations. They also need to work together to smooth out your database deployments.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.