Author: way0utwest

  • Bringing Database Devops to DevConnections

    Another alliteration post. Maybe I’ll make all my titles tongue twisters to test your talents?

    This is my new favorite session. I’m slated to deliver this at a few different conferences this fall, and hopefully you’ll come see it in Las Vegas at DevConnections on Oct 10-13.

    The talk is called Bringing DevOps to the Database, and this presentation is designed to show how many of the software development processes that we take for granted with .NET, Java, and other front end applications can be applied to the database. I’ll cover how version control, Continuous Integration (CI), and Release Management (RM) can be applied to database changes.

    This has been a focus of mine as an evangelist for Redgate Software, and I’ve found a number of ways that you can adopt common, best practice software engineering in the database development process. I’ll show how Redgate makes this easy, but I cover the process in a tool independent way. You’ll learn what needs to happen. How you get that working is up to you, but if help is needed, Redgate can cover that part.

    If you’re looking for some training, some inspiration, and are a full-stack developer, or multi-technology IT worker, think about coming out to DevConnections. There are a slew of technologies being covered (VS, ASP.NET, C#7, Azure, PowerShell, AWS, SQL Server, Sharepoint, Exchange, Hyper-V and more).

    Register today, use 500SPKR to save $500, and I’ll see you in Las Vegas this October.

  • Encryption Keys Matter

    Perhaps the importance of protecting encryption keys is even greater than we realize. It appears that the NSA and the US Government have been able to read encrypted traffic for some time on the Internet, perhaps for most of the last decade. There was an exploit in Cisco PIX routers, which I’ve used in many companies, that allowed the NSA to gain the encryption keys used by VPN traffic. In fact, it is likely the NSA could actually penetrate the networks on either end of the VPN with those keys.

    Some of you may be against back doors for governments in encryption products, and some of you may be for allowing governments access with legal protections. However, the most disturbing part of this for me is that no organization knew their communications, supposedly secure, were being intercepted and read. The nature of the digital world is that exploits can copy information without the holders of that information being aware of the effort. This is much harder with physical items, where the movement of an object from one person to the next is easily noticed.

    One of the very difficult things with protecting the data in our databases is trying to understand when someone has actually retreived information they shouldn’t. This is much more difficult than just tracking changes to data, which is more straightforward. We have auditing mechanisms that easily track changes to data, though most of us don’t have this set up or configured to catch all changes. In practice, that might be good enough to prevent data quality issues, but it doesn’t necessarily protect data from read disclosure.

    Monitoring what information is accessed is far harder than tracking changes. Do you know if someone in Sales is accessing a single row with customer details or the sensitive information for all customers? Can you tell when a request is legitimate for an application or if there’s an unusual query that might be seeking massive amount of data for download? Those are hard questions, and ones that I think can only be handled by a large amount of activity monitoring along with machine learning assistance to look for patterns in user activity. Other features like Row Level Security can help limit the inadvertent mistakes made by developers or users, but not necessarily prevent a single user, especially a malicious user, from querying information.

    Apart from the activity issues, we should ensure that where it is possible, we should be using encryption to prevent accidental disclosures outside of our applications. I think Always Encrypted has possibilities for the database, but the key management for this, as well as key management for VPNs, disk encryption, and other protection mechanisms, needs both more maturity and the open disclosure to prevent back doors from being included in products. We also need more maturity in our software development that takes the implementation and protection of encryption mechanisms seriously.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.7MB) podcast or subscribe to the feed at iTunes and Mevio .

  • Bringing DevOps to the Database

    Abstract:

    DevOps is a hot topic in today’s software development world. However most of the knowledge and experience with DevOps is based around application software and ignores the database. We will examine how the concepts and principles of DevOps can be applied to database development by looking at both automated comparison analysis as well as migration script management. Automated building, testing, and deployment of database changes will be shown.

    Level: 200 – Not to technical, but you should have some idea of how software development works to get something out of this talk.

    Goals:

    1. Conceptually understand how DevOps principles can easily be applied to the database in either state based or migrations based changes.
    2. Learn how databases can be automatically built and tested in a CI/CD environment
    3. Understand how database releases can be automated, but are different than those deployment/releases made for other software.

    Demos:

    This presentation has a few demos where I show how I stored database code in a VCS, perform an automated build, and use a scripted release process to move code from one database to the next. The demos are pre-built, so I just show how they work. I can do this with:

    • git, VSTS Build and VSTS Release
    • git or SVN, Team City, and Octopus Deploy

    Downloads:

    There are a few versions, as this talk has evolved.

    GroupBy – Bringing DevOps to the Database Groupby.pptx

    Live!360/VSLive – VSLive-Orlando 2016.pptx

    SQL Saturday Cambridge – Bringing DevOps to the Database.pptx

  • Ignoring Database Version Control

    I find myself surprised that so many people resist keeping their code in version control. I think this is less people all the time, but I don’t have good percentages because at this point I think a fair number of people embarassed enough to not answer the questions. However, in quick conversations, I still find people that don’t think a VCS is necessary. I can’t conceive of this for application code, but more and more I think this also makes sense for database code.

    I ran across a blog that asks you to find a way to keep your database code in a VCS. While I’m glad that the author provides some ideas on how you might script this yourself in a pinch, I think that’s silly. There are tools to help, whether paid for ones, like SQL Source Control or free utilities. In either case, it makes sense to work with something that has been tested a bit rather than inventing your own. It seems that running scripts is simple and easy. In practice, these tools add value because of their maturity. Some of the mistakes you’ll make with your own software have already been made, and solved, with many of these frameworks and paid for tools. Don’t spend your time maintaining software scripts when you need to be developing other software.

    I also think the author makes the process sound a bit simpler and easier than it is. In practice, moving between schema versions in a database is typically a one way process, and rollback scripts are of dubious benefit. A discussion of this post on Hacker News shows some of the shortcomings in trying to version database schemas and easily move between versions. In practice many database schema changes can be rolled forward and back, and the corner cases that are very difficult for tools to solve, aren’t items that occur too often. Really, it’s table changes that are hard, and having a good process helps here, but these are usually one way changes.

    However, the reason I think version control is important is that database development is often disconnected between the database and application. Changes might occur in the database long before they are used in code, or they may need to be undone in development, perhaps in a destructive way that isn’t possible in production. The history of changes, and understanding of what needs to be changed, is much easier with a record in a VCS.

    These days there are a variety of ways to capture your schema changes and record them. I would encourage you to investigate some method and give it a try. Even building the habit of manually recording changes by all developers will help you to move faster and faster in your development process without worrying about losing track of what’s happening.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.8MB) podcast or subscribe to the feed at iTunes and Mevio .