Voice of the DBA

Category: Editorial

  • The Ease of Containers

    I first heard about containers in 2012. I went to a conference on software delivery and talked with a gentleman that was using them in his Java application, mostly because the Java app had version issues and would leak memory. He could run hundreds of instances of the app on each server and handle a load, allowing each to be built, run, and then get destroyed in minutes.

    Since then, I’ve come to appreciate containers as a way to isolate workloads of specific applications. They became very popular a few years ago, and many developers and companies were looking at them. Since then, the hype has cooled, but I find that the used of containers is still growing, and certainly my use is growing as well.

    In the last year, I’ve experimented with a few different pieces of software in containers. Rather than try to install a number of dependencies on my system, I’ve downloaded a container, mapped a volume, and been up and running much quicker than I would have been otherwise. My experiments with Jekyll were one of the recent examples.

    I also got a demo from someone at Redgate recently that included a container file, allowing me to use VS Code, but develop and run in a container, avoiding dependencies on my machine. I was up and running with this software in minutes. I was truly impressed with the ease of getting started, and the simplicity for me to actually run and debug code.

    I still don’t see a lot of database container work in production, or even in development. We get requests from customers, but often it’s a wish, and they aren’t even sure how they’d get started. While I think database development with containers is fantastic, you do need to have a good dataset available that you can use inside the container to keep resource usage low and make this a viable environment.

    I still expect the future of database development to be in containers, especially as we start to have more and more applications connecting to multiple data sources. That’s going to take some time, but I still think learning about, and experimenting with, containers is a great skill for you to have. It’s also an impressive topic on which to have a few stories ready for your next interview.

    Steve Jones

  • Distributed SQL Databases

    One of the challenges that SQL Server, and many RDBMSes, face is scaling out. While SQL Server can grow to handle a large workload on one piece of hardware, eventually there reaches a point when a single machine cannot handle the workload.

    Microsoft is trying to handle this with some new tricks. We have the Hyperscale editions of Azure SQL Database and Big Data Clusters, on premise. I don’t know that either of these will end up providing us with scale out for our OLTP databases, but they are options.

    There are other options. Google has Spanner, which is a distributed SQL database and CockroachDB has attracted some attention. I caught an article that talks about some of the reasons you might look at CockroachDB, which is a cloud system, but one that gives you scale, and perhaps more importantly, lack of lock-in with a cloud vendor.

    I don’t know all the ins and outs of a distributed, scalable SQL database, but I do know that scaling one table, with lots of concurrent access, can be tricky. There are improvements in bandwidth and technology that might allow things to scale well, but I can’t help but think that conflict resolution will be a challenge if many people need to write to the same rows.

    I also know that often we aren’t writing to the same rows, but to the same table. Distributing that is easier, but when we start to look at lots of tables with referential integrity, I wonder how scalable this is. After all, the data needs to get committed and then written to multiple places for this to work well.

    The one advantage in the cloud is throwing hardware at problems like this, often in a more cost effective way than an organization can do in their own data center. Whether this works well remains to be seen, but certainly there are customers and investors that see this as a part of the future.

    Steve Jones

  • Disclosing the How

    Many of us have written a computer program of some sort. Maybe just a script to complete a task, maybe complex software the clients interact with. For all of us, I would guess that we’ve been asked to do something, or we’ve asked ourselves to build a program, and we use that request as a specification for our code.

    When we do this, how many of us get this right the first time? I’m sure some do, and the simpler or narrower the task, the more likely we get it right. However, we also sometimes make mistakes and realize that our specification wasn’t correct or that we interpreted the specification incorrectly. Even when I write code for myself, sometimes I realize I haven’t described when I need clearly enough, even in my own head.

    That happens when I control the entire process. When someone else is my client, there are plenty of ways in which we can make mistakes, especially in decoding the specification. As we use more and more software to make decisions for us and automate our world, I wonder how often we’ve assumed the software works in a way different than it actually does. With machine learning and artificial intelligence programs, I think there is an even greater chance of a misunderstanding.

    Recently I saw that in New York City there is a proposal to disclose how software is used in hiring decisions. Humans are barred from discriminating by law, but what about software? What if there are bias that can come out in the software based on how someone enters data?

    I think there are certainly potential issues in having software evaluate individuals, though I do think software will do a better job than many people, once we’ve worked out the bugs and we know how we want it to work, and then we can compare that to how the algorithm produces results.

    These are tough issues, and difficult projects. I also think it’s important that we analyze the data that is used in testing software, to be sure that it doesn’t contain its own biases. Ultimately I think having public data sets that are evaluated by many individuals is important. I could see a reference set of data and the expected results, or some range of results being required to ensure software meets the goals we want as a society. This, along with the disclosure of how algorithms work will make our use of software better over time.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Republish: Making Guesses

    I’m skiing this morning in Crested Butte and then coming home, so you get a republish: Making Guesses.