Voice of the DBA

Tag: containers

  • Copy Files to a Docker Container

    One of the things that’s handy when working with containers is being able to move files in and out of the container. Certainly you can do this from a network share or other similar mechanism, but what about a straight copy?

    Docker has a command with the cp parameter to do this.

    docker cp WideWorldImporters.bak SQL17:/var/opt/mssql/data/WideWorldImporteres.bak

    The structure is the from:to, separated by a space as two parameters. If you’re going to the host, then just the path and filename, or just name if the current folder) for the file. For the container, you need the container ID or name with a colon (:) and then the relative path.

    A quick way to begin changing and customizing your containers by moving data in and out as needed.

  • More Kubernetes Concepts

    This post continues looking at my process of learning more about Kubernetes. I’ve been working through the 50 days of Kubernetes (K8s). Now that I had a break from some travel, I went through a few videos in a short period of days. This post combines a few looks at Scenarios, storage, and stateful sets

    Scenarios

    This was a good look at how one of the founders see the Kubernetes being used. The initial idea was microservices, where many services each live in a container and communicate. Managing lots of containers, and lots of instances of each, is where Kubernetes excels.

    The second scenario is the diurnal or cyclical scenario where a load goes up and down each day/night, week/weekend, etc. Having Kubernetes manage the increase and decrease of workload is valuable, and if you’re in the cloud, then you’re not investing heavily in hardware that is unused.

    The third is with complex workflows, where one thing, then another, then another happens. I’m less sold here, but I do see that spin up/spin down, and having these workloads/processes in containers rather than on certain machines makes sense.

    Lastly, AI/ML places where all parts of your flow can access the Kubernetes API and spin things up and down or move workloads to better hardware, like GPU/FPU concentrated systems, that can help keep things moving.

    Good to see the different ways that this might be useful, though I was wondering if I’d see anything database related. I didn’t.

    Volumes and Storage

    The important thing for database workloads is storage. This is a constant question when I talk containers. What about the disks and storage and how does this work.

    This is a basic video, just noting that volumes are objects in K8s. They are attached to pods, but can be mounted (or not) inside containers in the pod. There is temp storage (/emptydir, need to look this one up), but we can also abstract away other storage with Persistent Volume Claims. These are the aspirational storage requirements that the container/pod needs.

    I’m not completely clear on these, and the explanation is a little light, so I’ll have to learn more later.

    Stateful Applications

    The last set of learning for this post. More relevant because SQL is a stateful application. Or so I thought.

    This really looks at stateful sets in Kubernetes, which are the ways in which we want orderly copies in a replica set. This means hostnames and DNS entries in stateful sets are ordered, and 0 based indexed. If I have a host of SalesSQL with 3 sets, I’ll get

    • SalesSQL-0
    • SalesSQL-1
    • SalesSQL-2

    Likewise, I may have salessql.mydomain.com, but I’ll also have the –0, –1, –2 DNS entries as well. The Kubernetes platform also will scale up and down from the end, adding or removing the next ordered element container.

    There can also be volumes associated here, or not, as your stateful application may have the need to use the same volume (like a clustered SQL Server) or separate volumes, like AGs. In any case, it’s a light explanation that I need to know more of.

    One thing I didn’t really like is that the data stores were associated as “cloud native storage applications”, not data stored or data platforms. Application developers I think sometimes forget there’s more to data than storage.

  • Kubernetes Pipelines

    This post continues looking at some of the Kubernetes concepts I’ve been learning with the 50 days of Kubernetes (K8s). Specifically in this post, I’m writing and thinking about the flow of containers from code to a Kubernetes cluster.

    The video talks about pipelines, which is the way in which you get code running in a Kubernetes cluster. The start looks at the idea that we don’t want arbitrary containers running in the cluster, so we limit access to the cluster to somewhere like a container registry, like Azure Container Registry. This ensures that only those containers loaded into your space get onto the cluster. No pulling from Docker Hub or some arbitrary location.

    However, then we worry about developers loading containers into the registry that haven’t been checked. So, we don’t give them permission, we give it to some CI/CD pipeline, which means any code written by a developer that builds a container has to be checked in.

    Now, we have a pipeline flow that we can control. As Brendon mentioned, this isn’t simple, and really, I think you have a lot of work here before you want to trust things running on a production Kubernetes cluster. Really now you need these in your CI pipeline:

    • Unit tests
    • Vulnerability scanning
    • credential scanning ( I see this from MS for Github repos)
    • every other type of scanning you can do

    In this way, you ensure some quality level in your containers as far as security goes and code working. The big win here, at least for me, is that you can continue to raise the bar by adding more tests and scans, something that’s hard to do for developers without automation.

    Not a lot new here, but certainly the idea of putting access and control into the CI/CD flow makes sense.

  • Are Windows Containers a Good Choice?

    I like containers. I think they’re the future of the SQL Server data platform. I also think this is likely on Linux, with Linux versions of SQL Server. You might disagree, and that’s fine, but that is where I see this moving over time.

    Why? I ran across a great post from Joey D’Antoni of DCAC that outlines one reason: Kubernetes. This is the way we will start to orchestrate SQL Server instances in the future, whether disparate or scaled-out, and this is really Linux technology. If you go Linux, with Kubernetes and containers, you’ll be able to easily move your systems as needed from your data center to developer machines (perhaps with SQL Provision from Redgate), to the Azure cloud, or over to the AWS cloud, a partner Kubernetes install, etc. It’s flexible.

    Windows Kubernetes will be less flexible. Inherently because less people use it, while more and more people are using, supporting and deploying Kubernetes on Linux.

    This won’t happen soon, as I think it’s 5-10 years before a good majority of SQL Server moves to 2019+ and containers become more mainstream for app and data work, but this shift is starting.

    This also doesn’t mean Windows goes away. I really like Windows. I’m comfortable with it, and I plan on having a Windows laptop for the forseeable future. What’s more, I can run Linux containers on Windows, thanks to the WSL. Even Microsoft sees containers with Linux as a future on Windows.

    Like Joey, I don’t want to dismiss or denigrate any work for Windows containers. I think it’s a good idea, and certainly some people will want to run their Windows apps in Windows containers. I just don’t think it’s the future for the data platform.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.