Voice of the DBA

Tag: syndicated

  • Redis University

    As if I don’t have enough to do, I started a data structures class at Redis University. Someone recommended this to me as another way a company has structured learning, so I decided to give it a try. There were some things they liked that we might use as Redgate University.

    I decided to start with the 101 course, since I have a rough idea of what Redis is, but now how it works. The class has a number of videos, with transcripts alongside that teach various concepts. One thing I like is that if I get lost, I can glance at the transcript, click it, and the video jumps back to that point. That’s handy, as life sometimes distracts me.

    The quizzes every video or two are nice, but I wish there was a bit more chance to practice things in the middle. I could certainly grab a Redis VM or container, but I have no real idea of how to access it, and I’d have another project. I would actually suggest some “launch lab” points after each set of videos with some practice tutorials or exercises.

    The class has two weeks, with a few chapters, each of which is about 6-7 videos. I don’t know about you, but that’s a lot in a week. If I could abandon some other things for an hour a night, maybe, but that can be tough to do consecutively. I’d really rather this was a 4 week class, with each of the items listed as a set of learning and practice for one week.

    Overall I learned some interesting things about how Redis works. With the power and speed of this memory based key-value system, I can see why some of our customers are using it. If you are thinking about it, or your app uses it and you want to know more, maybe check out their university.

  • Kubernetes Pipelines

    This post continues looking at some of the Kubernetes concepts I’ve been learning with the 50 days of Kubernetes (K8s). Specifically in this post, I’m writing and thinking about the flow of containers from code to a Kubernetes cluster.

    The video talks about pipelines, which is the way in which you get code running in a Kubernetes cluster. The start looks at the idea that we don’t want arbitrary containers running in the cluster, so we limit access to the cluster to somewhere like a container registry, like Azure Container Registry. This ensures that only those containers loaded into your space get onto the cluster. No pulling from Docker Hub or some arbitrary location.

    However, then we worry about developers loading containers into the registry that haven’t been checked. So, we don’t give them permission, we give it to some CI/CD pipeline, which means any code written by a developer that builds a container has to be checked in.

    Now, we have a pipeline flow that we can control. As Brendon mentioned, this isn’t simple, and really, I think you have a lot of work here before you want to trust things running on a production Kubernetes cluster. Really now you need these in your CI pipeline:

    • Unit tests
    • Vulnerability scanning
    • credential scanning ( I see this from MS for Github repos)
    • every other type of scanning you can do

    In this way, you ensure some quality level in your containers as far as security goes and code working. The big win here, at least for me, is that you can continue to raise the bar by adding more tests and scans, something that’s hard to do for developers without automation.

    Not a lot new here, but certainly the idea of putting access and control into the CI/CD flow makes sense.

  • Investigating Azure Databricks

    At SQL Saturday #884 – Pensacola, I dropped into Rodney Ladrum’s session on Azure Databricks (ADB) and the Traditional DBA. I had heard a bit about Databricks, and read a little, but I didn’t really know much more than a rough overview. I’ve heard quite a bit from Microsoft about running Databricks notebooks, but I didn’t necessarily know what that meant.

    I was hoping to learn a bit, and I did. This wasn’t really a Databricks session, but a look at how you might need to manage and work with Databricks as a DBA. This assumes that there are data scientists or other data engineers that need to work with and process data, choosing to do so with R, Python, or Scala, but wanting to use Databricks notebooks.

    I need to learn more, and stumbled on 30 days of Databricks. I can’t get to that right now, but I added a subscription to remind me to come back. Between Rodney’s session showing me the basics of working with this in Azure and the intro video, I know a tiny bit about the technology.

    Databricks is a company and a way of managing and running Spark analysis. Apache Spark is an open source project that implements an analytics engine. However, it’s complex to run, so Datarbricks (the platform) is used to make this easier. Azure Databricks is an implementation of what Databricks sells, under license I assume.

    I know a little more, as there are notebooks that can be executed under the Databricks engine, and the code in here can be bash, R, Python, Scala, probably something else.

    That’s my start. I’ll learn more over time. If I find time.

  • Learning more Kubernetes

    I’ve been slowly working my way through the 50 days of Kubernetes (K8s). As you might have guessed if you remember my first post, this has been more than 30 days. Life and work get in the way, but I’m working through the series of posts and videos.

    A couple of interesting things I’ve watched lately:

    Serverless Kubernetes and Serverless on Kubernetes – I worry about the idea of containers needing to spin up to meet serverless workloads, but maybe not. It’s a few ms for these containers to spin up, especially for small functional environments like Python or .NET. I suppose if there are issues, you can schedule more pods to be available or more replica sets to spread the load. This makes sense to me, and containers are often a perfect use for this on Kubernetes since you want to have lots of small items, but one large endpoint for people to hit things. I suspect this is how the FaaS implementations work on Azure and AWS.

    This video also talks about the need for a virtual kublet to allow the API to get things ready, without having underlying VMs, a prerequisite for scheduling items on the cloud. This is the serverless kubernetes concept. I’m not sure I completely understand this, but I get the idea here. We don’t have hardware provisioned, we’re running Kubernetes, and we want to push some of our load into the cloud. Since we don’t have a node assigned to us, we want to schedule on a virtual node that the cloud provider will actually spin up when we hit it.

    Fascinating and not something I’d have thought of before I saw this video.

    How the Kubernetes Scheduler Works –  This is interesting. Scheduling workloads and pods in different places is important. Flexibility while meeting demands matters. The idea of hard (required) and soft (optional or preferences) constraints for choosing where to run pods is fascinating. Another video by Brendon Burns. Most of us might care about these items since we may want some spreading assurances for instances or minimum resource requirements for our instances.

    For example, we might have a hard constraint that our pod (container) needs 128GB of RAM. This might limit the nodes that we want to run this pod on, and the scheduler takes this hard constraint into its decision making process.

    For soft constraints, we might prefer that a reporting instance not run on the same node as a OLTP instance, but if there are no nodes available, perhaps we’d live with this. That’s a soft constraint, and the scheduler tries to honor these, but it isn’t bound to prevent scheduling on those nodes.