Voice of the DBA

Category: Editorial

  • A Kafka Introduction

    I’ve heard of Kafka before. I know it’s an Apache project and you can download or read more at https://kafka.apache.org/. I knew it was a way of moving data around, some sort of ETL tool useful for moving things around. More like a message and queueing system, which is a tool that seems like a great idea, but one that everyone struggles to work with.

    And one that seemed complex. The overview is that Kafka is “a distributed system consisting of servers and clients that communicate via a high-performance TCP network protocol. It can be deployed on bare-metal hardware, virtual machines, and containers in on-premise as well as cloud environments.”

    Would I need that or use it? In a lot of my database work, I’m not sure that it would easily fit into most of the OLTP applications or data warehouse systems. Maybe. Hard to tell. Their description of event streaming and the definition of an event make it seem this is a catch-all system for moving log data around. One that be so open-ended that it ends up requiring a lot of configuration for “my” system.

    Here’s their definition of an event: An event records the fact that “something happened” in the world or in your business. It is also called record or message in the documentation. When you read or write data to Kafka, you do this in the form of events. Conceptually, an event has a key, value, timestamp, and optional metadata headers.

    Recently I watched a Kafka presentation at THAT Conference (which was a fantastic event). In the talk, this sentence caught my eye: “[Kafka is] a pipe to move data from A to B, C, D”. I’ve certainly had that need, and sometimes configuring lots of pipes is work. If you’ve ever worked with replication and the publisher/subscriber model you likely get a twitch in your eye if a ticket is opened to configure a new subscriber. Not because the configuration is hard, but because the ongoing admin can be a pain.

    The talk dives into some of the complexity of designing and implementing a Kafka system. For developers that might write to the stream or read from it, things seem simple. For admins and architects, less so, and I can’t help what happens when a reader goes down. I have nightmares of replication subscribers being down and transaction logs not being reused.

    Kafka doesn’t seem as complex as I thought before, but it certainly doesn’t seem simple or easy. Kafka is not a panacea for moving data around, but it is a well-understood and widely used technology. Those things mean more to me now that I find myself considering the challenges of maintaining a system over time and hiring staff who understand it. It’s something I’d consider using in the future, and maybe something I’d like to experiment with a bit more and learn how it works at a more practical level.

    If you use it, or know more, I’d be interested in how well Kafka has worked for you, either as a developer or admin.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Less Junior Staff

    As I’ve been working with some AI (Artificial Intelligence) technologies, what I’ve often found is that they produce junior-level code. The code I’d expect from someone early in their career or inexperienced in a particular area. That is code that likely works, but isn’t efficient or clean or perhaps incomplete in some way.

    I’m sure AI technologies will improve, and we’ll be able to train them better for our environment. Just like we train junior developers to be better. However, what does that mean for junior people across the next decade? I ran across an interesting post on the death of the junior developer, which speculates we might have a problem as an industry.

    The post references an article from Gene Kim, where a law firm sees a similar problem with their junior people, who are associates. That position might be equivalent to the junior developer in software. Someone with more experience and knowledge often reviews work and helps shape it, even though the junior person does the work. With AI, however, we might not need the junior person. Instead, the AI produces work the senior person has to review. Finding issues with associate work is a lot like finding hallucination problems in AI responses.

    The same could be said of coding. There is plenty of poorly written code, but if senior people become good at writing prompts and getting the same code back that a junior developer would write, then how many junior people do we need? Arguably less, though you might still need a few. Or you might think that you need all the junior people and you’ll get 10x more work done, clearing your backlog. Certainly, I know most developers, DBAs, and other IT people have a large backlog of problems.

    However, the problem with junior people using LLM (large language model) AIs and getting more done is that they might generate a lot more bad code, so much that your senior people can’t find the time to review the code and you end up with systems that contain even more technical debt than you have today. Perhaps we even find systems that don’t perform well enough for regular use or create constant issues that your developers try to fix with AI, which might not work. I can certainly see things deteriorating rapidly.

    There’s a great quote in the Gene Kim piece: “I believe this furthers the case that AI helps the experienced people far more than inexperienced people — the seniors more than the juniors.”

    I’m starting to think that might be the case. Senior people are going to become very productive, and very valuable. Junior people are going to struggle, and while they’ll get work done, the quality will vary. Maybe that’s good, or maybe we will start to see a rapid divergence of not only productivity but salaries. If you can hire a senior person to produce better code at the same rate as your 5 junior people, maybe you’ll want to pay that senior person $200k a year and reduce junior rates to $45k a year.

    I don’t know that we’ll see rapid changes, as many organizations are slow to alter the way they hire, code, or structure their staff. However, as there is success by others, especially when it’s touted in places like the ETLS, I can see other managers being influenced. That will filter over time to those who hire to pick the productive, senior-level people who can showcase some code skills in an interview. Craft a prompt to solve a problem, get some code back, refine it, explain where and why you’d change your prompt or use parts of the code, and we might see the AI-capable people getting hired quickly, and for fantastic compensation.

    I don’t know that I think this reduces a lot of junior staff, mostly because of organizational inertia, but I do think that learning to be better at your craft and learning to use AI is likely to increase your future earnings.

    Steve Jones

  • Your Computer Science Education

    I know a lot of people in this business do not have computer science degrees. While some do, I suspect it’s a minority. I’m certainly curious, so if you want to share your education experience in a comment, tell me if you have a degree and what the focus was, as well as answer a few other questions.

    At DevOps Days in Minneapolis recently, professors Fox and Sen from Macalester College talked about their computer science curriculum for growing the next generation of professionals. Along the way, they also asked the audience these questions:

    • what were you required to learn?
    • what courses were key?
    • what topics were the focus?
    • what was the teaching style?
    • what was missing?

    While these are good questions for any curriculum, these are interesting points to reflect on for any sort of learning. If you learned about technology in the military, on the job, or by yourself, what did others (or you) think was important and required? What did you feel was left out of your learning?

    Of my small group of 4 that chatted about this, only one of us had a CS degree. I started in CS, but I actually have an economics degree. I switched to business, sensing more opportunity in the 80s there. However, I’ve continued to learn, even taking some classes post-graduation, that helped me learn more about computing.

    It was an interesting look at a modern CS major, with lots of comments from the audience. The professors left us with their questions about how to look to the future, address AI, and even if they should teach an operating systems course, something both feel is missing from Computer Science at their college.

    I think computer science is important for the world to advance how we build systems, but a lot of the deep theory on topics isn’t something that many of us need to learn. I would like to see a software engineering major come about that emphasizes more of the knowledge we’ve learned about building software, with emphasis on version control techniques, software design architectures, and process flows. While DevOps has been amazing, a lot of that knowledge could be used to better teach people how to build software in different ways. Distributed systems, database theory, performance measurement, and even different ways of managing code are things that need deep treatment, not just a module in another course.

    Let me know today what you think of your education in computers.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • A Lack of Architecture and Planning

    A few weeks ago, I was sitting in the audience, waiting for my turn to speak at DevOps Days in Minneapolis. Just before me, Xe Iaso delivered a funny and thought-provoking talk on building a social network on a whiteboard. It was very well done and had me feeling nervous about following that session.

    The talk is a bit of a satirical look at an interview Xe had for a company that tried to get them to derive an architecture for a large distributed system. It was interesting to hear Xe note that often we have architecture diagrams of what we’d like to have, but never an explanation of how we implement a large system, especially one that has to grow as our workload grows.

    This talk was a nice analogy of how often we get into situations where many of us can’t believe our system was structured this way. We often wish we could completely redesign things from scratch, and we’d do it better. Why didn’t the previous engineers think things through?

    Watch the talk. It shows how a lot of software is built. We build what we think we need, but when we get overwhelmed, or often when we get unexpected pressures from others, we make decisions that seem to be the best ones in the moment. However, a few years later, with a few of these decisions behind us, we realize that each of those choices was too short-sighted. We have a mess of software the seems cobbled together rather than well-engineered.

    I don’t believe we can engineer everything well from the beginning. I also don’t believe in early optimization, mostly because I think we are asked to build a lot of things that are never (or lightly) used. Who uses the paintbrush in Word or the FACTDOUBLE() function in Excel? However, I do believe that we ought to write code that performs well the first time, following patterns we (should) know well. We certainly should assume that whatever code we write will see at least 10X more data in production than in dev/test, so prepare for processing more data.

    I’m sure many of you know of a codebase and system that doesn’t run well. I hope most of you have another application that does run well and is fun to work on. Hopefully, the goal of your organization is to turn out more software like the latter than the former.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.