Voice of the DBA

Category: Editorial

  • Less Junior Staff

    As I’ve been working with some AI (Artificial Intelligence) technologies, what I’ve often found is that they produce junior-level code. The code I’d expect from someone early in their career or inexperienced in a particular area. That is code that likely works, but isn’t efficient or clean or perhaps incomplete in some way.

    I’m sure AI technologies will improve, and we’ll be able to train them better for our environment. Just like we train junior developers to be better. However, what does that mean for junior people across the next decade? I ran across an interesting post on the death of the junior developer, which speculates we might have a problem as an industry.

    The post references an article from Gene Kim, where a law firm sees a similar problem with their junior people, who are associates. That position might be equivalent to the junior developer in software. Someone with more experience and knowledge often reviews work and helps shape it, even though the junior person does the work. With AI, however, we might not need the junior person. Instead, the AI produces work the senior person has to review. Finding issues with associate work is a lot like finding hallucination problems in AI responses.

    The same could be said of coding. There is plenty of poorly written code, but if senior people become good at writing prompts and getting the same code back that a junior developer would write, then how many junior people do we need? Arguably less, though you might still need a few. Or you might think that you need all the junior people and you’ll get 10x more work done, clearing your backlog. Certainly, I know most developers, DBAs, and other IT people have a large backlog of problems.

    However, the problem with junior people using LLM (large language model) AIs and getting more done is that they might generate a lot more bad code, so much that your senior people can’t find the time to review the code and you end up with systems that contain even more technical debt than you have today. Perhaps we even find systems that don’t perform well enough for regular use or create constant issues that your developers try to fix with AI, which might not work. I can certainly see things deteriorating rapidly.

    There’s a great quote in the Gene Kim piece: “I believe this furthers the case that AI helps the experienced people far more than inexperienced people — the seniors more than the juniors.”

    I’m starting to think that might be the case. Senior people are going to become very productive, and very valuable. Junior people are going to struggle, and while they’ll get work done, the quality will vary. Maybe that’s good, or maybe we will start to see a rapid divergence of not only productivity but salaries. If you can hire a senior person to produce better code at the same rate as your 5 junior people, maybe you’ll want to pay that senior person $200k a year and reduce junior rates to $45k a year.

    I don’t know that we’ll see rapid changes, as many organizations are slow to alter the way they hire, code, or structure their staff. However, as there is success by others, especially when it’s touted in places like the ETLS, I can see other managers being influenced. That will filter over time to those who hire to pick the productive, senior-level people who can showcase some code skills in an interview. Craft a prompt to solve a problem, get some code back, refine it, explain where and why you’d change your prompt or use parts of the code, and we might see the AI-capable people getting hired quickly, and for fantastic compensation.

    I don’t know that I think this reduces a lot of junior staff, mostly because of organizational inertia, but I do think that learning to be better at your craft and learning to use AI is likely to increase your future earnings.

    Steve Jones

  • Your Computer Science Education

    I know a lot of people in this business do not have computer science degrees. While some do, I suspect it’s a minority. I’m certainly curious, so if you want to share your education experience in a comment, tell me if you have a degree and what the focus was, as well as answer a few other questions.

    At DevOps Days in Minneapolis recently, professors Fox and Sen from Macalester College talked about their computer science curriculum for growing the next generation of professionals. Along the way, they also asked the audience these questions:

    • what were you required to learn?
    • what courses were key?
    • what topics were the focus?
    • what was the teaching style?
    • what was missing?

    While these are good questions for any curriculum, these are interesting points to reflect on for any sort of learning. If you learned about technology in the military, on the job, or by yourself, what did others (or you) think was important and required? What did you feel was left out of your learning?

    Of my small group of 4 that chatted about this, only one of us had a CS degree. I started in CS, but I actually have an economics degree. I switched to business, sensing more opportunity in the 80s there. However, I’ve continued to learn, even taking some classes post-graduation, that helped me learn more about computing.

    It was an interesting look at a modern CS major, with lots of comments from the audience. The professors left us with their questions about how to look to the future, address AI, and even if they should teach an operating systems course, something both feel is missing from Computer Science at their college.

    I think computer science is important for the world to advance how we build systems, but a lot of the deep theory on topics isn’t something that many of us need to learn. I would like to see a software engineering major come about that emphasizes more of the knowledge we’ve learned about building software, with emphasis on version control techniques, software design architectures, and process flows. While DevOps has been amazing, a lot of that knowledge could be used to better teach people how to build software in different ways. Distributed systems, database theory, performance measurement, and even different ways of managing code are things that need deep treatment, not just a module in another course.

    Let me know today what you think of your education in computers.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • A Lack of Architecture and Planning

    A few weeks ago, I was sitting in the audience, waiting for my turn to speak at DevOps Days in Minneapolis. Just before me, Xe Iaso delivered a funny and thought-provoking talk on building a social network on a whiteboard. It was very well done and had me feeling nervous about following that session.

    The talk is a bit of a satirical look at an interview Xe had for a company that tried to get them to derive an architecture for a large distributed system. It was interesting to hear Xe note that often we have architecture diagrams of what we’d like to have, but never an explanation of how we implement a large system, especially one that has to grow as our workload grows.

    This talk was a nice analogy of how often we get into situations where many of us can’t believe our system was structured this way. We often wish we could completely redesign things from scratch, and we’d do it better. Why didn’t the previous engineers think things through?

    Watch the talk. It shows how a lot of software is built. We build what we think we need, but when we get overwhelmed, or often when we get unexpected pressures from others, we make decisions that seem to be the best ones in the moment. However, a few years later, with a few of these decisions behind us, we realize that each of those choices was too short-sighted. We have a mess of software the seems cobbled together rather than well-engineered.

    I don’t believe we can engineer everything well from the beginning. I also don’t believe in early optimization, mostly because I think we are asked to build a lot of things that are never (or lightly) used. Who uses the paintbrush in Word or the FACTDOUBLE() function in Excel? However, I do believe that we ought to write code that performs well the first time, following patterns we (should) know well. We certainly should assume that whatever code we write will see at least 10X more data in production than in dev/test, so prepare for processing more data.

    I’m sure many of you know of a codebase and system that doesn’t run well. I hope most of you have another application that does run well and is fun to work on. Hopefully, the goal of your organization is to turn out more software like the latter than the former.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Analyzing Breached Data

    A few of you out there might be data scientists who profile data regularly. Probably a fair number of you do import/export work and learn to check data values, perhaps with counts, distincts, or other aggregates. I don’t know if the performance tuners out there look at the skew of data or the details of what is in a query that needs improvement. However, all of you are likely familiar with data and trying to query it for some type of meaning.

    One of the largest data breaches occurred with National Public Data. Troy Hunt analyzed the breach as a part of his work with haveIbeenpwned. The piece is an interesting analysis of the data, trying to determine both it’s legitimacy as well as what is actually included in the breach. It’s a fascinating read and I encourage you to look at it not just from the data analysis side, but also to be aware of what data about you is being aggregated and sold by companies.

    The read is interesting as it is a bit of a detective story, digging through data in a folder, which is something I’ve had to do. I’ve had people in previous jobs just dump a bunch of data on me and ask me to load it into a database. Or a table. Often without them knowing what type of data it is, what formats, do files relate to each other? Are there multiple tables worth of data in a file? All questions I’ve had to ask myself (and answer), and similar to what Troy did to analyze the breach.

    Data is very important to many of us, in different ways, but I’m often amazed at how few people actually understand how to organize data and ensure others can track the metadata about their data (what their data represents). I’m guessing this is why every person that gets an extract of data to load into Excel formats it in different ways.

    In many cases, people want the ability to query data, but they prefer to just focus on one table that contains a lot of information. They don’t want to know how to “join” data together. I think this might be the reason we see so many views in databases, and why we have views built on views. Each new client of the database needs their own view structure.

    The world of data is a mess, even inside an organization. Once we start moving data between organizations, it’s truly a mess. We might bemoan all the inefficiencies and work we do to move, change, and re-load data as custom, human ETL machines, but there is one great thing about this tangled web. It provides for steady, secure jobs for many of us with no end of work in sight.

    Steve Jones