Voice of the DBA

Category: Editorial

  • Where We Need Better AI Disclosure and Responsibility

    There are a lot of contract and gig jobs in the world today. It used to be this type of work was widely spread throughout programming and technology, but these days many types of jobs are commonly completed using contract workers. I like the flexibility of contract work, but I also think that these workers need to be better at saving and planning for the future because of less employment stability. Usually they are paid more, but they need to use that to reduce the risk of being unemployed.

    One trend that I’ve seen taking place in some of these positions is the use of software and AI to determine if a worker is doing an acceptable level of work on a regular basis. Amazon might be one of the highest-profile companies doing this, especially as they expand into the delivery business. They are using the power of computers to manage an army of workers rather than traditional human managers. This includes terminating them. There’s an article that talks about some of the experiences of their workers.

    I don’t know how their system works, but I do know the frustration of trying to work with a company that doesn’t use humans for many tasks. If you’ve ever tried to contact Google, you know that it’s incredibly difficult to actually communicate with a human. Google seems to think that its automated systems can handle all situations. They might handle many things, but they don’t do a good job in plenty of situations, and there is little recourse to have a human intervene.

    I do think that AI and ML can help our companies better interact with the world in many cases, but these systems are certainly looking for broad patterns. Maybe these patterns handle the middle 80% of cases, or maybe it’s more like 60%, but there are plenty of situations where humans ought to be involved. Maybe more important, when someone uses these systems to make decisions that impact human life, there should be some explanation and understanding of how the model impacts this specific situation. We want to know why the computer comes to its conclusion in medical care, employment, legal issues, or maybe anything other situation.

    There is work being done to try and explain how these models work. The important thing, however, is to ensure that while we may understand the model, we also need to disclose the reasoning to those affected by the systems. Any appeal process should include this explanation, and likely with a human involved at some point to help evaluate the model for accuracy, fairness, or any other measure that is relevant. To me, we ought to require this of companies using AI models in their business practices.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Eyes Wide Open

    Not many of us work in startup environments, but many of us do work with new databases that are created for new applications. These might be carefully designed, thrown together, or your database might be constructed by an ORM. In any case, I find many people make decisions and write database code for today, solving the problems that they see in front of them. they often do this with little data and a single system. They might have their eyes on using some new technology, and they decide on a data strategy without really considering what they will need later.

    That seems to be what happened with Expensify, which started in the financial industry. Their system had requirements for low response times, multiple locations, and detailed logging. This required a robust database architecture, which turned out to be helpful when the company pivoted to a new business model. Their CTO talks about some of the problems he sees with startups making database decisions. I think many of these lessons are helpful for all organizations that are trying to ensure their database can grow and meet their needs.

    I think one of the most common problems I see is that developers and leaders get enamored with new technology. There is a lot of promise in some of the platforms and designs that are being put forth. Some are even proving themselves in high-profile situations, but not all. For most of us, however, we aren’t going to be solving the same problems in the same way. As the article notes, we’re not Google, but we’re also no Uber, Facebook, or Spotify. Choosing to mimic their choices because of their success doesn’t necessarily map to our business model. I find no shortage of companies that struggle to adopt some new platform because they built a proof-of-concept and assumed the way the system works with small amounts of data. This becomes an issue later with the moderate or large amounts that they have over time.

    I also see companies creating complexity, with the chance that they will need to deal with many petabytes or exabytes of data at some point. Face it, most of us will barely deal with terabytes of data in any particular system. We ought to plan for a high-performing system at that scale, not worry about a future that will not likely come. At the same time, we aren’t going to be dealing with megabytes of data, so if your developers only test on MBs, they are going to miss problems.

    I like the advice to go into your decisions with your eyes wide open. Don’t copy others, and realistically think about what you will need. I believe that engaging a data professional early is helpful. Developers do some amazing things when they build software, but so often the majority of them don’t really think about the challenges of a database system. They don’t consider low response times or ensuring there are HA and DR (two separate things) strategies. They also forget about the challenges of aggregation and reporting lots of data. Most humans work with a few rows of data at a time, which is what developers do on their systems. When you need to aggregate things, or all your customers are generating a workload, that’s when a data professional can help ensure you’ve properly indexed entities and planned for a demanding workload.

    I do like the common sense advice that most startups won’t outgrow a relatively modest single database server. Many applications might not as well, but that doesn’t mean you can put all your eggs in that one server basket. Make sure if it dies that you have multiple people that can recover it and ensure your system is quickly running in another place. There are different ways to handle this, but engage someone that knows your platform and have them ensure you have some staff, operations or developers, that understand how HA and DR work in your environment.

    Lastly, be secure. I really like the idea of always using stored procedures. I know this becomes a pain for developers, who now write code in two places, but this really helps you ensure better security, and maybe more importantly, ensures you can tune one part of your code regularly, the database side, without impacting the other side.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Republish: Are You a Tech Company?

    Still camping, heading back tonight, so I’ll see you Monday. Meanwhile, has anything changed for your organization? Re-read Are You a Tech Company? and let me know.

  • Republish: New IT Departments

    I’m off to South Dakota for a break and some unwired camping. You get New IT Departments to read. Has anything changed?