Voice of the DBA

Category: Editorial

  • Republish: Securing Your Instances

    I’m off today, enjoying the Thanksgiving holiday with family, so you get Securing Your Instances again.

  • AI Medical Record Scanning

    Artificial Intelligence (AI) systems continue to pervade many industries, usually where there is a lot of data and human developers struggle to build solutions that handle the complexities of the problem. Often the experts in these subject areas can’t fully articulate the reasoning behind how and why they use data to arrive at some conclusion. We developers often struggle to get clear specifications from clients in simple cases, so I certainly understand why AI might be attractive in complex problem spaces.

    I think AI is a promising way to try to tackle some of these issues, some of which are important to humans. These systems can achieve a focus and analysis of complex data in a way that very few people, if any, can. The sheer volume of data and myriad of relationships among the various different metrics captured eludes the ability of most people to properly analyze.

    I found an article that looks a neat non-intrusive way of analyzing medical records from patients to detect blood issues. In this case, an AI looks at the records of treatment and test results, looking behind the doctors and nurses to catch patterns that can indicate blood poisoning. The signs are subtle, and in today’s world where the humans are often overloaded, hand-offs between people can be incomplete. This means that doctors and nurses sometimes miss things. The AI doesn’t order treatment or prescribe anything, but raises a flag to alert humans. The medical staff can then review things, examine the patient, and decide on the treatment.

    Early results seem promising, and more importantly, this is the type of lever that computing can bring to leverage human expertise and help humans. Not only can they better treat patients, but potentially this can help enhance the understanding of what signs a doctor or nurse should look for in future patients. When a flag is raised and an issue detected, the staff can go over notes or discuss how they might have caught this themselves. A little continuous learning applied between humans and machines, just what we’d want in a DevOps software pipeline in technology.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The Challenge of New Platforms

    I saw a customer asking about Exasol recently, which is an in-memory, columnar database. I know nothing about it, and it might work great, but we don’t support it. I didn’t think much of it, as I’m sure the customer has a reason for choosing this platform.

    Later, however, I wondered if this was a good idea. Another customer had inquired about Knack, which I had never heard of either. Not only that, it’s an online database that appears to assemble its “tables” from data stored in other systems. Strange, but I’m sure it works well for some companies, especially those without many software developers.

    I often find there are developers, or even analysts, that find a new platform that appears to work better for their particular problem. Sometimes they’re excited, sometimes they have some experience in the past, but there seems to be a regular push to add new types of technology to many organizations. Often technology that isn’t substantially different in function from something that already is in place.

    Whenever you add a new technology to your organization, you are adding more than the capabilities. You are also adding the support of this system, which means not only training end users, but also training the staff that has to support the software. If you do this too often, you risk having staff that don’t know how to keep things running efficiently. This is why we rarely see companies changing their core database platform. The change is a big disruption.

    What’s more, our staff is not consistent across time. People come and go, and finding new people can be hard. Especially those that know all of our technology stacks. I appreciate and would like to see more customers working with new platforms, but I also think this has to be something an org considers carefully. After all, many software companies limit the number of products and versions they support for this very reason.

    There’s also the problem of technology becoming end-of-life’d. While we might think database platforms are around forever, some of them have disappeared over time, and even if they exist, support goes away. We might find ourselves with the need to upgrade multiple platforms, each of which requires different knowledge. Our staff might spend a lot of time learning and practicing upgrades for disparate platforms, knowledge that doesn’t transfer to the next upgrade.

    I am not advocating for everyone to run SQL Server (or PostgreSQL or MySQL or DB2 or Snowflake, etc.) I do think there are reasons why we might choose to use Synapse or Teradata instead of an Oracle database. However, I think the list of platforms ought to be limited in some way. Just like our list of programming languages should be limited. Having a handle on our domain of skills makes it easier to find, train, grow, and build knowledgeable staff. Adding to the list ought to be done slow and carefully, after some debate, discussion, and voting.

    I am glad that there are so many RDBMS platforms, and NOSQL platforms. It’s great to see people building new and improved databases. This work is how we get amazing datastores like Neo4J, Redis, and ElasticSearch. At the same time, I do think caution about adding new platforms is warranted inside of organizations. Reusing the knowledge we have should be the first thought, with the decision to grow based on a true need, not just someone’s desire to play with something new.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Self-Service Databases

    When I first started work as a software developer, I knew that getting an environment set up where I could compile a project might take a few hours or a few days. The complexities of how people built software projects, the dependencies, and more were handled in a very immature manner. These days I can often download a project from GitHub and compile it in minutes, with many of the issues handled automatically with various tools such as complex config files and package managers.

    In the world of database development, I often find people struggle to get a database set up with a project. Whether you are coding in SQL, C#, or Java, many organizations haven’t got a standard way of bringing up a database environment for developers. Some have a shared database for a team, which can be better, but can also lead to collisions of code as the pressure to get software built grows and team size increases.

    This leaves out the challenges of data, which can be changed by others, or lack the size/shape/skew to properly reflect what the production code has to handle. There are no shortage of bugs that come from incomplete data sets used in development that are unrepresentative of the live database.

    Do you have a way for developers to self-service the database assets they need? Whether this is a new dev database, a refreshed one from prod, or even a point in time from development.  Is there a way developers can get a database into the state that matters for them. When developers are ready to work on a problem, the last thing we want is a significant delay to them getting started.

    One of the things DevOps stresses is that we want to reduce friction and tedious, manual work were possible. Let the computer manage the tedious things. This should be how we manage those database environments. We also want some consistency among all developers, QA, CI, and any other environment, to ensure we can reproduce problems to troubleshoot issues.

    The customers I work with that have the smoothest development processes, and best ROI from the resources spent on development, have a way to hydrate a consistent database environment for their developers. More importantly, the developers can self-service this. When a single developer, or a team, decides to reset their environment, they can do so, applying in-flight code changes from version control to bring back the database to the state it was previously in.

    Application developers do this all the time, as they restart their application. Removing their abliity to change code, restart the app, test it, and observe results easily would be crippling. Yet, we rarely reset dev databases, and often, don’t have consistency between environments.

    Self-service is one of the key indicators of a mature software development organization. This is easily achievable for many app projects, but not simple for databases. Relatively few groups have figured out how to do this well, but those that do are much more productive and efficient than everyone else. Traits that are becoming more important in this highly competitive world with increasingly fickle customers.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.