Voice of the DBA

Author: way0utwest

  • The Evolution of AI

    I saw a study recently where an AI system was used to analyze code and trying to decide if authors were good. The conclusions were things we’d expect, and quite a few people laughed about this on Twitter. After all, if AI comes to the obvious answer, is it useful? Perhaps, but it’s also a little disappointing.

    I’ve felt that way about AI for some time. Years ago I went through an AI demo for the Titantic data set, coming to the conclusion that the lower you were in the ship, or poorer, the more likely you were to have died. I went through a tutorial with Microsoft on flight data, trying to determine why flights were late. It turned out the later they left, the more likely they were to be late arriving.

    While this might seem silly, and perhaps obvious, I think that’s OK. We don’t expect these early AI systems to be better than humans, at least I don’t. However, what I do expect is that they can do at least as well as humans, and having them return results that we could figure out is a good step. To me, this also gives me confidence that a computer can be used to analyze a complex problem or situation.

    There is more evolution coming in the AI and ML worlds. I have no doubt that these systems will improve, and likely find niches where they are valuable and very helpful in making decisions. The world is gathering more and more data, and it’s becoming hard for humans to analyze it, and maybe more importantly, react quickly. The rate at which data can change is overwhelming, especially for humans. I think that we will need computing systems to help process the vast amounts of data we continue to accrue.

    While I don’t know that I want AI/ML systems making decisions for me, I do think they can help reduce the burden of looking over data and help humans to better focus their analysis in certain areas. A symbiosis of computing software using AI, data, and a human to closely watch it is what I think will really help us become even more efficient and focused in our particular organization.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • What Happened to Hadoop?

    Six or Seven years back, Hadoop was the big thing. It was going to solve our big data analytic needs, it would provide cheap storage and query power with commodity servers. More and more companies were going to be using it. Microsoft invested in HDInsight, SQL Server got Polybase to query data directly from HDFS. I was seeing the Hadoop elephant everywhere. I still remember popping into a few sessions at SQLBits to try and learn a bit more about how Hadoop worked.

    In the last couple years, Hadoop has somewhat dropped off the radar as “the thing” that most companies need to become data driven and deal with large amounts of unstructured data. I found this analysis that talks about why Hadoop hasn’t taken over the world.

    The short answer? Real time needs, cloud computing, and containers. I think really, though, the complexity of Hadoop became a problem. It was too hard for most companies to deal with, and with too few willing to invest in the large infrastructure needed and skills required to manage a system. I’d say the same thing about Kubernetes, but it’s evolving rapidly to become easier, and it’s dirt simple in the cloud. I suspect we’ll see more Kubernetes cloud deployments than on premises.

    The other issue is the Hadoop batches jobs, which isn’t what many organizations want. They already deal with, and complain about, plenty of relational batch jobs. Whether this is ETL to a warehouse, cube processing, or some other delay. They want queries on data in place, which is becoming more commonplace all the time.

    Of course, one other important point from the piece is something I believe. The relational database, or data warehouse, is not going away. It’s still important to many organizations, and it’s useful to handle lots of reporting. With the growth of the SQL Server platform, you might even do more AI/ML analysis on your data in place, without the need to move it to an HDFS platform.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Setting 2020 Learning Goals

    I deliberately didn’t set any goals at the start of 2020 since I was taking six weeks off for a sabbatical. That’s done, and it’s time to look forward.

    I know I have a few things that will be important for work. Oracle, PostgreSQL, and MySQL will be on the list as we expand and try to sell more database development tools. We’re also working hard on improving SQL Monitor, so finding a way to improve me query tuning, analysis, and monitoring skills will exist.

    The goals I want to set here will be more personal goals that may relate to my career, but they’ll not necessarily be things I do at work. I’ll carve out some time elsewhere to get through these.

    Reading

    I like reading, and do a lot of us to escape and relax. I completed around 120 books last year, but almost all were fictional or non-technology books. I had hoped to get through 6 tech books, but that didn’t work.

    This year I’m going to scale back and tackle this in two ways. First, I want to get through at least 3 tech books that relate to specific coding or other skills. I’ve still got a couple from last year, including a Power BI one, so that’s one goal.

    The other one will be 2 career related tech books. I started Making Work Visible, and I need to finish that, along with another one.

    Goal:

    • 3 technical books
    • 2 non-tech, related books

    Projects

    I do a lot of demo stuff and build lots of PoC type things, but I need a couple projects. Specifically, I want to get projects done that will help at work, but will be useful in some way to me, or perhaps even others. With that in mind, and dovetailing slightly with those items above, I want to tackle these three projects and get them in a somewhat useable state:

    • A Power BI report that updates from a database
    • A mobile app reading data from somewhere
    • A website that showcases changes and data from a database.

    Not big projects, but getting something organized and relearning some tech about reporting and coding is important to me. I have a few rough ideas for which topics to use for these, but I want to try some basic PoC and see how things go.

    Checking In

    I think I need to check in monthly with these goals and start to see how I’m making progress. I’ll set reminders weekly to spend time here and then monthly to update some stats on where I am.

    I think this feels slightly ambitious, but we’ll see how 2020 goes. Hopefully better than 2019.

  • Automatic Refaction of PII

    Data privacy and protection has become a hot topic. Across the last few years, as I’ve worked with customers that deal with the GDPR or other laws, they’ve become more concerned and careful with their use of data. I find less and less resistance from developers about using sensitive production data in development environments, but still too much.

    Using data that’s stored in databases or other text files is one thing. What about data in less structured forms? I’ve dealt with a few customers over the years that recorded customer interactions for various purposes. Call center or financial organizations commonly do this, and sometimes deal with sensitive information. I know I’ve given my date of birth, credit card number, bank account number, and other data to representatives at various times.

    Those calls are often recorded, and the IT staffs have often had to ensure extra security is applied to these files. Not everyone has access to listen for various reasons, but certainly when there is sensitive information inside the audio (or video), this data needs the same protection we’d apply to data in other forms. Providing protection, or redacting the information, isn’t an easy task.

    I saw recently that the Amazon Transcribe service will now redact some PII information automatically. This service can be configured to automatically remove the information in text, which is fantastic. This is a great way to start to use technology in a safe way to ensure that we have less data  leakage when we re-use data. Certainly people might look at this data to better train reps, but it’s also likely someone wold look through transcripts to determine why customers are calling in and use that information to better design applications. In either case, there isn’t any need to expose PII data to them.

    This doesn’t protect against the data inside the audio, but perhaps companies can delete and remove those recordings sooner with transcripts available and more quickly reduce their potential attack surfaces. We’ll always have some liability, but reducing that and not unnecessarily creating issues is part of what we want to do when protecting data.