Category: Editorial

  • Containers and Databases

    There’s this push to use ever thinner and lighter weight computing techniques. We moved from mainframes to servers to blades to VMs, and now many developers are looking at containers more and more. What started as a Linux idea popularized by Docker has come to Windows, with container support in Windows Server 2016. Just recently SQL Server 2016 Developer edition was announced as a container, which makes deploying a SQL Server easy, with no install.

    Is this a good idea, though? Containers certainly make sense for many applications. Got a flaky Java app that crashes? Put it in a container, avoid dependencies on the OS, and if there’s an issue, restart it quickly. Got a service that needs to scale? Put it in a container and run multiple instances. Messaging, endpoint services, middle and front end apps, all of these can make fantastic use of containers.

    Containers fit in quite a few spaces, but do they make sense for databases? I’m not sure. After all, one of the advantages of containers is that they are stateless. Drop one, restart another. That’s not something we want to do with databases, as we need the data to persist. Containers also help when the application is unstable and may need restarting, but SQL Server, along with most database platforms, is very stable to run.

    Where do containers make sense? I think for development and test environments, containers have value. These are places where we may need to stop and recreate an environment quickly. Certainly  a container with a small amount of data, say a curated set of test data, is a great way to try or test code, then destroy the container, modify something, and repeat. A DevOps process, repeatable with containers.

    In production, however, I’m not sure that there is much of a place. Perhaps Express makes sense on laptops, where we can avoid the install of a SQL Server, but certainly the data needs to persist outside of the container. We’ve seen how to do this with WinDocks, and I’m sure some vendors will deploy containers in this way once they are available on laptop OSes. However, on servers? I don’t see the place where a SQL Server container makes sense, but please, let me know if you have a place for a containerized database.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Human and Machine Learning

    I was reading about the Microsoft Cognitive Services and their wider release in preview to more developers. There are a few of the many machine learning services that anyone can use to build more intelligence into their applications. The entire Cognitive Services include some interesting sets of APIs that allow us to build new features and even new capabilities that we might never have considered without the power of machine learning to change the behavior of systems over time.

    With machine learning, we have more and more ways to analyze data, allowing our systems to actually react to the data and become better at their particular task. I think there are certainly some dangers in how these systems might be hacked, but for many uses, I think ways that these platforms can work with speech, images and more might really alter the way in which we decide to build new applications in the future. The goal, at least according to Microsoft, is to have the AI services help humans accomplish tasks, not replace them. That’s something I hope actually comes true.

    I do think that we don’t quite know how these machine learning systems will grow and interact with people over time. They work by analyzing data, and altering behavior based on data, which means that we need to better understand the implications and effects of various algorithms and systems. I hope that more companies and developers spend time experimenting and working with various APIs and services to learn more about them. I’d like to see more projects and proof-of-concept systems.

    Some of us will just build things that aren’t useful, or that don’t even work well. That’s OK. We need to experiment, and understand these are experiments. These aren’t guaranteed ways of producing information from data. As long as we get that, or at least understand some of the work we may do for clients will need to be thrown away, that’s fine. I know there’s pressure in many companies to be efficient and just work on things that help the organization move forward. I’m sure some companies will make bad decisions, or even abuse their AI systems. I’m also hopeful that more will realize that some experimentation is necessary if you want to find new ways to create richer, more reactive and customized systems based on the massive amounts of data we collect every day.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Secure Medical Data Challenge

    Securing our databases and preventing the release of information our organizations have collected is an ongoing job for many of us. We patch our systems, ensure that our logins and users are granted just a few rights, encrypt data and backups, perhaps have various monitoring solutions, and certainly deal with plenty of stress at times. We work with network staff to ensure firewalls protect our systems. It may be a regular part of our job to actually argue with others that security is important for databases. If we work with regulated data, such as financial or medical information, then we may even have the struggles of compliance with auditors or even regulators.

    All of our work can be for naught with a few simple mistakes that someone in our organization might make. Perhaps an employee takes a copy of production data on a laptop home and loses it. An employee might use the same password for a secure portal that they do for Facebook or some other social site, getting hacked and exposing our systems to others. We might even have an employee click on a phishing email or insert a random USB drive into their laptop and compromise entire infrastructures. As a security professional once noted, we have to win every time. The bad guys only have to win once.

    It can be distressing, and even more so this week as I read this piece about medical data and Frank Abagnale, the inspiration for Catch My If You Can. In it, Mr. Abagnale states that he doesn’t think technology will ever defeat social engineering, which is distressing to me. He may be right, though I certainly hope that machine learning and other technologies, along with lots and lots of data, will find ways to catch abnormal queries and data extraction, which often are a signal that potential data loss may be under way.

    What’s more distressing in the piece for me is the fact that some of this data, like the birthdays and SSNs, are stored for years. Unlike credit card data, which is more valuable right away, unchangeable data becomes more valuable over time, so it pays to keep it around. Maybe the most distressing item might be someone using your identity to get services, which then become billed to you. How can you prove that you didn’t consume the services? I think that can be difficult, especially as we use more and more digital information that doesn’t necessarily tie directly to a particular person at a point in time. This might be especially true as we store digital pictures of signatures, which often are a poor imitation of what a person’s actual handwriting looks like. I shudder to think of those being used in a court of law.

    Perhaps even more disconcerting is the idea that children’s information is being taken at early ages, perhaps being sold decades later. I have no idea what to do here, or what I’d want others to do. These are going to be data problems we deal with for a long time, and many of us may end up collecting incorrect data in our organizations, thinking it’s correct. I don’t know many companies that have good processes for correcting data that’s incorrect when it’s received. Far too often we assume the data is correct, and only worry about ensuring the bits in a file are transformed correctly to the same bits in a database.

    Security is an ongoing problem, with no easy solution. There is one thing I’m sure of. We, as data professionals, are going to be the ones frustrated  by many of our efforts at security being thwarted by someone we work with.

    Steve Jones

     

  • The Multilingual Programmer

    At the recent SQL Konferenz in Germany, the keynote was from Michael Rhys of Microsoft. His talk was on the evolution and design of the U-SQL language. If you haven’t looked at it, U-SQL is what the Azure Data Lake (ADL) uses, and it’s designed to improve your ability to query various data sources in the ADL. If you want to know more, and begin working with U-SQL, we have a stairway you can go through.

    Michael opened his talk by looking at the languages he’d learned in his career. He started with APL and moved on from there. He asked if anyone had used APL, and there were few of us. It was my second language at University, and one I didn’t enjoy. The nature of the language was un-intuitive to me, and I was glad I only suffered for a few months. If you’d like to try it, you can tryapl.

    I thought this would make a fun discussion, so I wanted to ask: what languages did you learn for programming and in what order?

    For me, I started with BASIC, and a little assembler with early systems. I moved to Pascal in high school, trying to develop fun games and computer assisted homework help for myself. In University, I began with LISP, which caused plenty of people to drop out of computing. I’m not sure if that was a good idea or not, but I enjoyed that. From there, I went to APL, Assembler, Fortran and C before switching away from computers for a bit. When I returned, C++ was all the rage, and I soon found jobs that paid me to write FoxPro/Clipper code, then VB, then a touch of Java before the web became popular and I worked in ASP and ASP.NET. Along the way SQL became more and more of my career, and I’m glad it did.

    These days I’m trying to improve my C#, PowerShell, and Python skills, more for fun than anything else, but those are sueful as both languages are useful in data work. I haven’t done much with R, but I have fingers crossed that the sp_execute_external_script call that allows a parameter of @language=N’Python’ gets added to SQL Server before I need to learn any R. After all, most of the R libraries exist in Python, and I find the language much more intuitive.

    Let us know today what your journey has been, and if you haven’t been a developer, maybe its time to learn some programming skills. After all, I think that’s important for a DBA.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.0MB) podcast or subscribe to the feed at iTunes and Libsyn.