Voice of the DBA

Category: Editorial

  • Common Data Challenges

    Working with data has proven to be a challenge for most of my career. It’s been fun, and certainly fulfilling, but there are constant challenges involved. Let’s take away the hardware and admin challenges of keeping systems running, backed up, and performing well. I’m thinking today of the struggles of just data.

    There is a short article that talks about three common data issues, and these are some of the same ones I’ve struggled with for most of my career. Are these challenges any different than they were 20 or 30 years ago? I’m not sure, and I was working with databases and software nearly 30 years ago.

    ETL is a constant challenge, even today with tools like SSIS and Biml that make it much easier to build flows that migrate data from one database to another. ETL is such a challenge that many people make a very comfortable living helping organizations meet their every changing needs to move and prepare information for end users.

    The other challenges noted in the article are getting a complete data picture because of missing data and not trusting or believing in data. The latter hasn’t been as much of a problem for me.  I might describe it differently as more often we aren’t sure what weight to place on certain data. The world is messy, and often we collect data that we think might be valuable only to realize later that it doesn’t mean what we thought or our our hypothesis was incorrect in the first place.

    I think the challenges are part of what makes this work interesting. Our employers and clients might view the effort and time involved as frustrating, and I wish I had solutions to make our process quicker and smoother. Actually, I do think that advances like SSIS have made things quicker, but the world has grown more complex. We deal with more data from more systems, in the still chaotic, messy formats of the world.

    What are your challenges with data today? Are they getting better, worse, or still the same as they always were?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Data Incident

    I was reading a Google whitepaper and I think this was the first time I’d seen a potential loss of data referred to as a data incident. The paper deals with the response from the Google Cloud team if there are potential data loss issues, outlining a four part process for responding and ensuring the customer can get back to work. Azure has a similar plan, though AWS seems to expect the customer to do more of the work.

    This is the first public disclosure of a process, and it’s a good one. In fact, everyone should use this or have something similar in their organization. The way that the world seems to be moving, it’s likely that most of us will suffer a breach at some point, and we ought to know how to respond. Without some plan, it’s easy to have a situation spiral out of control.

    I’ve had formalized DR plans as well as incident responses for events such as a DDOS or virus attack, but the issues with data are somewhat different and probably deserve a dedicated plan of some sort. While I’ve often seen these handled under the security team response, that might not be enough for data services, especially when you have responsibility for data that is different than managing a software system or service.

    Data security is becoming a more important part of the data professional’s job, with legislation and best practices starting to require that we recognize a shared interest in sensitive data that extends beyond our organization. Whether this is something you like or dislike, it is becoming a part of our work.

    Personally I like having a crafted plan that is available for use in crisis situations. It can be hard to remember all the possible issues, and certainly crisis seem to occur when our best people aren’t available. Even more important, a plan means we have something we can practice to be sure that everyone understands how to react, limit the exposure of data, and even ensure that we can get services back up and running as soon as possible.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Creepiness of AI

    Last year, I watched a keynote talk from Matthew Renze about AI. In his talk, there were examples of the amazing things that Artificial Intelligence can do, as well as some of the creepier things have have been developed. It was an interesting talk, one that gives me inspiration and hope for the benefits of better computer algorithms as well as the concerns for various issues that we may be unprepared to deal with as a society.

    One of the more controversial items that occurred recently with AI was the Google phone call, where a computer answers a call and interacts with a human. What’s disconcerting here is that the person doesn’t know this is a computer, and there are speech patterns the computer uses, like um interspersed in the answers, that deceive someone. While this certainly might be helpful in scheduling situations shown in the call, there is a downside. Could you imagine artificial personas used in telephone scams or phishing situations? A help desk knowing some information and then asking for verification of other data?

    There are perhaps greater concerns, such as the work done with imitations of President Obama. There are fake speeches, generated by computer. While movie studios might want fake actors used to reduce labor costs, do we worry about the implications of a computer actually being able to imitate one of us in a video call?

    The use of AI and ML, with lots of data an organization might have gathered could be good and bad, but certainly opens the world to more problems than benefits if there isn’t mandatory disclosure of the cases where this is used. Since there are always going to be criminal elements that don’t obey rules, this might be very scary.

    There are certainly other issues, such as Target predicting a pregnancy, which was the first really, creepy data analysis thing I saw. That one is a few years old, and still bothers me as it was accurate, but an unrefined use of the data. A good example of where marketing groups are a bit too excited to use AI/ML technologies and don’t think through the implications. Fortunately this case seems to have dampened some of enthusiasm for prediction in retailing.

    Perhaps this item that is a bit funny, but it is also very worrisome for me. It’s the case of an AI system playing video games. The AI system decided the best way to get the best score was to pause the game. Rather than compete and try to do better, the computer decided to just stop. A completely unexpected outcome, probably because the feedback and expectations weren’t explicit. Since it seems quite often humans don’t specify their requirements or expectations very well, I could imagine this being a very large issue in AI systems as they are used more often. It could even be deadly or problematic when a system does something we didn’t anticipate, and impacts human health.

    Most of us won’t work with AI much as a technician, other than providing or managing some of the data. I do expect AI and ML systems to touch more and more of our lives, perhaps using our data for good, perhaps not. Hopefully we can help steer applications into the former more than the latter situation.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.7MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Parameter Fun

    Recently I was editing a document about SQL Server on Linux and the author noted that if you type “sqlcmd” without any parameters, you get the list of possible parameters back. I tried that on my Windows laptop and immediately got an error that no credentials were supplied. Apparently sqlcmd on Windows attempts to connect to the default instance.

    I thought the author had made an error, but sure enough, when I connected to Linux and ran just “sqlcmd”, I got a list of parameters. What was more fascinating to me was that if I ran “sqlcmd -?” on either platform, I got the parameter list, as I would expect. However, if I ran “sqlcmd /?”, this worked on Windows, but returned an error on Linux.

    A long time ago I wrote some command line utilities to help our network team manage a Netware 3.x environment. I was proud of my work and knew that the team would appreciate a few of my tools. When I first showed me boss and started to explain what it did, he stopped me and ran the name of the utility with a /? at the end. Nothing returned, and he told me to redo the work and ensure that /? always returned help for the tool.

    I’ve kept that habit for years and I’ve often tried that with new programs. Most have worked, including bcp and sqlcmd. Somehow, that hasn’t continued to this day. For sqlcmd.exe on Linux, only -? works. If you run Docker, you need a –help, though -help works. Other programs might be more strict, but it’s surprising to me how many different ways we’ve implemented help and parameters.

    PowerShell has Get-Help, but then uses single dashes for parameters. A number of newer cli tools, especially for Linux, seem to want two dashes for parameters, though a single dash often works. I’ve seen a few tools that mix single and double dashes, depending on which parameter. I did find this note that on Unix a few single dash parameters can be combined, so the double dash indicates we are using one parameter, not multiple ones. That makes sense, though I would argue that -abc meaning -a -b -c is a fundamental design problem in and of itself.

    The evolution of help and parameters seems funny to me. It’s likely caused by someone implementing the parameter short cutting in Unix at some point that now requires double dashes for multi-character parameters, which is really a case of a short-sighted design in Unix. In any case, understanding the behavior of parameters and help is a useful skill, especially in the current environment that tends to implement more scripts with command line utilities.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.1MB) podcast or subscribe to the feed at iTunes and Libsyn.