Voice of the DBA

Author: way0utwest

  • Off to SQL in the City 2018

    I’m off today on a plane, flying to the UK for this Wednesday’s SQL in the City broadcast. This is our first event of 2018, and I’m excited. We’ll be talking GDPR and security, not the most exciting of topics, but we also have some cool new products to show off. I’m excited to see what people think of the work that our development teams have been doing.

    I’m also doing the keynote, which is both a little exciting and scary. Usually Simon Galbraith, founder and CEO does these, but for some reason I’ve been picked this time. I hope you enjoy what I have to say, and hopefully I don’t mess things up.

    I’ll also be showing you some improvements to SQL Clone with integrated data masking. This is a very well thought out product, which will really help many of you prepare databases for development and test without exposing sensitive data.

    Register today and I’ll see you, or rather, you’ll see me on Wednesday.

  • Build a Career Radar

    I got started in database work during an internship. I was asked to program some Lotus 1-2-3 spreadsheets to help a department. I was pretty proud of the mess I made, since it helped someone with scheduling. Another person noticed and dragged me over to work on some dBase stuff, eventually moving into Clipper and compiled code, even some basic event driven Windows work. This led to some Visual FoxPro and eventually a real database backing the application. The choices I made in moving from Lotus to Clipper to FoxPro to SQL Server (eventually) defined my career.

    I was reminded of those times in reading about technology radars. I’ve seen a few of these from companies like ThoughtWorks, and we’ve even experimented with them at Redgate. The idea of a radar is it’s a living document that assesses risks and rewards of technology. I didn’t create one explicitly early in my career, but I’ve always been interested in reading and learning about lots of technology, with an eye on trying to decide how useful, popular, and valuable different things would be for my career.

    There’s a nice piece from Neal Ford at Thoughtworks that says you should consider building a technology radar. Certainly if you’re in the business of creating technology you might want an organizational radar, but for your career, you might think about the technologies that are around you.

    I didn’t explicitly build my own radar, but I did evaluate Windows v. Linux, betting on the former since I considered the overwhelming inertia from Windows 3.1 and Windows 95. I evaluated Oracle v. SQL Server, and at times I almost wish I’d have chosen Oracle (for the pay drawn working with a cumbersome set of tools), I’m glad that I went with SQL Server. I’ve enjoyed the product most days. There are other choices along the way, including my decision to sit too long on VB v C#, but I can’t guess 100% right. Even now, as I look at R v Python, both choices that are tangential to my work, I’ve thought for a few years that Python was a better choice. I’m glad it was added to SQL Server and I expect its use to eclipse R in a year or two (or three).

    It’s hard to keep up. It can be overwhelming. Eugene Meidinger compares keeping up with learning to the stages of grief. It’s not a bad analogy, and I find many people someone on that spectrum, trying hard to figure out what and how to learn, not to mention finding the time and money to invest.

    I do think that it is important that most of us keep up with changing technology, at least a little. The world is changing and while some, or even many, of us stick with the same job for a long time, I don’t know anyone these days that expects to never change employers. Having a variety of skills, being able to learn quickly, and most importantly, muddle through projects that you know little about are skills that will help you find your way in an uncertain employment situation.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.2MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • An AI Loophole

    I’ve been working through some of the GDPR legislation, trying to find ways to explain this more clearly to customers and ensure that our products make sense in light of this law taking effect. Redgate is focused in this area and not only do we need to ensure we are compliant, but we also want to ensure we are building tools that help ensure you are compliant.

    In article 14, there’s this text: “the controller shall provide the data subject with the following information… the existence of automated decision-making, including … meaningful information about the logic involved.” That sounds a little concerning for those of us that work with data. It’s not always easy, but we can explain how a SUM or AVG function works, even with a complex OVER() clause and lots of joins and criteria.

    What do we do with a model running under SQL Server Machine Learning Services? The output from those scripts and models is often created by the model, without any obvious way to determine how the results are determined. The requirement to explain is enshrined in a law, one that many people are concerned about. With all the ways that ML and AI systems can get gamed and perhaps contain biases based on the data used to train the model, I can certainly see no shortage of people asking for explanations of decisions or conclusions.

    Fortunately Article 14 also has this part: “the provision of such information proves impossible or would involve a disproportionate effort …” That seems to give companies an out if they are using current systems about which little is known about the black box of machine learning. Certainly organizations are still charged with protecting the data subjects rights and freedoms, but this seems allow for the use of technologies that we can’t quite understand.

    I doubt this was the intention of the authors, though I do hope that this doesn’t prevent the use of newer tools and technologies. What I’d like to see take place is more research and understanding into how the various algorithms we want to use for ML and AI technologies work, perhaps with some more detailed analysis of the inner workings of the models.

    GDPR is going to be an interesting regulation that may have dramatic impacts on the world of data. I’m both excited and concerned to see how things move forward from here. Hopefully this results in better and more responsible data handling and doesn’t degenerate into a series of long term legal battles.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.5MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Brush Up on Your ETL Skills

    Many of us that work with data will find requests and demands to import or export data at some point. Plenty of us have regular processes that perform these actions, and we may regularly troubleshoot or enhance these activities. In fact, I know some people have a full time, or nearly full time, position just dealing with ETL operations.

    Working with data in disparate formats and the myriad of inconsistencies even when formats are known is a challenge. Integration Services is a useful tool, but many us find that we need to pre or post process data separate from a simple import or export. Some of us may prefer using T-SQL or other languages, such as R or Python, to process data rather than programming SSIS. It seems that I often find that every client wants a slightly different format or change to their data that a simple query export won’t handle.

    These days, as we add in Machine Learning and other downstream processing activities, it seems that there is more and more of a need to process data beyond imports and exports. After all, it seems that the majority of the time in any ML project is spent preparing and transforming data. In addition, in Article 15 of the GDPR, there is language that notes a data subject has the right to request a copy of the data relating to them when it is being processed by an organization. I don’t know how often someone will want to get data about themselves or their organization, but I’m sure it will happen more than it happens today.

    I think this means I’ll need to brush up on ETL skills, perhaps to ensure I can easily extract out a copy of an individual’s data. In fact, I probably should compile some scripts now to ensure I can let someone know what we information keep at SQLServerCentral that would fall under GDPR. I think it’s just email addresses, but I could be wrong.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.9MB) podcast or subscribe to the feed at iTunes and Libsyn.