Voice of the DBA

Category: Editorial

  • SQL Server Phone Home

    A few versions ago Microsoft added the Customer Experience Improvement Service to the SQL Server platform. This is the CEIP service, and it comes with SQL Server. It is designed to get telemetry from your operation of a SQL Server instance on your premises. If you have Standard or Enterprise, you can turn this off, but if you use Developer or Eval, you cannot. Brent wrote a short description of this service recently, which is a good summary.

    When this first came out, there was a lot of concern with regards to data privacy, but I suspect most of this is overblown. Microsoft is bound by the GDPR, and my conversations with employees over the years have convinced me they take this seriously. Not just the legal staff, but many of the developers were surprised by the detail and documentation that they had to provide in order to gather data.

    Microsoft documents about what they collect, as well as the access restrictions at Microsoft and the data retention. I don’t see anything here that I am too concerned about, but I do think this is a great template that many organizations could use to document what data they capture from customers, how it is used, and how long they keep it. Whether you are required to do this or not now, I suspect more of us will be required to do this over time. You could start doing this now and be prepared early.

    This isn’t a fun task, and it’s very tedious, but if you capture this data and adjust it as you evolve and alter your schema, it’s not too bad. If you also include a place to document this, you can then work on this over time, not as one big, long, really annoying project. As data documentation has become more important to many of the customers I deal with at Redgate, having the ability to work on this over time is helpful in any size organization.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Chopping Off Data

    Do you know the difference between XLS and XLSX? They’re both Excel formats, and many of us might just use one or the other. After all, the latest versions of Excel work with both, and if you’ve been using a spreadsheet for years, perhaps you stick with the older format when exchanging data with others.

    As many of you might have seen, Public Health England recently learned there is a difference with large amounts of data. They found data was being chopped off in a spreadsheet because they were using the old XLS format, which only supports 65k rows. The newer XLSX format will support a million rows, but both numbers are far below what SQL Server, MySQL, PostgreSQL, and other platforms support. Those platforms support billions, and most are limited only by the storage available.

    I know that pandemic has had many groups scrambling to assemble and analyze data. We have people building dashboards and gathering data together in numerous ways, from paper and pencil to Excel to (hopefully) enterprise databases. An import into a relational store would make more sense than Excel, but I also understand that setting a schema, dealing with ETL and different formats from different sources, and other issues are a pain. There is a reason data professionals get paid a lot of money for these tasks.

    To me, this highlights one of the issues of working with SQL Server, MySQL, PostgreSQL, etc., in that they are cumbersome and difficult to get started with. Even if scientists chose Cassandra or MongoDB, there would be issues, because there aren’t easy, simple client tools that facilitate work with data sets coming in disparate text files and formats.

    I don’t mean to excuse this, because IT professionals should know better. If you’re using XLS, stop. Data volumes increase and you don’t want to realize you’ve hit the limit after data is lost.

    I get the ease and convenience of using Excel, but stop using it for major projects once we realize these are important. Once you realize this is data that needs to be intact, secured, and protected, put it in a real platform. Excel, PowerBI, and most tools can query SQL Server.

    Use those tools where you need them and where you can, just don’t use them as your database.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • Career Goals and Inspiration

    Many of us start working in technology because we enjoy programming or software or data or some other aspect of computing. We usually have some excitement that gets us motivated to start learning or apply for a job. Over time, we may move our area of focus, pick up new platforms, change jobs, and more. I’d like to think that most people actively make choices, rather than fall into the next situation, but I see the latter more often than the former. People take jobs quickly, when offered, even if the opportunity isn’t one for they were aiming.

    This certainly isn’t all bad, as there are times when we get opportunities we weren’t expecting. We may not realize what possibilities are available. However, I do regularly see people not thinking about the next step in their career until they need a job or they hate the one they are in. With that in mind, asking you to think about where you might want to go or what to achieve, I have a question for you.

    What things have  you seen others do and accomplish in their careers that you want to do as well?

    If you look at people you read about in the media, or see speak at an event, or hear about from friends, are there things that impress you? Are there jobs, roles, projects, or something else you’d like to accomplish?

    I think there are some neat goals that Brent has written about in his Epic Life Quest. These are bucket list items, and it’s a mix of business, career, and personal goals. I’ve had a similar bucket list, but I let it go and only started to rebuild it this year, along with my partner as we think about our future together.

    I will say that one of my big goals was speaking at the Ignite (formerly TechEd) conference. That was one of the biggest events in my industry, and I wanted to get there. I did get accepted, but a surgery required me to cancel, which was disappointing. Still, it’s a high point of my career.

    I still have a few private goals that I am aiming for in my career, but we’ll see if I get them. If you don’t have any, perhaps this weekend is a good time to stop and think of a few. You don’t need to set them all, or even commit to them, but just start actively managing your career and make a few goals.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • The Degradation of the Turing Test

    The Turing Test from Alan Turing was proposed as a test of an intelligent system. Could a human determine if the other party in a conversation was a machine? This was an interesting way of imagining how powerful a computer might grow and the types of answers it might give to a human. Interestingly enough Turing didn’t argue about the correctness of the answers, just that they appeared to be from a human.

    In some sense, I wonder how many people would have been fooled by the GPT-3 bot on Reddit. It posted comments for a week to a variety of threads. You can look through the posts by the “thegentlemetre” user, but this one caught my eye, and as I read it, I was surprised how much this looks like things I’ve seen posted on the Internet.

    Is this AI bot intelligent? I don’t know, but I do think that the quality of comments and posts on all sorts of threads and articles seems to go down over the years. Maybe it’s humans that are becoming worse at online communications rather than computers are getting smarter. Really, I think both things are happening.

    AI/ML systems are getting better at mimicking what humans do, and I suspect that in many cases, especially in small samples, they can fool many people, perhaps most. That’s disconcerting, especially as I already feel many people are a worse version of themselves online, without feedback and social cues directly available. Having bots add to the volume of poor communications and comments doesn’t seem useful to society in general.

    While I do think that AI systems can dramatically help us with mundane tasks and tedious work, I also think there can be problems if they become rigid in their actions, without allowing for some flexibility. Humans have discretion, and while they might not use it fairly, or even in ways that their organizations approve of, but they are flexible. Seeing these posts, I wonder if the AIs can learn to be flexible as well. I think they can be.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.