Voice of the DBA

Category: Editorial

  • Machine Learning Challenges

    If you know someone well, you may think that you can predict what they’ll do in a situation. Those that know me might think that if I go to New York, I’ll make time to run in Central Park. That’s a good bet, and the last 7 or 8 times I’ve gone there to speak, that’s what I’ve done. However, that’s a very small data set, and you might not want to attach a high level of confidence to that prediction. There are certainly times that I’ve gone to New York and not run in Central Park.

    Machine Learning (ML) is a field that tries to make predictions about some data, based on previous known actions or results. By inferring relationships between lots of data and known actions, the ML model can guess what might happen with future data inputs. Lots of companies and fields are experimenting with ML techniques, with some of them working quite well. Others aren’t faring as well, and there is a lot of research underway to better understand the entire field. One possible explanation is emergence, which is discussed in this piece.

    If it’s raining, I likely won’t run. If it’s 20F, but dry, I will. Those that know me would inherently consider the type of weather as a new factor that has a high level of predictability for me. Time in the city, location, whether my family is with me, these are all other factors that would come into play. With enough data, and enough factors, you could likely come up with a model that predicted the likelihood of me running in Central Park when I visit New York City. It might not be perfect, but it would be good if you had enough data.

    However, that’s a very simple item, and few decision points. Would that work for a larger data set? I bet it would. If we were able to collect data from lots of 5k events, we could perhaps predict how many people would register show up. There are certainly more factors at play, and since we can’t control things like weather, the prediction might not be that valuable. Though if we could change the amount of resources the day of the event, maybe that would help reduce costs or make the event more successful.

    I think ML is a valuable tool that can help with quite a few problems, but in many cases, we just don’t know enough about the complex factors, such as emergence, that might impact us. I know our SQL Saturday organizers would love a model that helped them decide how much food or drinks to order for an event. My view is that there are likely too many unusual factors, but I wonder if we could start a data gathering project to come up with a model. Perhaps we should gather marketing metrics, registration metrics, and more, with the intention to predict the number of no-shows. Even if it wasn’t useful, it might be a fun project for people in this industry.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • In Depth Learning at the PASS Summit

    This past week the pre-conference sessions for the 2018 PASS Summit were released. There’s a good list of topics and speakers, returning and new presenters, that you can choose from. If you’re planning on attending the Summit, I would review the options and if any of these are relevant to your position, I’d bundle the cost of a session into your request for Summit funding. I’ve seen plenty of people ask for the registration fee without a pre-con and then try to negotiate additional funds later. Do yourself a favor and ask for everything at once.

    Are these sessions worth the time and money? Most are, and a single day packed with information is often easier to digest than a full week. However, you’ll still have days of random learning slots during the regular conference, so plan on spending a day learning and then shifting into a different mode for the rest of the conference. There is a lot of information presented in an all day session, and you’re likely to miss some of it. Most presenters provide a good set of materials to go with the class, but you should plan on taking lots of notes and then spending a few hours here and there across the next month reviewing and practicing what you’ve learned.

    I’ve often found that a pre-conference session is the best way for me to dive deep into a topic that I haven’t had the time to investigate on my own. Time is a precious resource and finding time in my busy work and family life is hard, especially for a full week class. I’d still like to get to a week long sqlSkills class, but I can’t seem to find the time. Maybe after next year when I have no more children at home and fewer commitments.

    I used to take a computer to classes, but the last few years I’ve moved back to paper and pen. I find that sketching out notes by hand is quicker and easier, I ignore typos, and somehow the act of writing cements things in my mind more. I’d recommend you try both, though certainly having your phone around to take a picture of something is incredibly handy at times. I’ve tried OneNote and a digital pen on a laptop, but again, it’s not as quick or smooth as pen on paper. The one thing I did like is that I could add pictures to my notes.

    A conference is a large investment, whether you make it or your employer does. Adding in a day of training on a topic is often a relatively small incremental increase. It’s worth the time and cost, if you’re already going to be gone for multiple days. I’m not sure if I’ll get to a pre-con at the Summit this year, but I’m hoping to get a day of watching and learning on a single topic in at one of the events I attend this year.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Believing the Data

    This is interesting to me. Let’s leave aside the moral, ethical, or political sides of this. Let’s just look at the data. A cop gave a ticket to a self driving car. Apparently the officer thought the car was within 10 feet of a pedestrian. The logs for the car show that it was 10.4 feet away. A negligible difference to me, but that might matter in a court of law.

    As we use digital data more and more to control vehicles, devices, even behavior, we are going to have problems, complaints, and unforeseen issues amongst humans and computers. Certainly our laws need reforming to deal with the digital world, but we also will have some cultural transformations to undertake.

    Much of our arbitration in the world has dealt with how well humans argue or debate topics. We are inconsistent, easily confused creatures, yet we often use our judgment and impressions to make decisions about whether we believe one individual over another. This has often been the case in law, even as more science with cameras, DNA, and other forensic techniques have tried to provide definitive proof one way or the other.

    As we use more digital devices, there will be, or should be, more logs and audit records of how systems behave. This worries me a bit in Machine Learning systems, but experts seem to believe we can unpack the rationale for decision making if necessary. In any case, we need to learn to trust and believe in the data, even if our eyes and instincts lead us to different conclusions. This doesn’t mean that the data makes the decision, but we can’t discard the data because we don’t like it. Certainly context matters, and in the original case above, we’d want to consider velocity and acceleration, and possibly other factors to determine if the car was too close.

    Ultimately, I believe digital records will start to prevail in more circumstances than we might be comfortable with. We’ll slowly change, and likely subsequent generations will trust data more than we do, perhaps even more than their senses. To me, this means that those of us that deal with data systems need to ensure there is extremely strong security and integrity of data records, and that we disclose algorithms and data processing techniques in a transparent way. Trust requires knowledge, which requires transparency. That’s something we certainly need to improve on as an industry.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.1MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Considering Other Views

    I love my Pebble 2 watch. It’s simple, the charge lasts for days, and it does what I want out of a device. However, it’s a device that isn’t made anymore, and after a fall, I have a crack in the screen. Since I like having some sort of watch in my wrist, and I like gathering some health data, I started looking for other options. I was reading a review of the Fitbit Versa the other day, and the first part of the review was a literal problem with much software design I’ve seen recently.

    The first item in the review talks about notifications. Those are valuable on a device like this, and one of problems is text size. There’s a quote in there: “The text size is tiny, even when you select the item”. That’s a problem I’ve had with many devices, including the Pebble. In fact, I see this in software overall as a problem with design. It’s one that comes about because I think that far too often we don’t consider a wider range of viewpoints.

    I’m getting older. A consequence of that is my eyes do not focus as well as they used to and I need to increase the fonts on screens. I can easily do this in browsers, and make SSMS fonts larger in most cases, but not all. I can increase the size of some things in Windows, but that can cause issues in others. My phone allows some font changes, but that can be limited. There are times when notifications or other text is hard to read, and this limits the usefulness of these features.

    Technology has been growing and expanding as I’ve aged. However, it seems that more and more often companies are using people in their 20s and 30s to design systems that will increasingly be used by an aging population. From dashboards in vehicles to labels in applications, it seems that far too many designers don’t consider the impact of their font, icon, and graphic choices on older eyes. Even the design of our operating systems don’t seem to have deeply embedded extensive flexibility of changing text and icon sizes. Sure we can alter resolution, but that’s a very intrusive operation and may break other things. Often we just need labels enlarged.

    Software and UX design are hard, and often we get caught up in our own viewpoint of how to build an application. We won’t ever completely solve that as users will always have different ways they want to work with our systems. The ability to change options, and especially deal with accessibility choices will grow as our systems move out to a wider and wider audience. What might have seemed simple and intuitive may be more complex for new users that haven’t evolved with out system.

    Our database software likely isn’t going to be often seen by end users, and certainly we don’t have control over the tools they use, but we still ought to consider  how our objects will be used by others. More descriptive names, extended properties that tools can read, and even views to remap complex structures are good ways to provide an easier interface to the data for report designers and third party tools. Those items come with a cost and have to be maintained as we change schemas, but that might be a small price to pay if we can prevent lots of support tickets requesting details about what OrdrLnPrc means.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.6MB) podcast or subscribe to the feed at iTunes and Libsyn.