Voice of the DBA

Category: Editorial

  • Measuring Effectiveness

    I’ve never been the guy that produced code the fastest, or the most code in a day. I haven’t been able to show that I work the most hours, or that I even keep great hours. I’m often late or erratic in the times I come and go from work. I don’t show that I am closing tickets in consistent times as I often will be slow to close new issues. I haven’t ever been the employee of the month.

    What I have been is effective. I get things done. I leave the organization and the job better than when I found it. I work to deliver value for the salary I’m paid. That isn’t something you can easily measure with many traditional metrics that people use, but there are ways to decide that I’m a good employee. It’s a little ephemeral and sometimes uncomfortable to have an employee that doesn’t seem to fit the same model as most other employees, but that doesn’t mean you wouldn’t want me working on your staff.

    This article on measuring DevOps tries to explain how you can determine if your DevOps process is working, which is a similar ephemeral way of working. When we implement DevOps, we don’t have large project plans, we don’t look forward to completing a system. We undertake a set of work, knowing that we don’t have an end date. We just keep doing work and getting things done that are needed. In this case, how can we measure if a project was on time, on budget, and finished?

    I’d argue, as would the article, that those aren’t good measurements. Those are the ways we would traditionally look at work, but in software, those aren’t often the things we want to look at. If we deliver software that meets some requirements, but users struggle to use it or complain, is it done? Most management would call it done, just like the airlines mark the plane as leaving on time when it pushes back from the gate. Standing on the runway for 45 minutes doesn’t count against that target, even though I’d call that leaving late.

    The article talks about picking things that bother a manager or team members about the software or their process. Some of these are easy, like the time to deliver a new server or recover from a failure. Others that I’ve found to be useful are the time to assemble a release or deploy software to an environment. While we can sometimes play with what these numbers measure by moving to smaller set of changes or altering our process, we’ll quickly find out if we have a lot of overhead in our work by adding these measurements.

    What about the software itself? Can I actually compare the time to deliver report a vs. report b? One might be much more complex than the other. That’s true, but we ought to average out delivery estimates over time, and we can certainly separate out query writing of the complex logic against the time required for formatting. In fact, it might be good to start to measure different parts of software delivery to find out if certain people are better at some parts, or if requirements from clients are causing unnecessary delays. I know I’ve certainly had some formatting complaints require more time to get right than the entire rest of the software.

    It can be hard to develop metrics that have outcomes and are truly actionable, but it’s a better way to determine if your team is improving. Relying on simple and traditional metrics is lazy, and allows for lots of argument and debate over whether a team is doing well or not. Tackling those specific items that are irritating to management or customers might take some work, but you’ll end up with a list of things that can be targeted for improvement and show progress that actually means something.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Badly Trained AI

    Most of us know that data is being used to make more and more decisions inside of all kinds of organizations from retail giants to banks to sports teams. We are constantly asked, or see reports, of data driven decisions. We often need to show some data that supports and explains the rationale for making some choice. As our populace becomes more data savvy, I expect this trend to continue.

    AI (Artificial Intelligence), and the related Machine Learning (ML), are becoming more and more widely used. From mobile phones to autos to trading systems, we regularly see new “AI capabilities” being added to products and services. No business or industry seems immune, and I’m sure many of you are seeing AI being incorporated or feeling pressure to start using some AI in your work. As you work with AI, or start to, you’ll quickly realize the importance of data in your efforts.

    This is true for the cleanliness of data, but perhaps even more important in the tagging of data sets. As Amazon learned, building an AI or ML system, is hard. They scrapped one system that was being used to rate resumes and help their recruiters sort through the volume of applications they received. Why? Because of bias.

    Apparently the system would downgrade women’s resumes for various reasons. To me, this is a perfect example of a principle I’ve had throughout my career: garbage in garbage out. In this case it’s not necessarily bad data that was the problem, but bad tagging of what was a good and bad resume, probably from the internal prejudices of a few people.

    There will be more dangers as we use ML and AI technologies in our work. It won’t be enough that we clean the raw data for training, but also that we clean and properly manage the tagging of what data sets represent the results we are looking for. Like in much of our software, it’s easy for us to only consider the happy path, to only tag those items we think are good results. That is useful, but we might also be unconsciously tagging other results as bad, which appears to have happened to Amazon.

    We can build systems that do a better, more rational job than most humans, but we need extraordinary care to ensure our training data lacks bias. Unfortunately, most people both think they’re not biased and are unwilling to spend extra resources to deeply examine the data. Two things that worry me about the future of our AI/ML systems that will inform us.

    Steve Jones

  • Republish: MY Data

    As I noted, I’ve been ill. This is being republished to give me another day to get well: MY Data.

  • The Giving Back Workload

    One of the amazing things about the #sqlfamily is that we often help each other in a very giving way. Certainly other technical communities have lots of sharing and helping, though often with more of an RTFM attitude. I don’t know how well many other industries share information with each other, but the nature of being connected with technology through blogs, free events, #sqlhelp and more certainly seems to facilitate lots of sharing.

    When SQL Saturday first started, Andy Warren and I worried that we would struggle to get speakers to donate time for a number of free events. The first year we had discussions about how to find speakers if we were to get 10 events scheduled in a year. That proved to be a non issue. It’s amazing to think that these days we have 6 or 7 events on a single date, and I’m looking forward to the time when there are actually 10 events on one Saturday.

    Our speakers donate a lot of their own time and money to give back to others. In 2018, I’m particularly impressed by Kevin Feasel, who I was lucky enough to see at a number of events and even share a bus ride on a double decker in Cambridge. I’ve also met a number of volunteers and attendees who have had similar schedules during a year or two, where they have lots of excitement. What often happens is that after a few years, they scale back, some of them even giving up on all of their efforts.

    It’s wonderful that people give back, but I do worry about sustainability, both for events and individuals. Andy Warren wrote recently about volunteer time and the load in life. I haven’t tracked my time, but I know that it sometimes is quite a load. While Redgate often covers my expenses, the time away from home to speak at events comes out of family time as I still need to work a normal week when I’m speaking on Saturday.

    I do think it’s important to volunteer and give back to others. Help those getting started, those less fortunate, or those that are inspired to move forward in life. There’s a time and place for all of us, and I do hope all of you think about where and how you can help others, either in this field or in other ways. If not now, then at some point in the future.

    Time is a valuable resource, and while I think Andy underestimates the amount spent on his endeavors, I think it’s good that after many years of helping others, he is trying to quantify the time spend and perhaps inspire some of you to think about where you might do the same. 50 hours a year is both a lot and a little. What are your plans for the coming year and could you spare 50 hours to help others? Leave a comment here, and let Andy know what you think of his plans.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.9MB) podcast or subscribe to the feed at iTunes and Libsyn.