Voice of the DBA

Author: way0utwest

  • The PowerBI Volleyball Report – Organizing Data To Start

    One of my goals this year was to build a report that I can present to kids and parents showing the skills progression of their kids. I have attempted a few times to put something together in Excel, but it wasn’t easy for parents to visualize, and I wanted something better to let them focus on a specific kid, without making the data hard to consume. Power BI makes this easy.

    This post looks at my data capture process, and how I evolved it a bit to make this easy to handle in Power BI.

    Data Capture

    I’ve tried a few ways to gather data during matches, but paper turns out to be the easiest way to ensure this happens quickly and fairly accurately. While there are a number of apps, I find them problematic as one wrong press means the data capture stops, and I can then miss the next item.

    I used to calculate all totals by hand and then put them in a spreadsheet for parents after each competition, which worked well, but this format isn’t easy for Power BI to deal with.

    2021-07-28 14_12_31-Stats2022_16Select.xlsx - Excel

    Easy for humans, but bad for reporting.

    As a result, I stopped to think what would be good and easy for Power BI. A table is best, and while I don’t want to bother with a database, I can modify my Excel formula easily enough to handle this.

    Since I will report on different areas, I decided to keep a master sheet for each report area. This means I have a “serve” worksheet, as well as others for Serve Receive, Attack, Digs, Blocks, and Assists. I can also add in new sheets as needed.

    Making a tabular format means that I added a few columns to this list. These columns are the slicers that atheletes and parents might want to use when they are reporting. In my case, these are:

    • date of event
    • event name
    • opponent
    • player

    With these columns, I can take my paper sheets, type in the raw data, and let Excel do a few calculations. This also means my main report is just a few sums from these raw sheets to get the totals above. From last season, I had data like this:

    2021-07-28 14_19_37-Stats2021_15Select.xlsx - Excel

    I also decided to enter data in the same order each time so that once I have a sum to copy data from this sheet for one player, I can copy/paste those formulas for the rest. This keeps the burden low for post game work.

    This also means that when I “Get Data” in Power BI, I just load data from each worksheet into a separate table in Power BI. This allows separate reports that are simpler to produce, as much of this data doesn’t make sense when combined together. This also means that I don’t have one huge table where I’m trying to manage data and potentially scrolling around a lot from left to right. This also means I can load this into SQL Server easily if I want to.

    This also means I need to set up incremental refresh in Power BI.

  • Daily Coping 2 Aug 2021

    I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag. 

    Today’s tip is to be a realistic optimist. Think about what could go right.

    I’m usually positive in life, but often I anticipate or think about potential problems and issues first. I think that was part of what helped me think critically about building or managing software, as I looked for problem spots early.

    However, as I get older, I know that often there aren’t likely to be big problems. There could be smaller ones, and we do need to try and mitigate those, but really, I ought to appreciate and think of the good things first.

    My daughter is driving herself back to college with a friend. It’s a long trip, 1775 miles, and she’s never done anything like that herself. As I’ve told a few friends, they all seem to worry first, or ask me about concerns.

    While I did think about ensuring she had a car that would make it, mostly I thought this was a great opportunity for her. A chance to experience the world with a friend, enjoy a long trip, manage her gas/food/restroom breaks along the way, and embrace a new challenge in life.

    I loved long trips when I was younger, and I still enjoy driving to cities. I think this will be a great event for her, the first time she’s taken a long road trip by herself.

  • Goal Progress for July 2021

    I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2021.

    As I look at goal progress for 2021, I’m going to follow a similar pattern as last year. I’ll give myself a current grade and report on overall progress in of the areas where I set goals.

    This is a two month update. I forgot to publish June, so here I am including both months together.

    Current Grade: C

    June was vacation month. I spent a few days in the Florida Keys, and then another week in Orlando coaching. In between, I was trying to get work done. Not as much progress as I’d like, especially as a lot of non-work time was spent catching up on chores around the ranch. An unusual level of things to fix.

    July was slightly better, but not a lot. I ended up with a couple trips away from home, lots of chores around the ranch, a business trip, and a bit of stress from a home renovation. As a result, I read a bit but fiction. I also tried to take a bit more time away from the computer.

    In any case, I’m still not pleased with progress for the most part, but I am learning that maintaining steady progress across the year is hard, even with relatively few goals..

    Reading

    The goal was 4 books (3 non-fiction, 1 tech).  I finished one in early July, and found a few more to add to the list. I haven’t been sure what to read, but I found a few career books. Now I need to find a tech book for the second half of the year.

    Here’s the current progress:

    I started with Lights Out as I think it’s easier to read.

    I’ll put a short review of the Project to Product book out in August month as well.

    Technical Skills

    I’m solving the Advent of Code three times.  I’m also studying for certification.

    In terms of work here, I did little with vacations. I should have spent a little time on Azure, but really, I didn’t do much. A little, but not much. For coding, I’ve just hadn’t put in much time. I started on day 7, but got a little stuck, and hadn’t completed the challenge in SQL, which is where I started. I should have started with Python or PoSH, as this is a recursion issue. I also hate recursion, and I’ve been avoiding this.

    For certifications, I started watching some videos, but I find myself getting too distracted with other things. So I’ve not been focused on more programming, which is really a state of where I am in my career. I should have done less here goal wise.

    Update:

    • Certification –  AZ-900 – 5%
    • Certification – DP-200 – 0%
    • Skills – T-SQL – 2020 Advent of Code – 5/25
    • Skills – Python – 2020 Advent of Code – 6/25
    • Skills – PowerShell – 2020 Advent of Code – 5/25
    • Skills – TBD

    Projects

    Little things are moving forward. I did send a note to the 2 groups near me offering to speak. The other one appears to have stopped meeting during the pandemic. I have scheduled a talk at one for August. Not sure about the second.

    I did a little more work on my volleyball report, and started to structure the data store in Excel better to ensure I can easily publish this early next season for the next team. I also started to write about how I’m doing this, so I think this is further along than it was before.

    • SQL Saturday Inc – 90%
    • Support events: I spoke at another event and a couple user groups
    • Speak at the 3 local user groups, at least one live presentation – 15%
    • Help organize a Denver/Colorado event, live or virtual – 20% (more conversations)
    • Complete my Power BI Volleyball report – I know lots of kids will use this, so I need to get it done – 70%
    • SQL Memorial – 90% – I updated this, sadly, as someone passed. I need to move this to straight GitHub Pages and get the custom domain set.

  • Deploying SQL Server Automatically

    I’ve had to install SQL Server many times over the years. Often it has felt that most of these installs were one-offs, a dev server, a new QA instance, a production server for a brand new application. A few times I’ve had to recover from disaster, including restoring master, but often, I just installed SQL Server manually because that was quick and guaranteed. I knew what needed to be done for most of the instances.

    However, I couldn’t be sure. In the past there were relatively few settings that were ever changed outside of the master database, but today there are more, and perhaps more importantly, the tolerance for making mistakes with any security missteps, is low. I’ve spent a bit of time learning to do unattended installs, and I’ve tried systems like FineBuild for installing SQL Server. In any size organization that might want certain standards set up, using one of these systems is important. Perhaps one of the better reasons to do this is ensure that your dev and test systems are configured the same as production to prevent any silly mis-configuration problems.

    This week I saw a post on using Ansible to install SQL Server on Linux, with all the various configuration. While I do think that it’s easier to install SQL Server on Linux programmatically, there are a number of items to set and configure. Following along the install, it’s interesting how much more this feels like something a developer would do, which is how many modern Operations groups approach installs and upgrades. Everything needs to be done without a human directly involved, often because of scale. There are so many systems we manage, often a mix of VMs, cloud, and local systems, which mean that a consistent, programmatic way of installing instances is needed.

    That’s likely one of the most important skills for a modern system administrator. We need to learn to use tools to get our work done. Whether these are free, open-source, or purchased, using a tool to work at scale is more important today than ever before. Even if you are part of a development group, learning to manage systems with code can help ensure you can scale and grow quickly, and more importantly, hand off this job easily to someone else when you reach that point.

    I think there are plenty of homemade scripts and tools that can help here, but almost every time I’ve seen one of these, it’s not very portable to other staff, especially if the author isn’t available. Too often these tools take shortcuts or are specifically tailored to the current environment and not the future one. These days, for many functions we tackle there are extremely well built tools available at no, low, or modest costs. I’d encourage anyone that is handling these tasks to learn about the tools available and pick one. I don’t know that I think any of these are necessarily easier to use, but they are all well documented and capable of handling most of your install, update, and configuration chores.

    Steve Jones