Voice of the DBA

Author: way0utwest

  • When to Use a Database

    One of the trends of the last ten years has been for many developers to try and avoid using a relational database where possible. Some look to NoSQL data stores, and others even consider flat file type stores of JSON or other formats that allow developers to work with speed and agility. Quite often it seems that applications grow to require some sort of relational store, often as an additional data store.

    I ran across an article from a data science and analysis developer that is often performing work in R or Python on datasets. At some point, the post notes that when your dataset(s) become larger than memory, you might want to consider using a local database of some sort.

    Actually, the first question the author asked was “when is your data too big?” Their answer: when operations take a long time, which was 20 seconds for the author. I tend to agree as I am looking for Notepad-like startup performance for apps, and query results in low 10s of seconds.

    Most people that perform some sort of data analysis understand tables. Whether this is in R, Python, or even Excel, the table structure for data is familiar and easy to work with. While some analysts might not be overly concerned about normalization, that isn’t always a problem for situations where data is loaded into systems and rarely (if ever) updated. In these cases, just having a database of some sort, could speed up your work.

    I think you ought to use a database early, if for no other reason than this is good practice with loading and storing data in a form that is persistent, scalable, and often can perform better across time with disparate queries and data manipulation. While quick experimentation is rapid with in-memory tools, I think a database is better suited to queries across time.

    I know I’m biased, but if you find data scientists and other analysts struggling with data sets, offer them a database. They can easily share data, you can protect it with backups, and you might find that you both learn a few things from working together.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Daily Coping 3 Dec 2021

    I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.

    Today’s tip is to enjoy new music today. Listen to a recommendation from someone else.

    I’ve had a lot of music tips the last 18 months. Today, I’m branching out. The other day in the car, my wife searched for an artist she’d discovered and played a couple songs for me. I enjoyed them, so I spent some time going through the artist catalog while working.

    Anderson .Paak is a singer/rapper/songwriter that is funky, soul-y, fun to listen to artist. If you like those genres, give him a try. Not always safe-for-work, so be warned.

  • Daily Coping 2 Dec 2021

    I started to add a daily coping tip to the SQLServerCentral newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.

    Today’s tip is to discover your artistic side. Design a holiday card.

    I used to write an annual letter for the household that I’d send out to friends and family. At some point we stopped sending cards and the letter ended. I miss those days, but with this tip, I’m going to try and do a fun card instead.

    My daughter has been gone for the entire semester. No trips back for fall break or Thanksgiving. So I decided to build a holiday card to welcome her back, using pictures I’ve taken of her horse for the last few months. Here’s an image of my attempt.

    2021-11-29 15_21_21-KJCard_2021.docx - Word

  • Goal Progress for November 2021

    I set goals at the beginning of the year, and I’m tracking my progress in these updates during 2021.

    I missed writing this in October. It was a stressful time with lots of deadlines, presentations, and travel.

    I slacked off a bit in November, coming out of a busy work time and needing time away from my technical work. At least, outside of work. I also started coaching, and that required more personal time, so less career time.

    Current Grade: C+

    Given where I am, I think I’m middle of the road. I’ve done well on reading. I’ve made some project progress. A little technical skill work, but not enough.

    Reading

    I’ve been working through a couple books. I like the feedback one, but I find myself stopping and thinking often. The PoSh book is a bit of a project, but I’m enjoying it.

    Here’s the current progress:

    Technical Skills

    No coding here on the Advent of Code. I realized, however, that I’m not a great programmer. Some of these problems stump me and slow down my enthusiasm. I also don’t love the process of problem solving on the computer just for fun. I have too many other hobbies for this to be a priority.

    I have, however, started studying for a certification. I’ve been going through the DP-900 modules, trying to learn a bit every day or two. I went through 3 modules in November after the Data Community Summit. I’ll keep going here.

    Projects

    No real changes here. Minor updates, but not enough time spent on these items. I have started to get organized for volleyball statistics for the new season and I need to document the process a bit to explain it to myself, as well as do some better data analysis.