Voice of the DBA

Category: Editorial

  • Live versus Stored Data

    I’ve been working with various technologies as experiments over the last few years. I’m curious how useful I find them, as well as how they might help the clients and customers I run into at Redgate Software.

    I think Jupyter notebooks are a technology that has a lot of promise, and their use is growing in many organizations. One of the interesting things with notebooks is they can store results inside them, which allows a transfer of information in an interesting fashion. If I re-run a cell, the results can change and comparing them is challenging, but that is a separate issue. At least I can capture the results and share them.

    I’ve also been working with Power BI at times. I saw a demo of a query from PBI to Excel, where the data was then stored in the PBIX file. This query was disconnected, but it could be refreshed. You can also configure how this works, so you get the choice of live or stored data.

    If you are distributing information to users, those of us in the database world often think about having live data available and queried from a database, but often the same data gets queried over and over, which creates a load on our system. I know the data is often cached in these situations, but cache is a previous resource, so limiting the repeat queries can be valuable.

    I do think this is a decision point for some applications, where we might choose to limit the amount of live data v stored data. There are times when speed matters more than having exact data, so cached or stored data works well. Sometimes the most current data is critical, and you need to query the database.

    How do you decide when you want a live connection to a database, and when stored data is acceptable. I don’t know that I have any rules, but I evaluate each situation and try to work with users to make a decision. I suspect most people do the same thing, but if you always use live data, let me know today.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Are You Empowered?

    Today, even the most clockwork of tasks, like factory floor labor, can benefit from some degree of innovation and creative thinking. There are even some companies that are taking ideas from individual workers and improving the way things work. This can work well with jobs that are repetitive and subject to automation efforts.

    The less people’s jobs can be automated, however, the more you need them to take initiative. When someone’s job isn’t easily automated, there is a need for innovation and creative thought. Despite the evidence of all these studies, few managers are willing to take this leap. Today, only a small percentage of workers feel empowered and act resourcefully; most feel disenfranchised or locked down.

    What might be worse is that CEOs and management often feel they are encouraging innovation and creative work. I think managers, team leads, and even the most productive people on a team,  often don’t realize how much they can dampen others’ desire or ability to contribute. While some of this might be malicious action on the part of someone, I think often it’s just a lack of understanding and empathy for how others view the world.

    To make yourself heard, often we expect you to be a loud, strong voice. That doesn’t always work, but it also seems to conflate the idea of being loud or forceful with competency.

    Some of the smartest people I know are those that are soft spoken, polite, and careful about what comments they make. Allowing them to share their thoughts and provide input has often proved to be a good decision on my part.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Data Privacy Grows in California

    A few years ago the CCPA passed in California. This was the first large scale effort in the US to provide some legislative action on data privacy in the wake of the GDPR taking effect in the EU. While there have been some actions under the CCPA, it didn’t seem to have as much of an affect on how data is managed as the GDPR did.

    However, in November 2020, the citizens of California passed Proposition 24, which was seen as a CCPA 2.0. This expands data privacy laws and allows consumers to direct businesses not to share their information.

    To me, this is a good trend in that consumers want more say in how businesses use their data. Unlike a law passed by a legislature, consumers would have to amend this in another vote. This places a greater burden on businesses, both to handle data more carefully, but also to ensure more cybersecurity. I think this just requires business to stop giving lip services to these items and actually back up what they claim they want to do.

    I also like that consumers have new rights, which is important. I’ve been dealing with the GDPR at Redgate Software for a few years and this hasn’t been overly burdensome to many businesses. It is work, but often many companies have played fast and loose with consumer data in the past, and now they need to respect their customers’ rights more.

    I wish this type of regulation were in place across the entire US, and I hope it becomes a focus in the future. While this can raise some costs, it also requires good practices for data handling and security, something that most of us want from every company. If you need to use data, protect it.

    Steve Jones

  • A Beginning Project

    One of the ways in which developers become better at their craft is by building software. They tackle a personal project or maybe modify someone else’s code. They may work through exercises, solving problems like the Advent of Code. As with many tasks, practice makes perfect.

    Database developers need to do the same thing. To grow their skills, they certainly need to find ways to improve their skills. That means tackling some sort of database project, which is often inherently more complex than a software project. Beginning code projects might ask you to reverse a string. Certainly a T-SQL project might do that, but what about storing a set of data for strings? What about setting up a server and database? There can be a lot more to database development than the programming code.

    I do think that learning to model data, and store it in an efficient, useful fashion, is a skill. Whether in a relational database or a NoSQL store of some flavor, you need to find a way to not only solve a problem, but get feedback.

    Today I wonder if you have a thought on a good beginner project? I found a list, but are there any that you think will help a junior database developer learn how to built entities and relationships?

    Maybe more importantly, how do you evaluate a good design? How do you get feedback, or test and evaluate your work? Often many of us make choices in the real world, and we may not learn how well they actually work until lots of code is written and an application stresses our design. Then it might be too late, or very difficult, to change things.

    If you’ve got ideas, exercises, or even ways to share designs, let me know today.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.