I’m either on a plane somewhere over the Atlantic or on the ground in London as you read this. Hopefully napping before a busy few days. I’m in town for the London Redgate Summit tomorrow, with some prep today, podcast recording, and trying to survive a 48 hour trip to one of my favorite cities in the world.
You get to re-read Understanding a Database today.
Managing a production database can be a challenge for many reasons. However storage growth has been one I see often as causing issues. From full logs to filled disks, it seems that we can struggle with dynamic workloads where we have no control over who can add data to systems.
Many DBAs have some sort of alert set on data files or disks, but even with those alerts, an unexpected load or runaway query could still fill a disk. I know I’ve been called back into the office because some business person decided to load an unexpected amount of data. In fact, more than a few times someone thought their load didn’t work for some reason and repeated loaded a large amount of data, filling a disk or log file.
Recently, I had a customer ask about tracking table sizes daily in their databases. They were looking to watch table growth. Another person dismissed this as not valuable, which was interesting to me. I’ve done this in the past, and I’ve found it valuable. No knowing the size every day, but having an idea of the growth factor. I didn’t keep this data long, usually a week rolling older data off as I added more. That was enough for me to trend how quickly a table was growing.
This also allowed me to set an alert if there was unusual growth for a table, and often track down a potential issue quickly. Either I’d realize our workload changed and tables were growing faster, or I could debug a sudden growth issue down to some sort of data load. Often I could catch an issue before I received an alert (or phone call).
I wonder how many of you track table sizes and if you find it beneficial. Is this something that helps you better understand your system or is the total size of the database good enough.
Or maybe you have so much storage allocated you don’t worry about space. Lucky you.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
This is a strange time for many technology professionals and their employment. It seems many organizations have open positions and are struggling to fill those slots with qualified candidates (one look at this). At the same time, many other organizations have followed the large technology firms (FAANG, MSFT) and laid off large numbers of their staff in the last year.
At the same time, with the hype and rapid growth of GenAI systems, we have companies that are looking to machines to fill some of their labor needs. Salesforce noted that they did not expect to hire more software engineers in 2025 as their plan was to use AI tech to help them write enough code. They’ve seen productivity gains with AI tech, so they aren’t adding more developers. They still need human developers, just not more of them.
While thinking about the labor market, I ran across this report from the Bureau of Labor Statistics in the US, which tries to project jobs across many different industries in the future. They typically are careful and conservative, and while they show growth in the need for database administrators and database architects (and software developers) they also admit they can’t quite tell what impact GenAI tech will have on the labor market. They do think there will be an impact, but for now they have the need for data professionals growing faster than average.
Why is it hard to get an IT job? I know plenty of people looking for work that have struggled to find a position that suits them. I think there are a few things at work here. First, many companies have been burned in the past with unqualified candidates. We know hiring is hard, and we will make mistakes with some hires. With the focus on cloud, new technologies, DevOps, and more, companies want to hire skilled people who can hit the ground running. I also think the supply of workers is high and companies are more choosy about who they hire.
Lots of companies are also investing in their own workforces, upskilling and re-skilling them in new technology. It’s often easier to train someone whose work ethic, soft skills, and other factors are known than hire someone new who might or might not be a good cultural fit. I know that I often see budgets for conferences and classes are low, but there are plenty of companies still sending people to classes, conferences, or providing opportunities to learn.
If you want a new job, or you’re struggling to find one, I can’t stress enough the importance of working on your own skills and showing prospective employers that you are investing in yourself. Take stock of your skills and work on projects, experiments, build things, blog about them, and show companies that you have the skills they need.
Don’t forget your soft skills as well if you’re investing in yourself. Learn to communicate well, both written and verbal, as that can often set you apart from other candidates. As someone that has interviewed others, those skills sometimes are more impactful than your technical ones.
Invest in yourself now, and you’ll be ready for the challenges you face in the future.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Recently there was some online complaints about social security numbers (SSNs) in the US being duplicated and re-used by individuals. This is really political gamesmanship, so ignore the political part. Just know that social security numbers appear to be one of the contenders used in many data models.
I found a good piece about how SSNs aren’t unique, and have a mess of problems. Despite this, many people seem to want to use SSNs as a primary or alternate key in their database systems. They also aren’t well secured in many systems, even though we should consider this sensitive PII data.
As we have moved many analog systems to digital ones, we often find that our assumptions about the rules governing data aren’t that well defined. I’ve worked in many systems where data elements were assumed to be unique in some way, but they actually weren’t. I’ve seen invoices, POs, and other “unique numbers” actually duplicated because of simple mistakes by humans. When we try to enforce uniqueness in a database, we end up with problems. Often we actually need to drop keys and make exceptions because the data must be entered.
This has led many people to not create unique constraints or even foreign keys in their systems, and I understand why they don’t. There are real problems when we assume the real world has the same strict structures we implement in code. I’ve seen systems go online and then FKs removed because of poor data quality. It might be a mistake, but it’s also a reality when we find there is existing child data without a parent. We might create a pseudo-parent at times, but we might also decide not to do so if that creates other problems.
I suspect over time the real world will migrate some of their problematic keys to something more robust. However, some that are used widely in older systems, like SSNs, are unlikely to change in my lifetime. There are far too many places where this is in use and I am not sure that there is any consensus to undertake the massive amount of work to implement something else. I suspect even trying to add digits to the value is a task we’ll put off indefinitely.
Be careful of using natural keys in your data models unless you are sure they are really a natural key. To me, I’m better off with some surrogate key in the event that my “natural key” turns out to not have the uniqueness I expected.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
Note, podcasts are only available for a limited time online.
Voice of the DBA 