Voice of the DBA

Category: Editorial

  • Portable Data Storage

    I saw a review on a rugged SSD, and it got me thinking about local storage. Over the holidays I needed to scan something from a device at the ranch, but to get the image to my computer, I needed a flash drive. It took me 10 minutes of searching around to find one because I rarely use them anymore. In fact, I couldn’t remember the last time I’d used one for a transfer between computers. I just send stuff through OneDrive/DropBox/GDrive or some other mechanism.

    Today I’m wondering if the rest of you have abandoned using portable storage or if is it something you find useful. How often do you use portable storage in your life? Apart from laptops or mobile phones, I’m wondering about portable disk drives or any device where you transfer data using a wire rather than wirelessly. Note, I’m not talking about going from one computer to another on a wired network. I mean moving the storage from one place to another with your hands.

    The only place I know I’m regularly using a portable drive is in the Tesla. There I have a rugged portable SSD that captures the dashcam and security footage. I had to transfer it from the drive to my computer last year when I saw an accident. I think that might have been the last time that I actually used portable storage (before the scan).

    I have lots of old portable drives, and even old CD/DVDs I’ve recorded that have SQL Server ISO, pictures, backups of databases, and more. However, I don’t use them anymore. I don’t know how many of those devices still work, and I ought to check them, load the backups into cloud storage, and then recycle (after erasing) the devices. Or trashing them. I’m guessing many of them are just old tech at this point no one wants.

    I think the era of using portable storage most of the time is likely gone. Even my friends who like to keep their own backups locally tend to use a NAS of some sort and wireless transfers, as opposed to plugging in. However, maybe I’m wrong. Are some of you still Luddites with storage technology, if that word can even apply to digital storage technology?

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • Continuity Across Restarts

    There are a lot of database platforms, and each tries to convince you theirs is better. As Brent points out in that link, sometimes they just skip comparing themselves to other platforms because it makes them look better. They only look at the platforms they compete well against.

    For most of us, we often just need basic CRUD operations. I know that most RDBMS platforms would work for us, and sometimes NoSQL ones work as well, though I think that NoSQL isn’t necessarily better for many applications (maybe most). You may feel differently, but that’s my view. While I use SQL Server, I think the majority of systems I’ve managed or built could easily run on MySQL, PostgreSQL, or many other platforms.

    In any case, we want to store some data, get it back, maybe change it, and sometimes delete it. I think that last operation is fairly rare as most of us use soft deletes and because of this, our databases continue to grow like crazy. There was an interesting post from Brent on MySQL in AWS Aurora, where the buffer pool cache can survive a restart of the database engine. This is for the cloud version from AWS, not MySQL in general, but that’s fairly cool to me.

    And scary.

    If you read more, there are some limitations, and the value of this, to me, is limited. However, it’s a neat idea, since I could see some sort of restart of the database from a patch for the engine or even a configuration change. For MySQL, clients can’t connect while the buffer pool is initialized, which isn’t something that we deal with in SQL Server, but I can see this being useful. Or perhaps this helps if an Aurora node goes down and the next one picks up the database engine with a full buffer pool? Not sure if that actually happens from the docs, but if they kept the buffer pool separate enough or up to date on a second node, that would be great.

    In the real world, I wonder to what extent we find an empty buffer pool at startup to be a problem. Sure the first few queries are slow, but most of the workload isn’t the first few queries. It’s all the clients that come in the next 5, 10, 20 minutes.

    As Brent notes, this might not be something we want in SQL Server, but it is good to see cloud-enabled platforms trying something new. Some of those will turn out to not work very well, but for those that do, I bet we see other platforms start to adopt some of the ideas from new technology.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • Re-Evaluating the Cloud

    Last year 37 Signals (makers of Basecamp and Hey)  announced they were leaving the cloud. I wrote about the decision, and wondered if they’d look back at this as a great decision or one they’d regret and backtrack to the cloud again. They planned to build their own tooling, buy a bunch of servers, and run their own data center (or rather, rent space in someone’s data center).

    Recently there was an update, in an FAQ, about how the transition has gone. In short, very well. One of the founders, David Heinemeier Hansson answered several questions about the move and the financial status. They didn’t hire more people, so their payroll is the same. They used a service to unpack and rack their servers, so they could just connect to them remotely and not deal with hardware. They built their own redundancy across two data centers where they rent space, and they think they will save well over $7 million in the next 5 years. They had a $3.2million cloud budget per year, so that appears to be halved (3×5=15 – 7 = 8ish) with their move.

    There are some answers to various questions, which are likely of interest to many data professionals who might feel pressure to move to the cloud. First, they discuss resource optimization in the cloud. I think that’s really hard, to do well and if they gave the details of what they’d done, that might help others decide if 37 Signals did a good job, or if there are things others could do. I think many people might struggle to optimize their usage, especially if you are only responsible for one thing, like the database. In many orgs, once something gets deployed, or developers build a PoC using some cool, new service, it’s hard to get rid of it or even change how it’s used. I think 37 Signals has some agility here that many orgs struggle to implement.

    Rewriting things are cloud-native is the way to go, in my mind, and I do have customers and clients who see applications working better in the cloud. However, I might argue that rewriting applications is hard and expensive, and many companies aren’t great at writing software that’s efficient, so rewrites are hard. I also think in most organizations, developers struggle to understand what cloud-native means. I think DHH is a little off here, as their developers probably could make better software written for the cloud, but they didn’t want to spend the time and/or money.

    The points about security and reliability are fair, but I think those are a function of having good people implementing systems. You can get good security and reliability in the cloud or poor implementations of either. The thing about security is that the overall protections that detect some of the DDOS or attacks are better with Azure/AWS, but simple SQL injection or open firewalls are still a problem. Authentication and security are just hard for most people and often people do this poorly in the cloud. To be fair, most people don’t do this well on-premises, so at least in the cloud, there are some services/scanners/checks that let you know when you’ve messed up. Whether you change or implement their suggestions is another whole debate.

    Is reliability better? AGs are hard in SQL Server, as are distributed clusters. The cloud services do this better for most people, but maybe not for you. That brings me to his super engineer point. He doesn’t think he has super engineers on staff, but I disagree. I think they can pick good people, especially because of their profile, and they don’t need a lot of people, so their average engineer is likely better than the average engineer at most organizations. They built some great software, and they’ve built an amazing framework and tooling to deploy their software. Don’t tell me that’s even close to the capabilities of the staff at many organizations. We have really good engineers at Redgate, and I think we’re above average as well. I don’t know how we compare to them, but we haven’t had people publicly write and release code that thousands of other developers use. At least not at quite the same high profile level.

    The cloud has its place. It can work well and it can be very expensive. This journey is worth sharing with your management if they want to move to the cloud, especially if they want to lift and shift. That might create some flexibility and CapEx/OpEx changes that are worth it, but you ought to debate and question whether that’s reality or marketing hype from a vendor. After all, it’s not clear if a cloud move is really something that returns an ROI or one that reduces your profitability.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • Advice for Business

    During the last few months, I’ve seen a few different advice posts that caught me eye. One was Kevin Kelley’s blog (and his book) on life advice that I previously wrote about, I look at the book once a week or two, read 1-2 items, and think about them. I think some of them are geared more for younger people growing into life, but quite a few are still things that I appreciate as learnings or reminders.

    Recently I ran across another one, Sam Altman’s post looking back from a business point of view. I’ve worked in business for a long time, used to own one, and I tend to enjoy smaller businesses than larger ones. While I enjoyed my time at JD Edwards, I prefer companies with a few hundred people rather than 10,000 or more.

    This is an interesting list and one that looks at the world more from a startup perspective. Sam Altman was the CEO of OpenAI (maybe still is), and has worked in several small tech companies in his career. Some of these items are things that I’ve seen or used. Maybe one of the more interesting ones from a business perspective is about incentives. These really do drive and change how people work, and often management gets this wrong by incentivizing one thing, but saying or preaching another.

    However, the advice I think resonates more with me, as someone who works inside of an organization and with others, are the items that relate to people. One is about cohesive teams, and how they can get things done with both calmness and urgency. I’ve always wanted to interview with and know who I work with at many companies because having a team I enjoy and work with is powerful. I think Grant and Ryan are amazing, and I wish our jobs were a little more closely aligned. Unfortunately, we all tend to work on slightly different things most of the time, and work more solo, but I do appreciate the projects we tackle together.

    Another is that things that matter are important. You (and I) need a sense of purpose, which is why hard things that matter are easier than easy things that don’t. In life and at work. I also think that recruiting is important. I do look for people who I like, appreciate their views, and can work alongside more than those who know all the skills. We can learn from each other and teach each other when people have that potential (in addition to intelligence and drive). That’s why I think it’s almost always worth hiring good people, even if you don’t have a specific use for them. Too often we hire people for a need, and they’re way less qualified than others.

    Maybe my view is summarized well in the last entry: working with great people is one of the best parts of life. We spend so much time at work, so it better be enjoyable.

    By the way, if you want to find and enjoy great opportunities, learn to be better. Better at your profession, better at learning, better at being part of a team.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.