Voice of the DBA

Category: Editorial

  • The Pyramid of Data

    Data is an important part of our world, and arguably the most important asset in computing. All the rest of the devices, platforms, and technologies we use are designed to work with data, by manipulating, storing, accessing, and presenting data in new ways. We need devices and operating systems to host software, and applications to work with data, but the data is the key to fuel for every engine.

    I see there being a pyramid of data, with various technologies that are used to store and work with data. This is roughly how I see things, with various sources that store data as the foundation, and then systems to summarize and aggregate data, a new layer of analytics with Data Science, and the ever present ways of interacting with the data in order to use it for insights and decisions.

    (SSRS) (Visualizations) (Excel) (Power BI)
    (Data Science) (Artificial Intelligence) (Machine Learning)
    (Data Warehouses) (Data Marts) (ETL) (Data Streams) (Linked Data Sources)
    (SQL Server) (Oracle) (CosmosDB) (ElasticSearch) (Redis) (HDFS) (MongoDB) (PostgreSQL)

    That’s been my traditional view of the data pyramid, but cloud computing, the orchestration of containers, and better ways of analyzing data without moving it lead me to think that this is becoming more of a mesh inside the pyramid that multiplexes connections between layers. While I do think AI and ML systems will become more and more useful to a wider variety of applications and organizations, I do think the adoption will move more slowly than the hype suggests. Likewise, I think containers will grow slowly as there is a need to rearchitect many applications.

    Certainly cloud computing is becoming more and more commonplace. I especially am starting to see more smaller organizations taking advantage of cloud platforms that build SaaS, not for large scales, but for very small scale organizations. The platforms themselves are constantly lowering the cost of engaging with the cloud at small scales, and making it more feasible for application developers to deliver tremendous value and capabilities to very small organizations that aren’t, and don’t want to be, software companies. They just need services and capabilities without a lot of effort. The Power platform from Microsoft is likely to accelerate this with easy development for any semi-skilled software developer.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • DR Priority

    Those of us that act as DBAs or sysadmins for database systems know that DR is a point of emphasis for us. We usually spend plenty of time ensuring backups are working and practicing restores. The automated scripts and processes that people use are some of the most popular and discussed topics on SQLServerCentral.

    However, we can’t ensure every system is protected at the same level. It’s not cost effective to cluster or build AGs with hot standbys, or even warm standbys, for many databases. Often our organization will ensure some are ready to go and others will have to be dealt with if there are issues.

    William Durkin noticed recently that O’Reilly hadn’t prepared well enough for their learning site. They were affected by the fires and power outages in California and since they host some of their systems in an on-premises data center, there were issues. Certainly we might think they hadn’t prepared well for DR, and perhaps this is a fair view of their service, but perhaps they made the decision not to built out an expensive DR environment for a service that can tolerate some downtime.

    This week, I wonder how some of you look at the systems you support. Perhaps you are the person that has to make decisions, or perhaps your organization doesn’t fund DR well. I’m wondering, how do you decide which systems don’t get enough DR support?

    Certainly there are inexpensive, perhaps crafty ways that some people might plan for DR. I know I’ve cobbled together systems from spare parts to use for testing restores, with the idea that the hardware might need to be an emergency DR server for a single system or two in the event of an incident. If you’ve got ideas on how to be prepared even without organizational support, let us know today.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • The Redgate Way

    Recently Matt Hilbert, from Redgate Software, wrote a piece on our blog about our journey to DevOps. It’s a great read, summing up some of the things that we’ve learned in our journey across the last decade. Matt is a great writer, and it’s worth a few minutes of your time to check it out and think about all the things that we’ve been through.

    I’ve known some people at Redgate for 18+ years, and I’ve worked there for 12, so I’ve had the chance to see quite a few changes. When I started, teams worked for long periods of times, in what is really a waterfall methodology. They went through substantial phases of development, testing, and beta releases, with Brad McGehee and myself having plenty of time to learn a product and then know it would be stable for a year or so.

    That slowly started to change, as Matt describes, with teams moving to new methods of building software, experimenting and learning. The Prompt team was one of the most ambitious, working in pair programming and finding ways to release almost on demand. Over time, other teams caught up and built some amazing processes. We even had two teams working on some products, with alternating two week sprint cycles to allow them to release every week. That was an impressive coordination of software development teams.

    Even today, I’m constantly impressed by teams. They aren’t all always rock stars, but they often exceed my expectations. I will see some slowdowns when teams change their people, process, or tooling, but then they will leap forward. The Data Masker team has been impressive lately, with some fantastic productivity improvements being added to the software. I apologized recently for not taking the Data Catalog team seriously for almost a year, but they have done more than I ever imagined with that tool in the last six months.

    We do continue to improve our software development skills at Redgate. We do release often, but really, we try to also improve the quality of our software, improve the skills of our developers, while working to retain talented individuals and help them enjoy their chosen career. We still produce bugs, we never get everything done as fast as sales, marketing, or even me, would like, but I do think that the last ten years has been an incredible growth as set of software development teams. Now our challenge is more closely aligning all teams to work in a loosely coupled, but tightly integrated fashion.

    DevOps is really a better way to build software for most organizations and teams. It often doesn’t change a lot about the actual code we write, but it does get developers, infrastructure staff, and management to rethink how we work, especially how we work together. That’s the hardest part to get through to many customers. This isn’t an install-a-tool-and-things-are-great system. Tools do help, but your attitude, your focus, and your willingness to work as a team are more important.

    Read about our journey. We’ve taken 1,000 steps, but there is more for us to learn, change, and implement as we move forward. Think about how you would want to change things at your company, and maybe pass this along to your manager. It’s not easy, and it might not be quick, but it’s an incredible journey. It’s also very much worth the time and effort you put into it.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • The Conference Springboard

    It’s been a little over a week since the 2019 PASS Summit and Ignite conferences ended. These are two of the largest events for data platform pros, and quite a few people either attended or watched some streaming from the events. I didn’t attend sessions at either one, and I have been trying to stream a few of the sessions as I find time.

    John Morehouse, of DCAC, wrote a nice piece at the end of the 2019 PASS Summit as he was thinking about how to grow his career after the event. He included a few things that he wants to do, such as looking over notes, touching new contacts, and sharing his knowledge. His post is worth a read, and you might follow along and try some of John’s ideas.

    I am a huge fan of notes, and really, paper notes. I’ve had a laptop for decades, I’ve had a screen I could write on for eight or so years, and I’ve found that nothing replicates the feeling of writing notes on paper. Perhaps it’s the slower pace, or the ease with which I can scratch out something (as opposed to the backspace, backspace, backspace method). I find that writing things down helps me remember. I don’t go back and review them too often, but I do at times. If you didn’t take notes at this event, plan for the future. If you did, glance through them before you start sharing information with others.

    At a few places I worked, the boss actually scheduled a half day or so (or hour meetings across a few days) in order for conference attendees to share some things with others. Attendees might present something to others, or just talk about sessions they attended. I do like the USB sticks (or downloads) that PASS lets you buy. If you attended, they’re steeply discounted. If you didn’t, they’re still reasonable. This is the chance to capture the presentations and watch those that conflicted with your schedule. I know a few people will do lunch and learns, watching a session a day for weeks. Some user groups do this as well. It’s a great way to think about how you can improve something at work and show an ROI.

    I think networking is the best reason to go to a conference and not just because you might want a new job. Certainly that can be one outcome, but the best part of networking for me has been the ability to reach out later to someone and ask a question. This isn’t just to the MVPs, speakers, and other big names. I’ve made contacts that used similar software or had environments configured like mine. Being able to ask them if they have solutions has helped me at different points in my career. Make sure you reach out to people on LinkedIn if you met them. If you met me, please feel free to connect.

    A conference is a fun, busy, inspiring, tiring, and exciting event to attend. I’m very lucky I’ve attended many, and even if you’ve only gone to one, you ought to feel the same way.  Whether it’s a week at a large event or a day at a SQL Saturday, take some time and ensure you get something out of the week to carry you along for the next few months.

    P.S. If you want to share some knowledge with a wider audience, I’d love an article on something you learned at a conference.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.