Voice of the DBA

Category: Editorial

  • Faster Cloud Warehouses

    I think the cloud is a perfect place for a data warehouse. In many organizations, I’ve found that a data warehouse system is often the largest SQL Server database, both in size of database and also in terms of resources allocated. These systems often handle many complex queries for business users and are allocated a large number of CPUs as well as lots of RAM. Even then, many ad hoc BI tools or lots of “what-if” queries can bring the system to its knees, often causing lots of stress for database administrators during the periods of time when the system is in heavy use.

    Fortunately, many of these systems aren’t in use all the time. Often these are systems used by financial departments to “close the books” at month, quarter, or year end. It’s at these times when lots of resources are needed. Outside of these times, the data warehouse might be one of the least used systems, which makes it a perfect choice for a cloud, scale on demand, environment. Scale up when needed, down when not, limit your costs to the resources you need, when you need them.

    Microsoft has increased the capabilities of the Azure SQL Data Warehouse quite a few times across the last few years. I was thrilled to see ASDW separate out storage from compute, allowing customers to scale up the query, or compute, nodes independently of the storage used. Changes last year improved the ability to move data around between compute nodes as well as increased the number of concurrent queries.

    These improvements are perfect for data warehouses, and if you are looking to build a more responsive SQL Server based warehouse, you ought to take a look at ASDW. More and more customers are finding it valuable and cost effective in their businesses. What seemed to once be a niche idea has grown into a business that quite a few customers are using, with the demand growing.

    What’s more, the move to Big Data Clusters in SQL Server 2019 seems to have adapting some of this technology to the regular SQL Server product many of us use. These will separate out the storage from compute, something that should help many of us scale our systems to meet the demand of our workloads. I haven’t tried a big data cluster yet, but I’m looking forward to seeing how well one works and if it truly scales SQL Server further than I would have dreamed.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Lost Time

    Last year when Daylight Savings Time moved clocks back, I was wondering how the time change might affect different devices and situations in the world. One area that I wondered about was medicate and health care devices. While I’m sure most (hopefully all) medical device software developers have thought about this, it was something that struck me this year as I was fighting a cold.

    I was taking a medicine designed to help reduce the length of illness and was supposed to take this every four hours. That meant if I took this at 10pm, the next dose should be 2am. However, if I woke up at 2:10am, it would actually be 1:10am since the clocks would have moved back an hour. If my clock adjusted itself, then it would be 3:10am. Not a big deal for my dosage, but for some people, this might really matter.

    Certainly software can adjust for this, but can we humans? If I’m dosed every four or six hours, common time frames, both me and my caregiver might get used to a schedule. We might assume that 7am, 11am, 3pm, 7pm is the schedule. We know that 7, 11, 3 repeat throughout the day. Daylight savings time changes that. Once the clock resets, I need to move to 2, 6, 10 the next day. Or to 1, 5, 9 depending on the time of year. It would be easy for someone to think they’d missed a dose and double dose, or assume that medicine was given last out and skip a dose.

    What about sales transactions? I’m sure this is a slow time of the day and year, but still, I’ll get a skewed view of that hour. Either zero sales or double sales. What about auditing? Is this the perfect time to break into a system or perhaps game the tracking of events? If I came and went from a facility during the hour in question, could I leave before I arrived? How could we actually capture a true audit record here?

    Daylight Savings Time might have been a good idea at some point, but these days I think it’s silly and causes more potential issues than it solves problems. I’d like to see it go away and have us stick to some consistent schedule throughout the year. Standard time, savings time, split the difference. I’m fine with anything, just don’t arbitrarily move my clock during the year.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.2MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Justification Database

    I saw an article on the security at various Department of Defense facilities that control the ballistic missile systems in the US. It’s interesting reading, especially if you want to shake your head at bad practices. I think this article summarizes just how quickly the digital world has grown faster than the ability of humans to keep up and adopt new practices, especially where constant staff turnover is the norm.

    This is a good read for anyone in an organization that must plan for security and account for new staff on a regular basis. It certainly made me think a bit about how information is presented, disseminated, and the view from the other side. It’s easy for me to consider security and focus on that from my position, but for others that are tasked with other jobs, their view is completely different.

    There was one item in here that made me stop and think. There is a section, titled “No database with written justifications”, that notes that there wasn’t a database of reasons why someone was granted access. Through random sampling, the found that lots of employees didn’t have forms filled in completely with justifications and approvals. In some cases there weren’t even forms. I assume these are digital forms, but they could be paper.

    That made me think about the ways in which I’ve granted access to individuals in databases in the past. Often in small companies the request is verbal or in email, with approvals made the same way. Certainly there’s no tracking. In larger companies there might be tickets in some helpdesk system, but how do I track those back to clicking in SSMS to add a user to a role?

    This week I’m wondering how well managed your system is for managing and approving security changes? If someone was concerned that a hacker gained access through social engineering, could you track down where, when, and why someone was granted access? Are you regulated or audited? I know some people have great processes that limit the potential for abuse, but I also know that far too many people use exceptions to get work done. I wonder how many of you allow for exceptions to your process.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Ensuring Designs have Flexibility

    One of the biggest complaints from developers about RDBMSs is that they are inflexible and hard to change. This has led to many developers in the past using existing fields for a variety of purposes, sometimes putting data from multiple places into one field. In more recent times, this has led to adding or migrating data to new data stores, with the idea that the system can adapt easily to changing requirements.

    I ran across a post on system flexibility that somewhat illustrates the issue. The writing is a little hard to understand, but the author is thinking about the process of ordering food and how a system might need to handle some of the exceptions or additional changes. While the employees in a store might deal with this in an ad hoc manner, those of us building software need to think through the possible issues and account for (at least) some of them in code.

    Software typically has much more flexibility than we do at the database level, since anyone building software is usually building on top of the database system. However, those of us that design schemas need to ensure that we take care to consider the different ways that our data store will serve the needs of the end user and design in the proper structures needed to support the data that will be stored.

    In my experience, it’s important for the database developer to question the requirements of the customer. Just as a software developer might try and tease out additional features and functions that are needed, the database developer needs to carefully query the customer to ensure that there won’t be additional data that the customer assumes is being captured. We need to be aware of relationships and cardinality between the data elements, in order to better design a system that meets the current, and potential future, needs.

    Building and designing a database isn’t that hard, but it does take some attention to detail and a meticulous nature to think through how some real world situation is modeled. While I think small, frequent DevOps releases are a way to move forward, I’d also like to ensure that we capture as much detail as early as possible to minimize disruptions. After all, refactoring and changing the schema is harder in a database system than in an application software class.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.3MB) podcast or subscribe to the feed at iTunes and Libsyn.