Voice of the DBA

Tag: administration

  • Quickly Copy Data

    How quickly can you copy data from one place to another?
    How quickly can you copy data from one place to another?

    There’s no magic solution for many of the problems we face in SQL Server. Whether we are trying to maintain concurrency under high write loads, restore databases quicker, or ensure extremely high availability we need to make trade-offs. Sometimes those tradeoffs are frustrating, and people look to other solutions like NoSQL, not realizing that they are just making different trade-offs.

    I see the frustrations and comments constantly on the SQLServerCentral forums, and one recent discussion was no exception. In this particular debate, a user was frustrated by the requirement to copy backup files across the Internet to another location. The transfers were taking too long and the poster were trying to find some magic way of decreasing the transfer time. Increasing the bandwidth, the simplest solution, wasn’t an option, and that usually isn’t easy to do in a corporate environment. The poster seemed to think there had to be some magic way of reducing the time it takes to copy large backup files, and there isn’t any magic solution. The things that help reduce transfer time are often the common things we think of.

    Compression is an obvious solution. There are numerous free programs to compress files and even software that will compress the backups as they occur. There are techniques in Windows for speeding up copies, though many are Windows version dependent. Beware, however, since copies from your server might cause you memory issues. However there might be other, more creative solutions that people come up with.

    Imagine that you track the transfers, keeping meta data about the process. Perhaps you can find a way to send less data overall the next time. In some sense, this is the idea behind log shipping. If you only send the changes (the transaction log backup), you might save a tremendous amount of time and resources. That assumes, of course, that the changes in a period of time contain less data than the overall database. That might not be the case in some  workloads.

    There are other services that might help. Imagine that you back up to the cloud, either because you use a tool, feature, or service if your application exists in the cloud. Spinning up a new instance with a copy of that data in the cloud might eliminate the need for transfers completely if the other location can just access the data in the cloud.

    I’m sure some of you have other ideas, and I’m sure many of us would like to hear what creative ways you might have for moving data between locations.

    Steve Jones

    The Voice of the DBA Podcast

    No podcasts today due to some personal issues. The podcasts should return tomorrow.

  • Fear Fear

    Are you too fearful or just paranoid enough?
    Are you too fearful or just paranoid enough?

    I’m a conservative DBA. I get nervous when backups aren’t running, code isn’t in source control, and developers have access to production systems. I’ve had too many late night pages and weekend phone calls, not to mention many extra hours spent in the office from changes to systems that didn’t go well. That latter item leads me to limit the number of changes I make to systems whenever I can, including avoiding applying Cumulative Updates to SQL Server.

    When I read an editorial from Glenn Berry, I had to stop and think of whether or not I had a healthy respect for the problems that can occur from change, or if I was being overly conservative (or fearful). Glenn makes a good point that so many people do not upgrade or change their drivers, firmware, or other software. People don’t patch their SQL Servers, even with Service Packs. I’m sure some of that is fear, but some of it is neglect as well.

    For me the decision usually comes down to examining the reward/risk ratio, trying to understand if improvements are balanced by the risk of downtime. I do value stability above new features, mostly because if problems do occur, I will be the person fixing them. That doesn’t mean I avoid all changes. I think Service Packs need to be installed, though not necessarily the first month. I’ve also come to embrace some of the continuous integration (CI) and continuous deployment (CD) ideas as ways to both reduce a software inventory as well as hold developers to a higher quality standard. However if you want to deploy (and perhaps patch) in a continuous deployment environment, then you should ensure that your CI process performs strong checks and make sure your developers are holding themselves to a high level of quality.

    We change the way we work, and the tools we use in technology often. Change is a concept we embrace, and we should since the ways in which our systems work are regularly changing. Bugs are patched, new techniques and tools are developed that should make us more efficient and productive. Those don’t always work, and we should be wary, but we should also not fear change. We should evaluate each new possibility with the attitude that our decision to move forward “depends.” It depends on the ease with which we can integrate something or apply a change, and the ease with which we can roll back our changes if they do not perform as expected. It also takes practice to ensure that all those things are easy.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • The Speed of Azure

    We should be able to deploy a new SQL instance as easy as this.
    We should be able to deploy a new SQL instance as easy as this.

    As database administrators we seem to be slow to embrace new technologies and paradigms. There was a lack of enthhusiasm from DBAs for SANs years ago, and virtual machines more recently, at a time when many other technology professioanls were embracing these ideas. Even today there is resistance from some people, and sometimes with good reason. SQL Servers are not like other servers and have much different hardware requirements. Too often the virtual machine and storage administrators do not appreciate that SQL Servers need different architectures.

    Lately the cloud services push is seeing lots of resistance from DBAs. Various vendors and media hype the idea and potential savings, which then convince management that systems need to be moved or built in the cloud. There are some applications that fit better in the cloud, but not all of them. I certainly don’t want sensitive information in the cloud, at least until we work out some of the legalities for who owns, controls, and responds to subpoena about data.

    The applications that make sense seem to be those that are distributed with lots of paying clients. I ran across a blog that looked at some of the companies that have moved into the Azure cloud with pieces of their businesses and been quite pleased with the results. Quite a few of these are cost-scalable by clients, meaning that a new user or customer is paying some fee that makes it economical to add new databases and servers for that client. If they leave, you can shut down their server.

    It’s definitely easier to start an application in the cloud rather than move an existing one. The architecture is different from an on-premise application, and that means code changes. Not something many companies want to engage in, given the past success of such projects. However the cost savings can be significant for new projects, if those new projects involve investment in equipment, facilities, people, or some combination of all of those.

    However one of the biggest items I see mentioned why companies like cloud platforms like Azure is the speed of deployment. To me that means that the IT infrastructure people, from storage to admin to DBAs, are falling down on the job. There’s no reason we can’t deploy new machines as quickly as Azure these days.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Starting with Azure

    I’ve been meaning to do this, and I finally had the chance recently to start working with Azure. Red Gate is investing in cloud tools, and more development tools, so it’s something I need to work on.

    I went to the Azure page, and noticed a “Member Offers” link.

    azure_a

    When I clicked through, I could see the benefits. Below this screen shot is the list of stuff you get, depending on your MSDN subscription. You can see the exact benefits on the MSDN benefits page.

    azure_b

    I clicked the activate link and was taken to the Windows Live login page. After signing in, I got a page that asked me to create an account. It listed my specific benefits and the requirements for an Azure account.

    azure_c

    Since I have a mobile phone and a credit card, I clicked the arrow to proceed. The first thing is to verify your account with a mobile phone. I assume this gives me some sort of two factor authentication as well as a way for them to reach me for billing issues.

    azure_d

    After receiving a text and entering it, I was taken to the billing area. I know this freaks many people out, however I’ve had a number of friends use the MSDN subscription trial and not been billed after 5 or 6 months. I’ve had a few that have been billed, but it’s been low amounts, in the $5-10 range. Grant Fritchey (b | t) is one of them, and with all the writing he’s done on Azure, some presentations, and research for Red Gate, he hasn’t had issues.

    It’s your career, it’s worth $5 a month. If you find you’re spending more, shut stuff down. I’d give it a try, however, if there is any chance your career will include work in the cloud.

    azure_e

    Once you enter a credit card, they’ll get to work.

    azure_f

    The welcome screen gives you lots of options. I’m sure I’ll end up here quite often across the next few months as I try to be sure that I’m not running up my cost. I can expense it back to Red Gate, but I do try to spend wisely.

    azure_g

    What now? There’s lots of options.

    azure_h

    I have a few things in mind, but I’ll probably start with a few light experiments based on some presentations I’ve seen. Buck Woody has a nice set of resources to get me started, and I’ll likely give one of these a try.