Author: way0utwest

  • The Writeable Warm Standby

    I saw a question recently that went like this: I get one full backup from FTP. I’ll get daily log backups through FTP after this, but never another full. I need to restore this daily log backup and allow the group receiving the database to read/write the copy, and then reset it with the new log backup overnight.

    First, this is untenable. At some point you’ll have some issue with transfer, lose a log, or the database will go corrupt. I can guanantee you that at some point you will need another full backup. Not every week, or even every month, but you will need one.

    Second, this is a tough situation. I saw some answers, which I agreed with, but I started thinking about ways to get that data moved. My first thought it use STANDBY and move the data every day to a clean database. I’ve done this before, and in the GB range, even 100s of GB, this can work. It helps if you can whack indexes and constraints on the destination, but a copy of data table-by-table goes fast.

    However then I thought about other ways. You can’t take a backup of a standby database, nor can you take a snapshot. However while searching, I saw an answer to this post on SO.

    TL;DR: copy the mdf/ldf to a new database.

    That was interesting, so I decided to test it. Turns out, it works pretty well. HOWEVER, it’s dangerous, and I think you should be very careful about this. I wouldn’t count on this being production stable, and certainly not data stable. You better have other copies of this data.

    Here’s what I did. First, create a database with some data.

    CREATE DATABASE mydb;
    GO
    USE mydb;
    GO
    CREATE TABLE mytable(id INT);
    GO
    INSERT mytable SELECT 1;
    GO
    

    Next, let’s back this up, take if offline, and then copy files.

    USE master;
    GO
    BACKUP DATABASE mydb
     TO DISK = 'mydb.bak';
    GO
    

    Now we can copy the files to new files. There are UAC issues here, so you’ll need to add some rights to the files if you do this regularly. I left a comment in my script, but I actually did a CTRL+C,CTRL+V in the data folder, allowing the UAC permissions to work.

    2015-12-11 08_42_22-Photos

    Once that was done, I had new files:

    2015-12-11 08_28_15-Photos

    I tried to run at attach, but got a permissions error:

    Msg 5120, Level 16, State 101, Line 30
    Unable to open the physical file “D:\SQLServer\MSSQL11.MSSQLSERVER\MSSQL\DATA\mydb_reporting.mdf”. Operating system error 5: “5(Access is denied.)”.

    The solution I found was to run SSMS as an administrator. Annoying, but it works.

    At least for the OS error. However, then you get this:

    Msg 1824, Level 16, State 1, Line 35 Cannot attach a database that was being restored.

    You can’t do this. At least not easily.

    You can do this. First, delete the files you copied over, then run this:

    CREATE DATABASE mydb_reporting
    go
    alter database mydb_reporting set offline;

    The next step is to delete the MDF and LDF files, which will be mydb_reporting.mdf and mydb_reporting_log.ldf  by default. I could specify other names, and would if this were something I needed to script.

    Once those files were deleted, I’d next copy my files again and rename them. That would result in this:

    • mydb_reporting_base.mdf –> mydb_reporting.mdf
    • mydb_reporting_base_log.mdf –> mydb_reporting_log.ldf

    Now I can go back to SSMS. In SSMS, I do a simple ALTER.

    ALTER DATABASE MYDB_Reporting SET ONLINE

    Then I can run this:

    2015-12-11 09_03_28-Start

    I have a copy of my database. Can I apply logs and move forward? Let’s try. First, let’s add data and make a log backup

    USE mydb
    GO
    INSERT mytable SELECT 99
    GO
    
    BACKUP LOG mydb TO DISK = 'mydblog.trn'   
    GO
    

    Next we restore again. We also set the databases offline again.

    RESTORE LOG mydb_reporting_base
     FROM DISK = 'D:\SQLServer\MSSQL11.MSSQLSERVER\MSSQL\Backup\mydblog.trn'
     WITH  MOVE N'mydb' TO N'D:\SQLServer\MSSQL11.MSSQLSERVER\MSSQL\DATA\mydb_reporting_base.mdf'
        ,  MOVE N'mydb_log' TO N'D:\SQLServer\MSSQL11.MSSQLSERVER\MSSQL\DATA\mydb_reporting_base_log.ldf'
        , STANDBY = 'D:\SQLServer\MSSQL11.MSSQLSERVER\MSSQL\Backup\undo.log';
    GO
    USE master
    GO
    ALTER DATABASE mydb_reporting_base
     SET OFFLINE;
    GO
    ALTER DATABASE mydb_reporting SET OFFLINE
    GO
    

    Once again it’s file copy time. The UAC comes into play again as I copy and rename the _base files. However once that’s done, things work.

    2015-12-11 10_17_02-Photos

    This works, but I am not recommending this as something you should do, especially for critical systems. This can work, but it’s dangerous, and really not supported by MS.

  • The $90,000 Laptop

    A hospital got the opportunity to pay $90k for a lost laptop.

    There’s no excuse for this. If you have a Windows laptop, enable bitlocker today. If you have OSX, setup FileVault. If you’re on Linux, choose dm_crypt or something else. Go ahead, get that setup, save off your keys as a backup, and come back. I’ll wait.

    Now, don’t you feel better? I’m sure you do, and you are somewhat more protected from the random theft, misplacement, or loss of your laptop. All of those things happen regularly. Not to each of us, or many of us, but as the collective set of developers and DBAs around the world, we lose laptops regularly.

    Certainly some of us have precautions like never carrying data around. That’s good, and I’d recommend that. For those that need to develop on the go, they might have a curated set of test data we can use for development. That’s fine. We should make an investment in building test data and have that data used for unit, integration, and system testing.

    We should have investments in ensuring that our systems can not only be encrypted, including encryption of backups and networks, but that we can restore those systems in disasters. Make the investment in ensuring recovery, and everyone is more likely to accept encryption. We need to invest in a process for managing keys, revoking them, and re-issuing them. We need to be sure we can upgrade encryption algorithms over time. What is secure today might be easily broken tomorrow.

    I’d urge you to make all those investments, but until you do, at the very least, encrypt your laptops and desktops. While any random theft could result in a lost laptop, it’s not unheard of for anyone walking through your office to pilfer a drive lying around, or even one inside a desktop.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Graduation – An Exciting Day

    It’s quite an exciting day for me today. My oldest son graduated college this morning, finishing his Bachelor’s degree and beginning the next phase of his life.

    I’m quite proud of him and just wanted to put a few memories of him down related to data. He was the first kid in our household to seriously use a computer for school, and while his younger siblings are much more comfortable with computing than he is, we had some good times.

    • Setting up a networked computer and trying to limit access to various topics and sites on the Internet (a complete fail, BTW)
    • Reinstalling OSes for him after various viruses or other sites rendered computers unusable.
    • Discovering all my bandwidth was gone as my son found the wonders if BitTorrent.
    • Buying him a computer for Christmas, the first time I’d done that for someone.
    • Helping him learn some SQL as he studied GIS in college.

    It’s been an amazing time watching him grow, learn, and mature into an adult.

  • Security Decisions

    How many of you have written code that results in a security issue or data breach? It’s possible many of you have, but are unaware of the issues. I’m sure a few of you have been hacked, have had to clean up data or code, but often it’s not your code. Maybe it’s something you inherited. Maybe it’s code your team wrote together. In any case, the feedback loop between your action (writing code) and the result (a security incident) isn’t tightly coupled.

    I ran across a post from Bruce Schneier on how people learn about cybersecurity. The piece links to a research paper, and it’s an interesting read. It turns out the researchers see most non-experts learning from news, friends, and websites, often with biases towards actions that have had immediate negative consequences, but not necessarily those that are more serious.

    That has me wondering about us, as supposed expert, or expert-ish, developers and DBAs. How many of us get security training, or get this training updated? How many of us learn from friends, or websites, and re-use this knowledge over and over in our work, not necessarily understanding, or unsuring, that we are building in strong security into our systems. I suspect many of us just try to get by, writing the minimal level of security that “works” for us, not really understanding how there might be flaws or holes in our system.

    Our code, our configurations, our systems have much farther reaching impact than ourselves. In some sense, I think that a fundamental broken concept of information technology is the lack of security practices and architectures being built into our platforms and applications from the start. While convenience may get the boss off our back, or allow greater profit for our companies, it’s not helping our industry, or even our companies in the long term.

    I just wish I had some idea on how to change things.

    Steve Jones