Voice of the DBA

Tag: powershell

Boring or Scripting

Do you want to continue to perform boring, repetitive, mundane tasks as a part of your job? Many of you might not be challenged at work, or you might be burdened with a series of requests that repeat themselves over and over. They’re easy tasks, many of them probably take minutes. I’m sure there’s also a level of mindlessness that you find comforting at times with just working through a familiar task.

However many of you also get busy. You have no shortage of new tasks that get assigned to you on a regular basis. You probably also get stressed from your heavy workload at times. What do you do when you’re too busy to work through the mundane tasks, but they still need to be done? It can be a challenge to manage that burst in a workload if you haven’t prepared for it.

There is a way to remove some of the mundane administrative work from your job. It’s not simple, and it’s not going to solve all your issues right away, but over time, you can certainly reduce the burden of working on dull tasks over and over again, across multiple machines.

Learning PowerShell (PoSh) or some other scripting language. VBScript works fine, as does Perl, and there are others, but if you’re a Microsoft person, especially a SQL Server person, learn PoSh. It’s used in all the products, it’s becoming a standard for all Microsoft products, it works in the cloud, and it works with SQL Server. It takes some getting used to, and it certainly can help with repetitive tasks. It can also run all your SQL scripts for you, just in a more automated fashion.

You can grow your career, add a new skill, reduce your workload, and become more efficient at your job.

Or continue to be boring.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.3MB) podcast or subscribe to the feed at iTunes and LibSyn.

May 18, 2015
The Demo Setup–Attaching Databases with Powershell
I found another use for Powershell, one actually suggested by someone else: attaching specific SQL Server databases.

TL;DR I have a script that detaches all user databases from a SQL Server instance and then reattches certain ones. Full script at the end.

The Issue

We have a lot of demo databases on our demo VMs for Red Gate. Some specific databases are used to show things with different products, but it ends up with us having a few dozen databases on an instance of SQL Server.

That’s not the best way to show things to users, as they can get confused with so many databases. Specifically for us, we have a set of databases for one of our classes, a different set for a second class, and a third set for a third class. We do this because things need to be set in different stages for each class.

One of our sales engineers said it would be great if we could hide some databases when we didn’t need them. I immediately saw a use for Powershell here.

Approach

My approach to this problem would be this.
- detach all user databases
- attach specific databases by specifying the name of the database, and the mdf/ldf/ndf file names.
- use a batch file the user can double click on the desktop to run the Powershell script.
This seemed to make sense, and I started to tackle this on one of my machines in this manner. However because I detached all my databases first, all of a sudden working on things was a pain. As a result, I setup a new VM and created dummy databases there. I first worked on the attach piece, and then the detach part.

Detaching User Databases

This was fairly simple, and I’ve written about it before. In this case, I merely cut and pasted this code into my script.

$srv = New-Object ‘Microsoft.SqlServer.Management.SMO.Server’ $instance

#detach all user databases
$dbnames = $srv.Databases.name

foreach ($dbn in $dbnames) {
    Write-Host $dbn
    if ($dbn -ne "master" -and $dbn -ne "model" -and $dbn -ne "msdb" -and $dbn -ne "tempdb") {
      $srv.DetachDatabase($dbn, $false)

      }
    }

The first line is actually needed for both parts of the script, and we re-use that object later.

The script gets a handle to the databases object and then a collection of all the names. We loop through the collection and if we aren’t looking at one of the four system databases, we call the detachDatabase method.

Note that this means I’m in control of the instance and I know I don’t have a distribution database or anything else that might break. For me, I can safely drop everything other than master/model/msdb/tempdb.

Attaching Databases

I had to search around for some example code. I guess I didn’t have to, but the docs from MS can be tricky to put together, so I searched and found a few examples. Specifically, I ran across this post that described how to attach a single database.

I decided to begin by building up the db name and paths to the files. I started by setting a variable to the path and database name.

$sqldatapath = "C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\"

$dbn = "sandbox"

One of my databases is “Sandbox” and the path for all my database files is given as the default.

Next I build up the mdf/ldf files. In my case, I don’t have anything other than single mdf file databases.

$mdffiles = $sqldatapath + $dbn + ".mdf"
write-host $mdffiles
$ldffiles = $sqldatapath + $dbn + "_Log.ldf"
write-host $ldffiles

With these, I now can tell what I’m doing. I write the data out to the host, mostly so that if something breaks, the user can determine where. We’re all technical, but it’s nice to know what’s broken.

These are the important bits, but now I need a place to store them. At only one time in the script, I create a new StringCollection object.

$dbfiles = New-Object System.Collections.Specialized.StringCollection

I’ll reuse this object for each database. In this object, I store the database file names. I use the .Add method to get them in here.

$dbfiles.Add($mdffiles)
$dbfiles.Add($ldffiles)

Now I have all my parameters. I can call the AttachDatabase method.

$srv.AttachDatabase($dbn, $dbfiles, "sa", "None")

The documentation says I need an owner, and for simplicity, I use “sa”. I also can specify options, but I don’t care in this case.

This attaches my first database. However, I need to repeat this. I could build some loop and use some array, which is probably better, but for the sake of simplicity here, and preventing issues, I copy and paste this code multiple times. In my case, I have no more than 4 databases, for any environment, so I merely copy/paste this code and change the database name.

However, I don’t want to keep adding to my StringCollection each time. In between each set of databases I need to call, I add this:

$dbfiles.Clear()

Now I have a few simple scripts I can modify easily, and others can understand them.

The Batch File

The other thing I learned with the batch file is that it doesn’t have the same context as my editing session. I had to add a line to load the SQLPS stuff at the beginning for it to work.

Import-Module "sqlps" -DisableNameChecking

I also had to ensure the execution policy is set on each machine, but we tend to do that when we set up the machines.

Simplicity

This is the simple way. It’s really not the best way, and if these scripts change much, this is a problematic way of doing things. I really should have a loop with a list of databases in one place in the script. That way if I add or remove a database, I can easily do it.

That’s an improvement I’ll make.

Let me also say that I have a pattern of database names, and files. If I needed to handle different file locations and varying numbers of files, I think this approach actually works better. Each section of the script can be edited easily, and separately, without worrying about complex logic.

I like simple.

Scripts

The batch script is this.

powershell c:\Utilities\attach_demodbs.ps1

I call the Powershell host and give a fully qualified path to the script.

Here is one of my demo scripts, for two databases: sandbox and EncryptionPrimer:

<#

Attach Demo Databases

This script detaches all user databases and then attaches the following databases

Attaches
– Sandbox
– EncryptionPrimer

#>

Import-Module "sqlps" -DisableNameChecking

$srv = New-Object ‘Microsoft.SqlServer.Management.SMO.Server’ $instance

#detach all user databases
$dbnames = $srv.Databases.name

foreach ($dbn in $dbnames) {
    Write-Host $dbn
    if ($dbn -ne "master" -and $dbn -ne "model" -and $dbn -ne "msdb" -and $dbn -ne "tempdb") {
      $srv.DetachDatabase($dbn, $false)

      }
    }

$dbfiles = New-Object System.Collections.Specialized.StringCollection

$sqldatapath = "C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\"

$dbn = "sandbox"

write-host "Instance: " $srv.Name
write-host "Attach " $dbn

$mdffiles = $sqldatapath + $dbn + ".mdf"
write-host $mdffiles
$ldffiles = $sqldatapath + $dbn + "_Log.ldf"
write-host $ldffiles

$dbfiles.Add($mdffiles)
$dbfiles.Add($ldffiles)

$srv.AttachDatabase($dbn, $dbfiles, "sa", "None")

$dbfiles.Clear()

#attach staging
$dbn = "EncryptionPrimer"

write-host "Instance: " $srv.Name
write-host "Attach " $dbn

$mdffiles = $sqldatapath + $dbn + ".mdf"
write-host "MDF: " $mdffiles
$ldffiles = $sqldatapath + $dbn + "_Log.ldf"
write-host "LDF: " $ldffiles

$dbfiles.Add($mdffiles)
$dbfiles.Add($ldffiles)

$srv.AttachDatabase($dbn, $dbfiles, "sa", "None")

$dbfiles.Clear()
March 16, 2015
Parsing SQL Saturday Data – Getting Titles from the XML document

I’m continuing on with my project to grab SQL Saturday data and automatically insert it into a SQL Server database. In this piece, I’m picking up from the last one where I had a loop to load all XML documents in a folder based on a pattern.

This time I want to query the XML and get out specific elements and capture them.

The Source

The XML source looks like this for the sessions:

</event>
<event>
<importID>2102</importID>
<speakers>
    <speaker>
      <id>2102</id>
      <name>Jason Strate</name>
    </speaker>
</speakers>
<track>Track 3</track>
<location>
    <name>2520C (Conference room)</name>
</location>
<title>Using XML to Query Execution Plans </title>
<description>SQL Server stores its execution plans as XML in dynamic management views. The execution plans are a gold mine of information. From the whether or not the execution plan will rely on parallelism to what columns are requiring a key lookup after a non-clustered index seek. Through a the use of XML this information can be available at your fingertips to help determine the value and impact of an index and guide you in improving the performance of your SQL Server databases. In this session we’ll look at how you can begin to understand and query the structure of the execution plans in the procedure cache. Also, we’ll review how to uncover some potential performance issues that may be lurking in your SQL Server.</description>
<startTime>9/18/2010 12:15:00 PM</startTime>
<endTime>9/18/2010 1:30:00 PM</endTime>
</event>
<event>
<importID>2109</importID>
<speakers>
    <speaker>
      <id>2109</id>
      <name>Jason Strate</name>
    </speaker>
</speakers>
<track>Track 4</track>
<location>
    <name>2520D (Seminar room)</name>

I’m showing the end of one element, one whole one, and the start of a third. There is a lot of extraneous information in the document that I don’t want (for now). As a result, it’s not as simple to query this as I’d thought before. Especially as I’ll want to capture each session title and insert it into a database.

I decided to use a SelectNodes to get to the <event> nodes and then loop through them. The code looks like this:

# get the event node
$sessions = $doc.SelectNodes("//event")

# loop through the various //event nodes
foreach ($session in $sessions) {

Note that this is inside of the code from the previous post.

Inside of this loop, I decided to create another loop. Initially I didn’t, but that made it more difficult to determine the end of the event node and capture the values, especially the speakers. As a result, I have a sub loop at well:

# probably a better way, but I wanted to loop through the various elements and only pick out certain ones
foreach ($detail in $session.ChildNodes) {

If anyone has a better way, let me know. I’ll have all the code below, but this technique allows me to look for specific nodes. I know I could query for them, but since I’m looking for a few specific items, I thought I’d do this rather than multiple queries later.

Get the Title

I actually need the title and the speaker child node, but I’m doing titles only here. Here’s the whole node loop code:

foreach ($detail in $session.ChildNodes) {

# If we’re on the title node, get the value
if ($detail.Name -eq "title") {
    $title = $detail.’#text’
   }

if ($detail.Name -eq "speakers") {
    #placeholder
   }
#end foreach for $detail
}

Here if I have the title element in the foreach loop, I capture it. This allows me to use this variable later. I’ll go into the speaker code later, but for now, I left a placeholder.

That’s really it. At the end of the outer foreach, I write out the $event and $title variables. This gives me a nice output to the screen. From here I can easily substitute some ADO code to send this to SQL Server instead of the write-host, but that’s a good programming technique for me to see if I’ve got the data I want.

As you can see, there are sessions that I don’t want, but there’s nothing in the data for me to tag them as non-educational sessions. I’m not sure I care, since the speakers associated with these won’t impact my results for reports, so I’ll leave them.

Next Steps

From here I need to extract the speakers before I insert data into SQL Server. That will be the next step before I create the database and then insert data.

The Code

Here’s the entire code:

#ViewXML_Basic
# View XML file data from a website

$debug = 0;
# counter for events
$i = 1

#when do we stop?
$loopend = 400
$baseURL = "E:\SQLSatData\SQLSat"
$loop = 1
$doc = New-Object System.Xml.XmlDocument

do {
#start large loop

# get the filename
$sourceURL = $baseURL + $i + ".xml"

# debug information
if ($debug -eq 2) {
    Write-Host $sourceURL
    }

#test the path first. If it exists, load the XML
if (Test-Path $sourceURL) {
    $doc.Load($sourceURL)

    #trap the event number. This will be the ID I use in the database table.
    $event = "SQL Saturday #" + $i

    # get the event node
    $sessions = $doc.SelectNodes("//event")

    # loop through the various //event nodes
    foreach ($session in $sessions) {

    # probably a better way, but I wanted to loop through the various elements and only pick out certain ones
    foreach ($detail in $session.ChildNodes) {

      # If we’re on the title node, get the value
      if ($detail.Name -eq "title") {
        $title = $detail.’#text’
       }

      if ($detail.Name -eq "speakers") {
        #placeholder
       }
     #end foreach for $detail
     }

    write-host $event ": " $title

    # placeholder – insert into table here. $i, $title

    $title = ""
    $speakers = ""

   #end foreach for $sessions
   }

   # end test path
   }
# increment loop
$i++

#end outer loop
} while ($i -lt $loopend)

write-host "end"

February 25, 2015
Powershell Quick Parameters for Scripts
I was working on a script recently to manage a particular process and wanted to make it generic by allowing the user to pass in a parameter. I have seen lots of examples, especially those that work with SQL Servers, using text files and other items as parameters, but in this case I wanted an easy, quick, command like parameter.

This post looks at what I chose to check parameters. I had a couple requirements.
- display message if no parameter is passed in.
- display some help if /? is passed in.
I know that my cmdlets should contain help from the PoSh command line, and I’ll get to that. For now, I’m managing things the way I was taught when I wrote C. A /? should get me help.

$Args

I did a little research on parameters and found a few things, but decided to use the $args variable. This is an array of undeclared parameters. I grab the first value (the only one I care about like this.

$instance = $args[0]

Note the [0]. As with many things in Computer Science, we’re zero based arrays.

I could allow for other parameters, but this gets me what I want.

Testing

The test for /? is easy. That’s like this:

if ($instance -eq "/?") {
write-host "Please enter the instance you wish to detach all databases from as a parameter."
}

If this is equal to my help request, write something out.

Next I needed to add another test. In this case I found that I could easily look for NULL variables, or blanks, with the !. As in this:

if (!$instance -or $instance -eq "/?") {
write-host "Please enter the instance you wish to detach all databases from."
}

That worked well and lets me remind myself if I’ve forgotten to pass in a parameter. The one thing I experimented a few times with was the OR clause. I tried these, none of which worked:
- if (!$instance OR $instance -eq "/?") {
- if (!$instance) or ($instance -eq "/?") {
  if (!$instance) -or ($instance -eq "/?") {
A little experimenting got me to remember that PoSh is fairly consistent, and the plain -or should work inside the parenthesis.

Everything Else

When I first ran this without a parameter, my script froze. That’s because I hit the IF clause, wrote out the message, and then executed.

Fortunately I’ve done this type of stupid programming before, so I added this:

if (!$instance -or $instance -eq "/?") {
write-host "Please enter the instance you wish to detach all databases from."
}
else {

The rest of my script fits in the else clause.

Reference
- Automatic Variables – https://technet.microsoft.com/en-us/library/hh847768.aspx
- https://rkeithhill.wordpress.com/2007/01/06/checking-for-a-null-or-empty-string/
February 19, 2015