Voice of the DBA

Category: Editorial

  • The Pressure to Compromise Ethics

    There have always been those that seek to defraud, deceive or mislead their customers. In order to do this, someone inside an organization has always been willing to pressure employees to compromise their ethics and morals. Construction companies may use substandard materials or ignore standards for construction. Financial companies falsify performance records of products. Medical professionals may order unnecessary tests to charge more. Perhaps one of the common areas many people feel cheated is with used vehicles, where there are numerous stories of deceit from companies and individuals. I’ve had my own experiences with cars to support this.

    It may be no surprise to you this is also happening in the digital world, which seem fundamentally more disturbing to me. After all, software is constantly changing in a way that many other industries don’t. We can fundamentally rewrite the rules under which systems work by deploying new code, something many of us do on a regular basis. What’s more, we have rules and regulations that fundamentally prevent us from disclosing how the systems work, limitations that don’t allow for anyone to easily audit or evaluate what the programming might be doing.

    This is different than a car, where we can examine the components and test them, or have a third party perform this for us. It’s different than health, where we can solicit second opinions. Even construction has inspections from independent groups that can verify some specification is met. However, none of that exists for software. In fact, our industry resists remotely attempting to implement any sort of ethics by limiting access to source code.

    There’s an article that lists some of the issues developers have encountered. It also talks about teaching ethics as a way to reduce the impact, which is a good idea. But without any sort of accountability or transparency into the actual code, I’m not sure ethics will help much. There are always people that need a job, and may be willing to write code that continues to provide a paycheck because they aren’t sure they can get another job. I know most of us would like to think we wouldn’t succumb to pressure, but it can be hard to be sure what you’d do until you are confronted with a particular situation.

    I do think this is a problem that will grow, and become more pervasive in the digital world, precisely because no one really knows what software is doing under the covers, and it’s hard to even determine what might be happening if you could access the source code. After all, can you be sure that code you get is actually what is running? You can, but not easily.

    What I’d like to see is some sort of framework that allows us to specify the behaviors of our software, along with the data movement and handling so that end users would have an understanding of how the software should work. Maybe some automated way of producing a “contract” based on the code. Then we could resolve disputes without ever needing to examine the source.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Finally, Create or Alter

    There are lots of reasons to upgrade to SQL Server 2016, but this is the one for me. We finally get a CREATE OR ALTER statement in T-SQL. This not only makes lots of code easier to write, it means that the ways in which you might script and schedule your future deployments will be cleaner. This is an exciting change for implementing a simpler and easier Continuous Integration/Continuous Deployment system in your organization.

    It’s not perfect news for a few reasons. First, this is a SQL Server 2016 addition to T-SQL only. That means until you have most of your applications have moved to SQL Server 2016 SP1+, you won’t be able to use this construct. That’s OK, because it will mean that at some point most of our instances will be on SQL Server 2016 SP1 or later, and much of our code will be cleaner. We won’t resort to including IF statements in our deployment scripts. We won’t need to create stubs of procedures and functions so our code is embedded in an ALTER script. In essence, you won’t need to maintain two separate code constructs to make a change.

    This isn’t perfect, nor is it complete. We still don’t have CREATE OR ALTER for tables. That’s the place where I’d really like to get a consistent way of coding items. What I really want is a complete view of the table each time I change it. By this I mean that if I create a table like this:

    CREATE TABLE Students
(
    studentname VARCHAR( 200),
    status TINYINT
);

    Then I want to be able to add a column like this:

    ALTER TABLE Students
(
    studentname VARCHAR( 200),
    status TINYINT,
    DOB DATE
);

    Or alter a column like this:

    ALTER TABLE Students
(
    studentname VARCHAR( 200),
    status BIT,
    DOB DATE
    );

    Or better yet, have a CREATE OR ALTER for tables.

    I know this might be asking for a lot, but I really think that we ought to get a consistent way of coding databases so that we can reduce the mistakes and make our systems easier to understand. I’m sure this may require substantial engineering, not to mention a great deal of understanding of how this would actually affect our systems when run, but it would certainly make our code cleaner.

    I doubt we’ll see these kinds of changes, at least not until we have an ANSI standard that encompasses them, but I would hope that as an industry we would mature and improve the way we work with databases, not remain bound by tradition and history.

    Steve Jones

     

  • Hire the Right People

    I’ve seen this list of why projects fail going around the Internet for awhile (recently from Brent Ozar Unlimited).  There are a number of items in here, and if you substitute some other field for the data scientist part, you might have seen some of these issues as well. One of the items that really struck me was number 5, which is titled: You shouldn’t have hired scientists. The other part of the slide is that ETL needs data engineers (whatever those are) and reporting needs BI analysts.

    I with that we had good ideas and definitions of what different jobs are supposed to do in a company, and what skills are helpful. The more I look at job descriptions and responsibilities, the more I find that DBA, analyst, data xxx-anything are just amorphous concepts that vary dramatically from organization to organization. In fact, since the title that one has often determines pay scales, I think that we start to covet titles and work towards getting a title, regardless of the work we do. This isn’t helpful for anyone, least of all HR departments that try to match salaries to value and pay for particular jobs.

    There isn’t a great solution here. Our industry is young and changing more rapidly than any other in history. New jobs are created out of thin air, to meet the changing face of software development and system administration. Build master, data scientist, blockchain engineer, and a few others didn’t exist when I started working in technology. Whether those jobs are needed in an organization might not relate to whether or not anyone has those titles, or if they perform the tasks we might expect of those positions.

    I think that many of us are expected to be able to perform any task tangentially related to our position. Or we should be able to learn it quickly. DBAs should understand replication as easily as they might Availability Groups and configuring Extended Event sessions. A developer should be able to write C# as well as T-SQL. Both should be able to use SSIS to import and export data. While I’m sure many of us could do those tasks at a rudimentary level, are we really competent and capable at each of those tasks? Maybe enough for our organizations, maybe not.

    When we embark on projects, there may be needs to accomplish tasks outside of the core competency of our staff. That’s fine, and since many people in technology are willing (and excited) to learn new skills, this situation may be perfectly acceptable for our project. When there are core skills critical to the project, such as a deep understanding of large scale ETL processes, we are probably better off hiring people that have those skills or investing heavily in just-in-time training for existing staff. Hoping that the average DBA or C# developer can just “pick up some tips” is a recipe for project failure.

    Steve Jones

  • Monitor All the Changes

    Can you monitor every configuration change and setting on all your SQL Server instances? Can you get an alert every time code changes, or even if an option for an object changes, such as the changing of an EXECUTE AS or the rebuild options for an index? Do you want to know about every security change (new logins, grants, revokes, etc.). Can you keep up with every alteration of a SQL Agent job?

    You certainly can, but across any busy enterprise, all these alerts might result in a constant stream of items to review each day. In fact, in some companies, the volume might be high enough that this becomes a full-time job for someone. I’m not sure many of us think that it’s worth an employee’s time to actually review every change.

    Even if you decided it’s worth reviewing every change, is it possible to do a good job actually doing so? Too many alerts usually result in an individual starting to treat all of them as though they are the same priority. It becomes hard to differentiate what’s important to review and what’s not over time if you are always looking at a stream of changes across disparate systems. This is one reason why I never want to get success message, but even failure or change messages can be problematic when the volume is high.

    What can you do in a situation like this? Certainly there are alerts that are critical and need to be addressed right away, aren’t there? There are, but they are probably few. Making the decision about which items are important enough to review daily can be hard. I’ve typically only wanted critical alerts for backup tasks (after some retry) and privileged security alterations (add/change/remove sysadmin/securityadmin/processadmin/serveradmin). Those are items I need to take action on. Most other items, such as failed index rebuilds, job schedule changes, configuration alterations, I just want to capture and log.

    In one of the SQLskills newsletters recently, Paul wrote about an issue where replication settings had changed. The distributor had gone from 72 hours to 72 days, resulting in memory pressure for the workload. While I know this is a problem, is this the type of alert you’d define as critical? I’m not sure I would. In fact, this is the type of alert I’d want logged as a set of changes on this system so that I could review it when it seems that the system is not running as smoothly as it otherwise might be. In fact, this is the type of problem I’d hope I’d catch through performance monitoring, which might pro-actively catch performance degradation that would lead a DBA to review changes and metrics before a user reported the situation.

    Ultimately I want monitoring systems to help me find issues, and only notify me when I might need to take some action. To do this, the system might need to capture everything, but I want most of the items filtered until such time as the information might actually help me solve an issue. This takes some time to setup and tune as you discover holes in your monitoring, or you find that too much data is being passed on. Don’t be more afraid to add more data to capture, but be ruthless about removing extraneous notifications from the system. This is a case when too much information sent to the administrator is as bad as too little.

    Steve Jones