Voice of the DBA

Tag: software development

Masking Data

This editorial was originally published on Mar 7, 2008. It is being re-run as Steve is out of town.

I saw an interesting thread awhile back where one of our very talented community members was asking about how to go about altering data in an application for a demo. It’s a valid scenario and one that I’m sure many people have run into at some point in their career. You want to show data that’s somewhat real so that it showcases the application and what it can do, but you don’t want to show real names, amounts, or any identifying information.

A bit of a quandary and it seems that many people solve it in one of three ways. They just use test data, which is a very small set of data and doesn’t show as well. Or they may alter everyone’s name to something like “Steve Jones”, all phone numbers to 555-555-5555, etc., which looks funny.

Or you just show the production data and wink and say “I don’t usually do this, but since you’re such a valued client…”

So for the Friday poll: Do You Alter Production Data When It’s Copied?

Meaning when the data gets moved to a non-production system (demo, test, development, etc.), do you alter the data and obfuscate it to remove any identifying information. Make it “safe” data that can’t be used to somehow compromise your production system.

It’s a good practice, and one that I used to follow at a couple companies. I didn’t have any tools, but I did write scripts, load a few base tables of names, and then run those scripts as part of the restore job. They would randomly reassign new names to people, companies, addresses, etc. We would also redo phone numbers in sequential orders (555-555-0001, 555-555-0002, etc), and even randomly add products to sales or amounts to financial figures. It wasn’t perfect, and if you worked on the production system a lot you could guess which people were which, but it worked well for testing and client demos.

I actually ran into a product recently (Camouflage) that does this and it’s a great idea. It’s something that quite a few companies should be implementing to ensure that their non-production systems are that much more secure.

Steve Jones

October 28, 2015
From Great Idea to End Result

This editorial was originally published on May 5, 2011. It is being republished as Steve is at the PASS Summit.

What’s the time for you IT department to get from great idea to a resulting application? This is a very good piece from CIO magazine that finds many IT departments are seen as too slow. However there are a number of companies that are trying to innovate and find ways to increase the speed at which IT departments can deploy an application and respond to a business need.

One great quote in there is “velocity is more important than perfection”, which is a tenet that I have found to be very true over the years. It’s not that you throw junk out that isn’t well built or tested, but that you don’t try to meet every possible requirement or handle every little issue. The system has to be secure, handle errors, and meet the basic requirements, but it’s more important to get something done and in production than to have it perform and scale perfectly.

Is that heresy to the developers and DBAs out there? Perhaps, but I think this methodology has to go hand in hand with another mantra I heard fromJason Fried: do more of what works and less of what doesn’t. In this case if a system shows promise and starts to get heavy use, it receives more resources and perhaps gets refactoring in real time, even as it gets enhanced with new ideas.

“You want IT to be in constant test-and-learn mode”, another quote showing that IT needs to be working closely with the business to try ideas, learn from them, and move forward. The Agile style of development applies, and in some sense I think this is the future for the strategic IT department of the future.

For the data professional this means that you must learn to model quickly, and with an eye towards a flexible design that might need to change regularly. We need to understand the businesses we work in better so that we can anticipate how requirements might change.

Management has to buy into the idea that applications will not be perfect, they won’t be polished, and most importantly, they are essentially prototypes that either need to have addition resources spent on enhancements or they should be abandoned quickly. However I think this is a great way to develop internal applications that can provide a nice ROI, and be a more enjoyable way for developers to work.

Steve Jones

October 27, 2015
Aging Software

I wrote The Age of Software awhile back and noted that supporting previous versions of software isn’t necessarily a good use of resources for development teams. I especially think this is true of SQL Server. But does that mean we should abandon aging software platforms?

It’s a tough question. I’ve certainly talked about the case for upgrading, and the reasons why you might not. For any particular instance, however, I think that each of you has to make the case about whether the software works, or it doesn’t.

If it works, then it seems many of us will live with the old software and keep it running. As late as a few years ago I knew a company running SQL 6.5 with a piece of software built in 1996 and last patched in 2001. However this software ran a building key card system, and there wasn’t a good case to be made for upgrading.

For a software developer, however, when you look at aging pieces of software, even those that customers may pay for support on, is it worth maintaining skills and support? If you don’t have staff turnover, then perhaps. If you don’t, I do think that it might be time to let the product die.

I’m torn on the way we deal with software in our world. On one hand, I’d like to see customers given source code for end of life platforms in order to support themselves if they wish. On the other, I understand the IP concerns, and business case to let software die.

Ultimately I’m mostly OK with the current way most vendors support software. If it works for a decade and support ends, I can continue to use it. Until it doesn’t work, and then I am glad that most vendors have an upgrade for me.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and LibSyn.

October 7, 2015
Exercism

I’ve been looking to work on my programming skills a bit and try some new languages. I’d like to grow my career in a few different ways, as well as investigate where some of the new languages and platforms might be useful for data analysis. As I’ve talked to some developers and been looking around, someone recommended Exercism.io.

Exercism.io is a site that allows you to practice coding exercises and get feedback from others. When you visit the site, it’s an interesting look, and invites you to log in through GitHub. If you’re not a member of GitHub, and you’re a developer, you probably are making some sort of mistake.

Once you log in, you really need to download the command line client and execute it. Once you do, you can configure it to connect and download exercises. Each of these is placed in a folder, as shown:

For each of these languages, you get an exercise that you need to complete in that language. I’ve been playing with Python, and I had a first exercise of hello-world. In the python folder, were my exercises (you can see I’ve moved on).

The hello-world folder had a read me and a test file (it’s since gotten my program in there). The Readme has instructions and the test file is a set of tests that can be executed to check your program.

When you pass the tests, you submit your solution from the command line client. The submissions appear on the website and people can comment on your code. I am in no way commenting on any else’s Python code at this point, but I did get a comment on my leap year calculation.

I looked over the comment and then changed some code. I had to futz with the command line to get this to resubmit, but as you can see, I ended up getting a second iteration in there. No comments on that one, but we’ll see.

It’s an interesting idea to share code and get comments. I think mostly this is a way to formally practice some exercises and get comments from experienced users, but the volume means that potentially you won’t get comments on your solutions. I know many of the SQL users may, or may not, comment on solutions.

I think this is interesting, and I’m tempted to try to do something like this for SQLServerCentral. The hard part of putting together enough questions that others can practice in an organized fashion.

September 28, 2015