Voice of the DBA

Tag: software development

  • The T-SQL Paradigm

    This editorial was originally published on Mar 21, 2009. It is being re-run as Steve is on vacation.

    I ran across this post ranting about T-SQL. It’s got going pretty quickly with some less that professional responses. It’s cleaned itself up a bit, but it got me thinking for this Friday.

    My first programming was done in BASIC. From there I played with a little assembler before going on to Pascal and C in high school. The came LISP, APL, Fortran, more assembler, C++, and a little ADA in college. My professional career had me programming in Lotus 1-2-3, dBase, FoxPro, C, VB, and finally SQL with minor dabbling in HTML, XML, ASP, and some .NET.

    I think I’ve had a decent amount of programming experience in my life. In that time, I’ve found that most programming languages to work fine, and while I’ve enjoyed some more than others, I haven’t thought that any “sucked.” I wouldn’t want to write most modern apps in APL, but I don’t much think there’s any thing wrong with picking VB.NET v Java v C#. So for this Friday’s poll:

    Is T-SQL really a strange paradigm?

    Or SQL in general? Is it really messed up? Would the structures used by LINQ make more sense to anyone? Hardcore developers seem to like a different structure, but is it because it seems more natural to them? Or is SQL really such a strange beast.

    Many people seem to have trouble with SQL. Just read the forums here at SQLServerCentral to see all the issues that come when people try to query the database. But is it just hard to learn or is there a better way to build a query language?

    Steve Jones

    The Voice of the DBA Podcasts

    Everyday Jones

    The podcast feeds are available atsqlservercentral.mevio.com. Comments are definitely appreciated and wanted, and you can get feeds from there.

    You can also follow Steve Jones on Twitter:

    Overall RSS Feed:  or now on iTunes! 

    Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

    I really appreciate and value feedback on the podcasts. Let us know what you like, don’t like, or even send in ideas for the show. If you’d like to comment, post something here. The boss will be sure to read it.

  • Solving Problems at Midnight

    I couldn’t sleep last night. I was tossing and turning, struggling to rest, and it seemed I was waking up every 20 minutes.

    My demo broke. I have a talk that I’m giving at the Denver SQL Server User Group on Thursday, as practice for the same talk next week at SQL Intersection. After returning from my travels, I fired up my virtual machine on Monday and went through the talk. I’ve given a version of this as recently as 3 weeks ago, but I like to refresh myself. When I got to the third demo, I couldn’t get it to work.

    I tried a bunch of things, but the test I was sure would fail, wouldn’t. This is a significant part of my demo, and it shows how data volumes affect things. I reset the snapshot. I dropped and rebuilt the database and process. I even took a 3 hour break, going for a walk, picking up kids, and hitting a yoga class. When I returned at night, while my kids were doing their homework, I couldn’t get mine to work. I even faked the test, and caused a failure, but that wouldn’t work with an audience.

    I gave up around 9:30 and went to bed shortly thereafter. At least, I tried to, tossing and turning. My mind was all over the map, thinking about work, some projects at home, kids’ school, etc. At 2am, it hit me. All of a sudden I realized why the test wouldn’t fail. The join that was supposed to be slow, wasn’t, because it was missing some data. I was generating a large volume of data in two of the three tables, but in the third, I hadn’t linked it to my version control system. That’s why it failed.

    I don’t always have such great inspiration at night, but I have found that sometimes walking away from an issue and taking a break can help. I’ve even walked to a colleague’s desk and helped them with their own issue, and all of a sudden I realize how to solve my problem. I’ve learned to learn to be productive in other ways when I’m stuck on a particular problem, letting my subconscious mind analyze things in the background.

    This morning I linked my table and the test started failing. I have rarely been so happy to see red errors on my monitor.

  • Getting Started with CI and Databases

    It’s fairly easy. At least, I think it is if a salesperson can do it.

    Alex Yates, salesman extraordinaire at Red Gate Software, spent a lunch hour recently setting up a CI system so he could better understand how hard, or easy, it can be for customers. He wrote about his experiences using the Jenkins CI server.

    Check it out. If Alex can do it, I bet most of you developers and DBAs can as well.

  • The Challenges of Being Safe

    This editorial was originally published on Mar 19, 2009. It is being re-run today as Steve is away at DevConnections.

    One thing we recommend is to make sure that production data is always kept safe, which means that you keep control of where it lives and how it’s used. I saw a post recently where someone needed to send data to a business partner and wanted to obfuscate it.  I think that is a great thing to do and was glad to see someone asking for advice.  Sending data to business partners is required, but you want to be sure that you are not necessarily giving away too much information, especially identity, privacy, or financial information that you shouldn’t.

    However what about your test and development environments? I’ve seen people obfuscate data here, but not always. In fact, not usually. This is despite the fact that you might turn over developers often, expose that data to testers, or other people that might not normally have access, and the fact that these environments almost never have auditing enabled.

    We need to mimic production environments and data, but there are quite a few challenges with doing this in a safe and secure manner. Just having scripts to obfuscate data is a challenge in and of itself. I’d love to see a tool built that would do this automatically, though I wonder how many companies would actually buy it. I’m not sure there’s a great market for tools here until insurance companies start require it for your “data loss insurance.”

    But building those scripts is both hard and time consuming. How do you decide what to obfuscate? What values do you use? How do handle PK/FKs to ensure that things match up correctly if you’ve used personal information, like a SSN, as a key field?

    Beyond that, there’s much more. What about ensuring that data matches up correctly? Can you really determine if there is an issue with some calculation or relationship if you have random data. After all people many times will have favorite accounts that they know well and understand what the data should look like. A developer may expect certain order details or address information, and use that as a benchmark when developing new code. If the data is random every time his environment is refreshed, does that slow his productivity? How do you test things like URLs and emails if data is randomized?

    And what about when you alter your schemas? How much work is there to update your scripts?

    And what about mistakes? How often has someone pointed development code at a production server? What happens if they accidently run an obfuscation script against production? If you’ve scrambled data around, would you catch it quickly enough to restore before too many transactions were sent through?

    Using some type of data obfuscation or randomization is a great way to help ensure that your production data is kept safe, but it definitely makes for a much more complex environment, and likely, more headaches for DBAs and developers.

    Steve Jones

    The Voice of the DBA Podcasts

    Everyday Jones

    The podcast feeds are now available atsqlservercentral.mevio.comto get better bandwidth and maybe a little more exposure :). Comments are definitely appreciated and wanted, and you can get feeds from there.

    Overall RSS Feed:  or now on iTunes! 

    Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

    I really appreciate and value feedback on the podcasts. Let us know what you like, don’t like, or even send in ideas for the show. If you’d like to comment, post something here. The boss will be sure to read it.