Voice of the DBA

Tag: software development

  • Failed Projects

    This editorial is from way back, published in 2005, but perhaps still a good one today. This is being re-run as Steve is at the Data Platform Summit.

    Friday is poll day, at least as long as I can be creative with them. And of course, remember to put them out on the right day Smile This week is more of a software development issue than a strictly SQL Server one, but take it how you will and get your post in here for everyone to see. Humility and responsibility are the keys this week. So without further ado…

    Have You Ever Failed In A Software Project?

    A simple yes or no would suffice for the poll question, but I’ll set an example and give you my answer with an explanation. The short answer is yes. The longer one is abbreviated in the interests of space, but I have made an attempt to set things as I best remember them.

    When I came to Denver 7 years ago, it was with a startup company in the financial services market with a few simple software products, but some trouble getting them stable and focusing the IT group. I was excited at the opportunity, the dot com boom was, well, booming, and I was thinking I’d make a million with this company. Fast forward 2 years and I resigned, somewhat burned out, and thinking I’d done everything I could.

    Thinking back I think I had done lots and given a lot, but I still failed and part of it was me not taking a leadership role more than I did and part of it was getting burned out and part of it was not really forcing the company to stop and make good decisions regardless of the time involved. I instead put up a bit of a fight, but then went along with the constant “just patch it” and we ended up burning most of a year getting nowhere because the underlying base products were a mess. I resigned, the company lost about 30% of the development staff, floundered for a couple years and was sold. Not sure anyone made money on that deal.

    As I’ve looked back, I realize it was a dysfunctional company, but I think I could have made a difference, stepped up as CTO (it was offered to me) and driven things forward. Oh well, I learned quite a bit from the experience and grew from it.

    Steve Jones

     

  • Software Has Bugs

    Every once in awhile I have someone I meet complaining about software quality, usually with regards to some piece of Microsoft software. I’ll invariably hear how there shouldn’t be bugs in SQL Server or Windows, and Microsoft isn’t testing their software. This person will claim that they write software with no bugs. I can’t remotely believe this is true, aside from small, somewhat trivial applications. All software has bugs, but there’s no way to disprove anyone’s assertions unless you can get to their software and examine it.

    I do think that some people think their software is more bulletproof than it really is, perhaps because no one really does test the limits of their work. It’s easy to assume that there aren’t problems when no one has ever probed deeply in the different ways the code might execute. After all, this is one of the reasons that developers turn in code that they think works, but in which others find bugs. They haven’t explored enough different inputs against the software. This is also why I think unit testing needs to be constant and evolutionary, growing and changing as you find new issues and bugs in your code.

    Recently a vulnerability was found in the Samba software, and apparently it has existed for seven years. This means lots of software was vulnerable, and everyone should be patching as soon as possible. I don’t know how many people have port 445 open to the Internet, or even open locally, but plenty of people might now know about the issue, so patching makes sense. Scans of the Internet have turned up hundreds of thousands of devices, some of which are running versions that can’t be patched.

    This isn’t meant to complain about Samba, but point out that this is code that is open source, lots of people have looked at and developed against, and I would guess no shortage of security people have checked this for potential vulnerabilities. And still a bug slips through.

    Security isn’t hopeless, but it’s also hard. Software development, with few boundaries limiting the choices developers can make, allows new interactions among the building blocks of code that have never been seen before. Building code is much, much more complicated than any physical structure because we don’t have any of the inherent limits of the physical environment. This means that we will, and do, struggle to test all of our software.

    I don’t think we’ll ever eliminate all bugs, but we can vastly improve quality (as many companies have), by moving to a more DevOps development style where we seek to constantly improve quality as a part of software development. We also have more and more developers also seeking to improve their skills, learning new and better patterns throughout their careers, and hopefully, building better software.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.2MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Poor State of Secure Coding

    Security is becoming a bigger issue all the time. More companies are getting hacked and losing control of data. It seems every week there’s a new disclosure from some firm. Recently OneLogin had a breach, which is very disturbing as they provide a single sign on solution for customers. This year we’ve learned about E-Sports, XBOX, Playstation, IHG, Arby’s, River City Media, Verifone, Dun and Bradstreet, and more.  At this point, there’s no reason for any large organization to wonder if they’ll get hacked. They should be preparingfor when they get hacked.

    The state of coding is poor, with far too few developers understanding how to write secure code. Even trying to learn how to code securely is hard. Too many examples given show poor coding practices. If you search for secure coding practices, you’ll get information, but none of the sample applications, none of the common information that most people would use to write code, is returned. This is especially true of data access, where far too many examples use dynamic strings.

    Even if we had great developers, there are still issues. A look at a survey from O’Reilly and SIG shows that there are still plenty of companies that are interested in security, but don’t perform reviews or use tools. Certainly many organizations don’t invest in security tools or resources heavily, and many companies don’t want to spend extra time worrying about security when there are features to build and deploy.

    My wish is that large organizations would engage in constant pen testing and review of their systems, looking for vulnerabilities, and patching them. I would hope that insurance companies would start to deny claims when a patch for any software has been available for six months or longer. That might help reduce the number of issues from older libraries not being upgraded.

    I would also expect that any vendor selling software engage in some security review for their products. In fact, I’d hope that once a company sells a certain number of units, this would be required. I’m still amazed that this isn’t a requirement for purchase from more customers, but since most vendors don’t bother, perhaps avoiding purchases of un-reviewed software isn’t feasible. Maybe it’s just as well; even if we did have some sort of review, how many of us would understand what that means? How many of you really understand what PCI or HIPAA compliance means? How well has that helped us? I guess things could be a lot worse than they are today.

    Security is going to be an issue for a long time. All I can do is try to improve my own skills and ask you to do the same. Learn to code securely and try to improve the software you work on. It might only make a small difference, and you might never know if it helps, but I bet you’ll feel better about your own work.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.3MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Does Speed Compromise Quality?

    One of the parts of DevOps that is often hyped is the speed and frequency of releases. Starting with Flickr and their 10 deployments a day, to Etsy deploying 50 times a day, we’ve seen companies showcasing their deployment frequency. Amazon has reported they deploy code every 11.7 seconds on average. That seems crazy, but with so many applications and lots of developers, not to mention each change being smaller, perhaps it’s not completely crazy. With the forum upgrade here at SQLServerCentral, we had two developers (with occasional other sets of eyes reviewing code changes), and while we were bug fixing, we deployed multiple times per day.

    Is that a good idea? Does a rapid set of changes mean that quality is lower and more bugs are released? It certainly can. In fact, if you’re a development shop that struggles with releases and code quality, producing software faster is not going to help you. In fact, if management pressures you to adopt DevOps, and deliver code faster without culture change, without implementing automated testing, including for your database code, and using automated scripts, tools, or something to deploy software, then you are going to get more bugs out faster. You’ll still get to change direction quicker if you find you’re building the wrong software, but you’ll still end up becoming more inefficient because of bugs (And technical debt).

    There’s a fantastic video (long) about refactoring code in two minutes. A bit of an oxymoron since the presentation is nearly two hours long, but the video is from a real project. However, their approach is that good unit testing allows them to refactor code, to change things, without introducing bugs. That’s a big part of the #DevOps philosophy. I always note in my DevOps presentations that if you can’t implement unit testing, meaning you won’t bother, then you don’t get much benefit from CI, CD, or any DevOps ideas. Tests protect you from yourself (and others).

    In many of the DevOps reports, companies that release faster report fewer bugs and less downtime. Since Amazon has increased their speed, they have 75% fewer outages across the last decade, 90% fewer time down, and many, many fewer deployments causing issues. Turbotax made over 100 production changes during tax season and increased their conversion rates. The State of DevOps reports bear this out (2016 here). Thousands of responses show that speed doesn’t cause more bugs.

    Because they work differently.

    If your management won’t let you change the way you work, if you don’t implement automated unit tests (and other types of tests), if you don’t take advantage of version control, if you don’t ensure every change is scripted, then you won’t work differently, and speed will bring bugs.

    You can do better. Your company can do better. Will they?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.2MB) podcast or subscribe to the feed at iTunes and Libsyn.