Voice of the DBA

Tag: software development

  • Are You a Tech Company?

    Are you a tech company. That’s an interesting question. I’m sure some of you might not think so, working for a commercial organization that has some physical product you sell or some service you provide. You might not consider yourself a tech company. However, your organization might be, at least according to this essay at Medium.

    I certainly think software has changed, and will continue to change the world. I think data becomes is an important part of our system, but in conjunction with software and the ease with which we can gather, process, extract, and analyze the information inside of our data. In order to do that, I do believe that we need ways to better build and deploy software, whether that’s database software or application software.

    To do that, can we be afraid of mistakes? I think not. The opening premise of the piece is looking at rapid delivery from two points of view. Can a system be secure if you change it multiple times a day? Or perhaps it can’t be secure if you don’t have the ability to change things. Either view could be true, and most of us lean one way or the other. My view is towards the latter, not because I want to release multiple times a day, but because if there are issues, bugs, or security holes, I need to be able to change my software that day. I can’t wait for weeks or months.

    That’s one of the problems I’ve seen with quite a few systems in the past. The time to release a new version, to patch an issue, is far, far too long when potentially hundreds, or thousands, of malicious attackers could take advantage of a flaw this week. Or our company could lose customers because a feature is broken for long periods of time. While I don’t think the cost of fixing mistakes is zero, I do think that the cost of not fixing issues is also more than zero. Certainly at times the cost of not fixing something might be much, much higher than our organization would tolerate.

    Mistakes will happen. However, the point the essay makes is valid. We should learn from mistakes, and not try to avoid them, but continually push forward to improve our software. Above all, we need to find ways to engage our employees and get them to believe they can produce better software and are empowered to do so.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Mevio .

  • Ignoring Database Version Control

    I find myself surprised that so many people resist keeping their code in version control. I think this is less people all the time, but I don’t have good percentages because at this point I think a fair number of people embarassed enough to not answer the questions. However, in quick conversations, I still find people that don’t think a VCS is necessary. I can’t conceive of this for application code, but more and more I think this also makes sense for database code.

    I ran across a blog that asks you to find a way to keep your database code in a VCS. While I’m glad that the author provides some ideas on how you might script this yourself in a pinch, I think that’s silly. There are tools to help, whether paid for ones, like SQL Source Control or free utilities. In either case, it makes sense to work with something that has been tested a bit rather than inventing your own. It seems that running scripts is simple and easy. In practice, these tools add value because of their maturity. Some of the mistakes you’ll make with your own software have already been made, and solved, with many of these frameworks and paid for tools. Don’t spend your time maintaining software scripts when you need to be developing other software.

    I also think the author makes the process sound a bit simpler and easier than it is. In practice, moving between schema versions in a database is typically a one way process, and rollback scripts are of dubious benefit. A discussion of this post on Hacker News shows some of the shortcomings in trying to version database schemas and easily move between versions. In practice many database schema changes can be rolled forward and back, and the corner cases that are very difficult for tools to solve, aren’t items that occur too often. Really, it’s table changes that are hard, and having a good process helps here, but these are usually one way changes.

    However, the reason I think version control is important is that database development is often disconnected between the database and application. Changes might occur in the database long before they are used in code, or they may need to be undone in development, perhaps in a destructive way that isn’t possible in production. The history of changes, and understanding of what needs to be changed, is much easier with a record in a VCS.

    These days there are a variety of ways to capture your schema changes and record them. I would encourage you to investigate some method and give it a try. Even building the habit of manually recording changes by all developers will help you to move faster and faster in your development process without worrying about losing track of what’s happening.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.8MB) podcast or subscribe to the feed at iTunes and Mevio .

  • Practicing Deployments

    This editorial was originally published on Feb 18, 2013. It is being re-run as Steve is out of the office.

    It’s said that amateurs practice until they can get something right. Professionals practice until they don’t get something wrong. That’s the idea, and while professionals make mistakes, they make far fewer than those that don’t approach their craft as a professional task.

    Many of us in the data industry develop software in some way. Whether we write queries in T-SQL or build projects in .NET, we produce code to accomplish some task. I’d like to think that many of us improve our skills over time, preferably by practicing new techniques and learning from our mistakes. I know some people stick with the tried and true methods without gaining skill over time, which not only hurts one’s career, but also doesn’t give an employer a reason to value their work.

    However the deployment of software, which encompasses more than the developer(s), doesn’t improve in many cases. Deployment includes operational people’s skills, scheduling dowtime with clients, possible even the briefing of support personel. However the whole process is often performed poorly. Deployments fail, or require more time than expected. People view them as a pain, and software deployment tends to happen less often than it could, resulting in a large software inventory.

    There’s a great quote from James Moore on how we can deploy software better: “…deployments are hard, but rather than long-winded planning, they need constant practice, testing and refining, and we could only do this by deploying early.” Red Gate Software has learned that deploying more often results in the company gaining skill in deploying software, resulting in more successful software changes in applications.

    The improvement you make in your software can bring tremendous value to your clients, but only if they can use those features in the software. Learning to push those changes out in a repeatable, professional manner is a great way to ensure your clients and customers trust you to deliver new features and enhancements that meet their needs.

    Steve Jones

  • Rogue Algorithms

    This editorial was originally published on Sept 5, 2-12. It is being re-published as Steve is on vacation.

    I have always thought that computers do some amazing things, but they tend to do what we tell them to do, even if what we tell them to do is not what we want them to do. In those cases we usually find computers are helping us make mistakes faster than ever before. As we automate and link more and more systems, this can become a bigger and bigger problem in the world.

    I ran across this short link on a rogue algorithm that caused a fluctuation in a number of stocks. It looks like this was a case of poor analysis and poor QA at one firm, but imagine if we had some type of “update” released to a large number of companies? This type of problem could cause a dramatic short term fluctuation in the stock market, which shouldn’t affect most of us in the long run, but you never know. If this caused a company to go out of business, and they were the company holding your retirement savings, you might feel differently.

    This isn’t necessarily a problem that affects just financial institutions and their custom software. This is potentially a problem in any business that writes its own algorithms. That includes many of our companies, and many of us. We write queries, reports, and develop algorithms that help “the business” analyze data and make decisions that can affect our companies.

    This is worrisome to me, especially when we have companies that press for more and more analysis, with algorithms written more and more quickly to respond to business events. Success in this area often depends on the developer having an understanding of the nature of the business and the meaning of data, and strong working relationships with the business analysts. That comes over time, and is easily lost when employee turnover is too high.

    I hope that companies are learning that there is value to retaining employees, especially those that work with data and have invested effort in learning more about their business. Unfortunately I think that it take a disaster like this for some managers to understand the value of retaining employees and their knowledge. Even more unfortunate is the fact some managers never learn that.