Voice of the DBA

Tag: software development

  • Ignoring Database Version Control

    I find myself surprised that so many people resist keeping their code in version control. I think this is less people all the time, but I don’t have good percentages because at this point I think a fair number of people embarassed enough to not answer the questions. However, in quick conversations, I still find people that don’t think a VCS is necessary. I can’t conceive of this for application code, but more and more I think this also makes sense for database code.

    I ran across a blog that asks you to find a way to keep your database code in a VCS. While I’m glad that the author provides some ideas on how you might script this yourself in a pinch, I think that’s silly. There are tools to help, whether paid for ones, like SQL Source Control or free utilities. In either case, it makes sense to work with something that has been tested a bit rather than inventing your own. It seems that running scripts is simple and easy. In practice, these tools add value because of their maturity. Some of the mistakes you’ll make with your own software have already been made, and solved, with many of these frameworks and paid for tools. Don’t spend your time maintaining software scripts when you need to be developing other software.

    I also think the author makes the process sound a bit simpler and easier than it is. In practice, moving between schema versions in a database is typically a one way process, and rollback scripts are of dubious benefit. A discussion of this post on Hacker News shows some of the shortcomings in trying to version database schemas and easily move between versions. In practice many database schema changes can be rolled forward and back, and the corner cases that are very difficult for tools to solve, aren’t items that occur too often. Really, it’s table changes that are hard, and having a good process helps here, but these are usually one way changes.

    However, the reason I think version control is important is that database development is often disconnected between the database and application. Changes might occur in the database long before they are used in code, or they may need to be undone in development, perhaps in a destructive way that isn’t possible in production. The history of changes, and understanding of what needs to be changed, is much easier with a record in a VCS.

    These days there are a variety of ways to capture your schema changes and record them. I would encourage you to investigate some method and give it a try. Even building the habit of manually recording changes by all developers will help you to move faster and faster in your development process without worrying about losing track of what’s happening.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.8MB) podcast or subscribe to the feed at iTunes and Mevio .

  • Practicing Deployments

    This editorial was originally published on Feb 18, 2013. It is being re-run as Steve is out of the office.

    It’s said that amateurs practice until they can get something right. Professionals practice until they don’t get something wrong. That’s the idea, and while professionals make mistakes, they make far fewer than those that don’t approach their craft as a professional task.

    Many of us in the data industry develop software in some way. Whether we write queries in T-SQL or build projects in .NET, we produce code to accomplish some task. I’d like to think that many of us improve our skills over time, preferably by practicing new techniques and learning from our mistakes. I know some people stick with the tried and true methods without gaining skill over time, which not only hurts one’s career, but also doesn’t give an employer a reason to value their work.

    However the deployment of software, which encompasses more than the developer(s), doesn’t improve in many cases. Deployment includes operational people’s skills, scheduling dowtime with clients, possible even the briefing of support personel. However the whole process is often performed poorly. Deployments fail, or require more time than expected. People view them as a pain, and software deployment tends to happen less often than it could, resulting in a large software inventory.

    There’s a great quote from James Moore on how we can deploy software better: “…deployments are hard, but rather than long-winded planning, they need constant practice, testing and refining, and we could only do this by deploying early.” Red Gate Software has learned that deploying more often results in the company gaining skill in deploying software, resulting in more successful software changes in applications.

    The improvement you make in your software can bring tremendous value to your clients, but only if they can use those features in the software. Learning to push those changes out in a repeatable, professional manner is a great way to ensure your clients and customers trust you to deliver new features and enhancements that meet their needs.

    Steve Jones

  • Rogue Algorithms

    This editorial was originally published on Sept 5, 2-12. It is being re-published as Steve is on vacation.

    I have always thought that computers do some amazing things, but they tend to do what we tell them to do, even if what we tell them to do is not what we want them to do. In those cases we usually find computers are helping us make mistakes faster than ever before. As we automate and link more and more systems, this can become a bigger and bigger problem in the world.

    I ran across this short link on a rogue algorithm that caused a fluctuation in a number of stocks. It looks like this was a case of poor analysis and poor QA at one firm, but imagine if we had some type of “update” released to a large number of companies? This type of problem could cause a dramatic short term fluctuation in the stock market, which shouldn’t affect most of us in the long run, but you never know. If this caused a company to go out of business, and they were the company holding your retirement savings, you might feel differently.

    This isn’t necessarily a problem that affects just financial institutions and their custom software. This is potentially a problem in any business that writes its own algorithms. That includes many of our companies, and many of us. We write queries, reports, and develop algorithms that help “the business” analyze data and make decisions that can affect our companies.

    This is worrisome to me, especially when we have companies that press for more and more analysis, with algorithms written more and more quickly to respond to business events. Success in this area often depends on the developer having an understanding of the nature of the business and the meaning of data, and strong working relationships with the business analysts. That comes over time, and is easily lost when employee turnover is too high.

    I hope that companies are learning that there is value to retaining employees, especially those that work with data and have invested effort in learning more about their business. Unfortunately I think that it take a disaster like this for some managers to understand the value of retaining employees and their knowledge. Even more unfortunate is the fact some managers never learn that.

  • Avoiding Stored Procedures

    This editorial was originally published on Aug 13, 2012. It is bein re-published as Steve is on vacation.

    I ran across this piece from a developer on why he avoids stored procedures and thought it made some good arguments. The primary thrust of the piece is that ORM tools (Object-Relational Mapping) have evolved to handle most of the requirements of many applications. They also save a ton of development time since so many stored procedures written are simple CRUD type operations.

    In many cases I agree. Having developers write stored procedures is silly and a waste of time. Procedures that select a few fields, or that update a table based on a primary key are mind-numbingly simple to write, but they take time to get in place. That’s time a developer isn’t spending thinking about the application and logic. Plus with any ORMs and tools like LINQ, you can write one line of code and let the ORM handle all of the work of getting or storing the data. Good points, and in many cases that’s correct. If you do mostly CRUD type work, this is a good reason to perhaps avoid stored procedures and let a tool do the work for you.

    Unless you use a different tool. There are plenty of tools, most of them free, that will generate that CRUD code for you. A few templates or snippets will handle the front end side of the call as well, building code to call stored procedures. If you’re actually typing this stuff over and over, you are wasting time.

    My brain started to wander when I saw “A database should be limited to the role of a persistence layer” which is silly sounding when you move beyond CRUD operations. It completely shut off when I saw “your stored procedures would need to be re-written in order to migrate to MySQL, Oracle or another database” since I think this rarely happens. If it does for you, fine, but the vast majority of apps never leave their initial database.

    There are benefits in ORM tools, but you need to understand how the ORM works, what it’s strengths and weaknesses are. Blindly following the basic pattern for your state-lookup-data-editing dialog for all reporting screens is a sure way to cause yourself some problems. Allowing the ORM to define your relational database, without spending some time thinking about the benefits of good database design and proper modeling is asking for performance problems, or even integrity issues.

    ORM tools are just that tools. Used well, they can perform admirably, but just as I don’t use a hammer to drive a screw into wood, don’t depend on your ORM handling everything database related for you in an efficient manner.

    Steve Jones