Voice of the DBA

Tag: software development

  • Failure Lessons

    orca
    Orca was a high profile software failure this year.

    I really wish that we would see detailed analysis of failed systems, if for no other reasons than to avoid repeating the same mistakes in the future. I’d hope that technical people that were engaged in building systems that didn’t work well would share their experiences, whether from development, deployment, administration, or even operations. I ran across a piece on the lessons of Orca, the web application that Mitt Romney’s campaign used, or tried to use, to manage their operations.

    It seems that there were a number of problems with this system, which is almost stunning. I’d think this is a well known process that includes a number of pieces of technology that are built into so many systems these days. Integration is never smooth, and the short time frame of an election campaigns doesn’t leave a lot of time for testing, much of which apparently didn’t get completed. The article talks about many of the same things that I’ve seen mentioned in the past when applications don’t work as expected. A lack of training, a dearth of hardware, tooling that doesn’t work, all of these have been reported for years in many software engineering journals and articles.

    Perhaps more analysis won’t help. I doubt that even a high profile failure would convince the manager of any internal software development project to spend more resources or lengthen their time line to prevent a problem with an application. Like most developers, managers are eternal optimists when it comes to software being completed, regardless of their past experience. They never seem to learn that pushing for faster releases, cutting features, and limiting testing will showcase the end product poorly for the customer.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Software Teams

    Teamwork is important

    Today’s editorial was originally published on Dec 10, 2007. It is being re-run today as Steve is at DevConnections.

    I definitely used to not trust developers. At least not with my database code, requiring them to send requests to me and I’d build tables, stored procedures, etc. Over the years as I’ve talked to others and become a little more mature, I’ve started to believe that I can trust them more and let them build objects. I still think I need to review their work, but they can be trusted to get the job done their way.

    Apparently I’m not the only one and this is a good blog entry on software teams and some of the issues with managing developers. In the post Jeremy Miller talks about the need for teams to work together with great communication and not be bound or regulated by external groups that just want things to flow a certain way.

    It’s probably a lot of common sense, but it’s really good advice that you have knowledgeable people within and outside of the development team. You include people in discussions about architecture or implementation, you invite collaboration and you trust your people.

    Trust.

    That’s the big word and despite the fact that many of us say we trust our teammates, too often we just do something outself rather than ask or expect someone else to do it. Or trust that they will.

    We may say it’s because it’s quicker, we don’t have to explain it, we were already working on something similar, or many other reasons. But often it’s that we trust ourselves more than others.

    That’s understandable, but it doesn’t build a strong team for the long term. Working with others, letting them make some mistakes, maybe even take a little longer than you would, will pay off over time as you learn to trust each other, count on each other, build that bond that only a team that has all members working as one unit can achieve.

    And achieve a synergy that produces from work from the team than would come from the sum of each individual’s efforts.

    The Voice of the DBA Podcasts

    The Great Music

    The podcast feeds are now available at sqlservercentral.podshow.com to get better bandwidth and maybe a little more exposure :). Comments are definitely appreciated and wanted, and you can get feeds from there.

    Today’s podcast features music by Joe Sibol. If you like it, check out his stuff on iTunes or at www.joesibol.com.

    I really appreciate and value feedback on the podcasts. Let us know what you like, don’t like, or even send in ideas for the show. If you’d like to comment, post something here. The boss will be sure to read it.

  • Walk on the Wild Side

    lou reed walk on the wild side
    Have you ever worked on the other side of your shop?

    In my career, I’ve worked in a variety of environments and positions. I’ve been a developer, and a production DBA, sometimes at the same time. Many of us aren’t working in environments where we are strictly performing one job. We often flex our work to meet the needs of our organizations, doing whatever is needed to get the job done.

    However many of us spend most of our time either producing software or supporting it, sometimes separated from the other side by internal rules or even regulatory requirements. Those of us that are successful usually learn to make friends with the people doing the other kind of job and these relationships allow us to bend rules and regulations to get things done quickly. That can be a good thing, but it often doesn’t really give us any appreciation for the job that others do on a regular basis.

    In a few large companies I’ve worked in, management felt it was important that the developers or engineers spent some time in the operations area. People would rotate out of their area for a month and spend time working in another area, gaining skills that might allow them to provide support in an emergency, but more as a way of helping them understand the impact of their work and how they might go back to their job with a new perspective in the future. The ultimate goal was for design and development to proceed more efficiently and at a higher quality level. It worked well at times, but often failed if the developer didn’t embrace the temporary assignment with a positive attitude.

    I’ve rarely seen administrators sent to work with development groups. Usually an administrator has to have an interest in development and is interested in making a career change. However I think that there is value from giving administrators some exposure to the development process. They might realize how complex and tedious it can be, how seemingly obvious problems slip through, and perhaps most important, they might learn how to better troubleshoot issues in production systems.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • A Software Warranty

    pocket knife
    Shouldn’t software developers give a warranty we can potentially void?

    Years ago I worked in a large company on the operations team. We were responsible for all production issues for the 6,000 people and the assorted machines, devices, and applications that come with a large workforce. There was a department that was aligned with our group that focused on engineering and various development groups that built different applications. The engineering group was good at working closely with the production team to ensure smooth deployments, but they weren’t on call and would at times respond slowly to develop solutions when problems occurred. They were, however, better than the development groups who often sent code to be deployed, and accepted bug reports back, but provided little support or assistance for problems with their code.

    I ran across this link from a DevOps person called You Write It, You Support It in the Brent Ozar, PLF newsletter. The piece makes a case for the problems that occur with some deployments, like a lack of, or surplus, of logging, switches to turn features on/off, error handling, and more. It’s a pretty good description of typical problems I’ve often seen, and it calls for developers to support the features that they write, even in production systems.

    I like this idea, though I don’t think that it should be a continuous expectation with developers required to support their code forever. I would like to see developers giving their code a “warranty” of sorts, perhaps a couple months of priority support when code is deployed into live environments with developers taking responsibility and responding to calls, even after hours.

    There are arguments to be made that developers’ time is better spent enhancing applications and applying their creativity to new ideas, but this leads to a human frailty. Too often we view a job finished as a job completed, and that’s not always the case. Doing a job well means more than completion. It implies a level of craftsmanship and pride in the finished product, both from the developer and the client.

    Developers should write code that works. If it doesn’t, then it’s not really finished and should be fixed. With that contract in place, we usually find that the developer spends a bit more time ensuring the product is built in a quality way that reduces the need for much support.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.