Author: way0utwest

It Happens

Your code is wrong.

Or it’s bad, or it won’t work as intended. I’m not sure that’s the viewpoint I’d like to take when I first start working on a project, but that’s what Nathan Marz told the attendees at the NoSQL2013 conference recently. He says that our code should be treated as something that might or might not work, and we should embrace the idea that the code can be wrong. It’s not intended to disparage developers or paint too bleak a picture of our skills, but to just be realistic in the sense that all projects have problems and software isn’t perfectly built at the first compile.

I think most of us do realize that our software also might not work as intended. Even simple projects can have bugs and holes that we’ll find when edge cases are introduced or the software is used in a way we didn’t expect. At least I hope it’s just unanticipated, edge cases and not general cases for most of us. If an application doesn’t work for the cases we’ve designed it for, then we need to work on our coding skills.

We know that building software is hard. We know that outside of a narrow domain for which we’ve often designed our software, it might not work as intended. Most of us expect to find bugs or problems in our software, and are mentally prepare to patch and enhance our work as needed. It’s almost inevitable, and even occurs when you design your own software, to do that one thing you need done. You’ll still maintain (meaning patch or enhance) the software over time.

I hope that’s not too gloomy a view of software.

Steve Jones

Video and Audio versions

Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

Follow Steve Jones on Twitter to find links and database related items and announcements.

Windows Media Video ( 12.6MB)

MP4 iPod Video ( 15.0MB)

MP3 Audio ( 3.1MB)

Feeds are available at iTunes and Mevio

To submit an article, rant or editorial,
log in to the Contribution Center

September 25, 2013

The Human Cost of Managing Data

If you’ve ever worked in a large environment, where the numbers of nodes number in the thousands, or tens of thousands, then you know many software packages don’t scale to that size very well. I’ve seen quite a few pieces of software that were well written, and ran well with hundreds of items, but choked miserably when the count crept past a few thousand. Typically those are edge-case sized installations, and most applications aren’t designed for, or tested at, those scales. I haven’t registered 1,000 instances in SSMS, but I could guess that it would not necessarily run smoothly with that many instances being examined.

Software designed at scale tends to take the “install another central node” and manage multiple central nodes individually. That quickly becomes a pain point for the humans that administer the systems as they must learn and remember where individual systems are being managed. It’s not much different for hardware as well. When a system exceeds the capacity of a single node, we often find that we are adding multiple modes and managing them individually. Many backup systems work this way, adding new units to handle the additional backup space requirements.

However there’s a staffing cost as well, and it’s one that can be overlooked as a company grows. More systems mean more backups, more maintenance, and more failures. Even with automation and standardization, there’s a physical workload increase on your staff, and also a stress level increase from managing more systems. It can be easy to overwhelm your staff, assuming they can easily handle the additional load because they have plenty of tools to do so.

Tools are required, and whether you build or buy them, you’ll make use of them if you want to minimize your salary costs. However you still need to make sure you grow your staff, train them, and give them time off. The last thing you want is to overload and burn out a small staff of one or two. I’d bet that the week your staff quits or takes ill, your systems will choose to fail.

Steve Jones

Video and Audio versions

Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

Follow Steve Jones on Twitter to find links and database related items and announcements.

Windows Media Video ( 16.9MB)

MP4 iPod Video ( 19.7MB)

MP3 Audio ( 4.1MB)

Feeds are available at iTunes and Mevio

To submit an article, rant or editorial,
log in to the Contribution Center

September 24, 2013

An easy travel week for SQL in the City and SQL Saturday #190

This week I have two more events in my busy fall schedule: a SQL in the City Seminar on development and SQL Saturday #190. However my travel plans are limited to getting in my car and driving from the ranch to the Denver Tech Center for both events. These are rare, local events for me and a nice break from the 8 or 9 other events the rest of this year that I need to travel to.

It’s going to be a busy week, however. I have to deliver the whole Friday event myself, which is a lot of speaking. I did 2 of the 3 sessions earlier this year, so it’s not a long stretch, and it’s certainly easier than the full day pre-cons and training sessions that you get from Brent Ozar, Unlimited or SQLskills, but I feel some pressure since I don’t have any supporting cast and it’s a local event. Plus this will be my first time delivering one of the sessions. Fingers crossed the demo gods are kind to me.

On Saturday I have just one session on branding, which I enjoy talking about. After that, it will likely be a shorter day for me as I travel again on Tuesday for DevConnections. However I hope to see a few friends and meet new people at the event. Come on by if you can.

September 24, 2013

Citizen Scientists

The tremendous growth of computing power, especially in the form of mobile devices, means that more and more people can analyze data themselves, or have applications do it locally, reducing the need for large central systems to perform all the calculations. Indeed some of the value from Hadoop and other cluster computing systems comes from using lots of small, relatively powerful systems to do more work than one single computer could.

However we can not only analyze, but also gather data. There is a rise in the citizen scientist, in which individuals can help scientists by gathering data and sharing it with others. This can reduce the costs of gathering data, but more importantly, gather much more, and varied, data than might otherwise be possible. There are a number of projects available if you’re an amateur scientist that wants to join in.

Most of us are data professionals and might be interested in working with more data. Perhaps we ant a set for demo purposes, but perhaps we want to work on a project that interests us and could use our technical talents. Many of the projects out there make their data available, and you should consider using such a set if it strikes your fancy.

More importantly, I would hope that more people would analyze the raw data and look to verify, or dispute, the conclusions of the project owners. As data professionals, we can help people learn how they can analyze and manipulate data better with the tools we have available. With more transparency, we can have healthier debates on the causes, the effects, and even the potential future implications of how to build a better world in the future.

Steve Jones

Video and Audio versions

Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

Follow Steve Jones on Twitter to find links and database related items and announcements.

Windows Media Video ( 17.5MB)

MP4 iPod Video ( 20.3MB)

MP3 Audio (4.2MB)

Feeds are available at iTunes and Mevio

To submit an article, rant or editorial,
log in to the Contribution Center

September 23, 2013