Voice of the DBA

Category: Editorial

  • NoSQL Is Not Everywhere

    How many places really use NoSQL? Facebook, one of the largest sites on the Internet, has had their Cassandra database service receive  a lot of press over the last year. In pointing to them as an example, many technologists use the argument that if Facebook uses Cassandra to run their extremely data intensive business, then it ought to be good enough for the rest of us.

    Cassandra might be great for your business, but is it the best choice? How does that argument apply when Microsoft.com, also one of the largest web sites in the world, serves millions of users a day with SQL Server as the back end? Twitter also uses an RDBMS to store it’s tweets. It uses the MySQL database, an RDBMS, though they are investigating and integrating Cassandra in places.

    The point is that NoSQL databases, like .NET assemblies or J2EE applets are just technologies. They’re just tools used to build an application, and they can be built well, or built poorly. Is Cassandra bad because it didn’t work well for Digg? No, just like those companies that have switched off SQL Server to some other platform aren’t necessarily doing so because SQL Server can’t meet their needs.

    It’s up to the architect to put together a great platform for an application, using whatever tools they choose. Hopefully they are choosing those tools because the fit the problem, or the skills of the developers. I’d be disappointed in an architect that chooses something because it’s new, or discards another technology because of some personal bias.

    Steve Jones

  • Big Data and SQL

    I ran across this article on 8 Big Data Deployments in Detail, which was funny since the detail consisted of an old system, a new one, a capacity, a date, and a paragraph. If that’s detail, then I wonder what a synopsis would be. Only one of the 8 was using SQL Server, and that had even transitioned to a ParAccel system. Most of the other systems used Oracle previously and they ranged from 7TB (not big, IMHO) to 2.5PB (which is big).

    Despite that lack of any real information on what these companies were  doing, I did see some interesting things listed.  There were a few notes that mentioned compression in a few places, often column store based compression that dramatically sped up processing for the systems.

    However the one really amazing fact that I noticed cam from Cabela’s, the sporting goods retailer. They had a short note that retraining their statisticians to be “more SQL-based” and made them more effective. They reduced the need in one area from 7.5 full time equivalent people (FTE) to 1.5 for the same work.

    Hopefully that didn’t mean that 6 people lost their jobs, and instead they were able to focus on other work and find ways to better help the company. That is fairly telling, that training people to understand SQL, and maybe write better queries and analysis themselves can make them much more efficient.

    It makes me wonder if the same thing might happen with developers. Some of you out there write software for a living with some OOP language, but are talented using SQL as well. Do you think that building those skills is  a worthwhile investment for other developers?

    I think it is. Now if I just had a way of convincing more developers this would help their careers.

    Steve Jones

  • Large Chunks of Data

    I saw a post recently where someone talked about trying to get better performance from a report. They were selecting 5mm rows from a table and wanted to see if there was a more efficient way to chunk out this data so that the instance would not report memory errors.

    My first question is what kind of report has 5mm rows of data? That’s just too much data.

    How long can a report be?

    From your experience, talking with people, looking at what they analyze, how big is a report? How much data can you really display on a report and make it useful for users?

    I’m thinking here in terms of the raw data you show. A pivot table can summarize millions of records, but realistically I thin kit becomes hard to examine more than a few hundred data points on a page. Whether they are raw data or aggregates of other data, it seems there’s some limit to what a report should provide.

    After all, that’s why we have drill-down 🙂

    Let us know this Friday what you think; what you have observed? Maybe we’ll  help others to build better reports that are more practical and useful to end users, as well as easier to develop.

    Steve Jones

  • Social Engineering Dangers

    DefConLogoI heard about a social engineering contest at this year’s DefCon hacker conference. The write-up said that every company targeted would have failed in a security audit, and these were some large companies, like Google, BP, Proctor and Gamble, Microsoft and more. It truly highlights the “a chain is only as strong as its weakest link” analogy being applied to companies, and I’m sure that the larger the company, the more weak links there are.

    Security is a constant battle. It’s hard to get right, and it’s hard to get people to take is seriously. Most employees don’t necessarily think that the information in a company is all that important. In fact, if you look at your databases, how much data in there do you think is really critical?

    No matter how important most of the data is, I’m sure there are some things that you would view as definitely worth protecting. I would also bet that some of that data that is worth protecting is co-mingled with other, less important data. Whether it’s in a database, on the filesystem, or somewhere else, often we have critical data mixed in with other data.

    Which means that we ought to try to protect it all. We ought to be applying strong security, and educating users that disclosing anything about the company to someone they don’t know, no matter how innocent the request, could lead to a breach of security.

    We want to help others. We want to be seen as people that make the organization function more efficiently. We can do that within the constraints of good security practices. It just takes a little effort to build the habit to stick to security procedures.

    Steve Jones