I ran across this article on 8 Big Data Deployments in Detail, which was funny since the detail consisted of an old system, a new one, a capacity, a date, and a paragraph. If that’s detail, then I wonder what a synopsis would be. Only one of the 8 was using SQL Server, and that had even transitioned to a ParAccel system. Most of the other systems used Oracle previously and they ranged from 7TB (not big, IMHO) to 2.5PB (which is big).
Despite that lack of any real information on what these companies were doing, I did see some interesting things listed. There were a few notes that mentioned compression in a few places, often column store based compression that dramatically sped up processing for the systems.
However the one really amazing fact that I noticed cam from Cabela’s, the sporting goods retailer. They had a short note that retraining their statisticians to be “more SQL-based” and made them more effective. They reduced the need in one area from 7.5 full time equivalent people (FTE) to 1.5 for the same work.
Hopefully that didn’t mean that 6 people lost their jobs, and instead they were able to focus on other work and find ways to better help the company. That is fairly telling, that training people to understand SQL, and maybe write better queries and analysis themselves can make them much more efficient.
It makes me wonder if the same thing might happen with developers. Some of you out there write software for a living with some OOP language, but are talented using SQL as well. Do you think that building those skills is a worthwhile investment for other developers?
I think it is. Now if I just had a way of convincing more developers this would help their careers.