Big Data is getting lots of attention. Will that mean more resources for managing it.
Many data professionals these days feel like they are working with too much data. Queries run slow, storage is constantly a concern, and our hardware is outgrown on a regular basis by the sheer amount of data that we need to manage. It’s a good thing that “Big Data” is in the news since it focuses more attention on the issues of data management and growth in our systems.
Some companies are taking the idea of managing their data more seriously. Boeing and Nike are talked about in this piece about coping with large amounts of data. With the vast amount of growth in the data stores these two companies work with, they had to find ways to ensure that business people could effectively find the information they needed in a sea of data.
It’s an interesting read and one that you might pass on to your management. The solutions aren’t new and innovative; they’re back to basics ideas. Have a central warehouse, use a data strategy that includes strong models and enforces data quality. Integrate your systems to allow a better view of your entire enterprise. These are the ideas that many data professionals preach, but are rarely found in many companies.
Sometimes a new perspective helps, and articles like the one above can help you convince managers of the need to raise your data quality and enforce some standards.
Steve Jones
The Voice of the DBA Podcasts
We publish three versions of the podcast each day for you to enjoy.
Watch your systems like a hawk. You’ll be glad you did.
An old phrase used in business is that knowledge is power. When you understand more about a particular situation, you can develop solutions and better understand which ones would both suit your environment better and solve any issues faster.
As someone that has worked in a variety of production environments managing all different types of platforms, I have found it extremely career-enhancing to have knowledge about the entire environment. Knowing what is happening at any point in time has allowed me to answer pointed questions about the applications. With comprehensive monitoring set up in database servers, I have often found myself in a position to proactively prevent issues, or make preparations for a quick response when I do find problems.
Over the years I developed a variety of techniques to monitor my own servers, often using third party solutions, enhanced with my own queries. I have learned that each system has its own idiosyncrasies, which often require custom queries to keep an eye on the pulse of the system. I’ve also learned that a regular exception report is the most valuable tool for me as a DBA. I assume most of my applications and scheduled tasks are working well; I just want a list of those things that have broken, or are not performing as expected sent to me every day, using the data from my various monitoring tools.
I have found that tracking metrics and analyzing the data is a valuable tool in my DBA tool belt. I would go so far as to say that monitoring the systems that you work with is not only a best practice, it’s essential if you want to be an exceptional DBA.
Steve Jones
The Voice of the DBA Podcasts
We publish three versions of the podcast each day for you to enjoy.
This week finished the SQL in the City 2012 tour, which covered six cities in the US this fall and London this past summer. It was a fun, hard, whirlwind event that had myself, Grant Fritchey, and a lot of other Red Gate employees working hard to bring you a fun, educational, and worthwhile set of events. We met a lot of great people, and hats off to Annabel Bradford at Red Gate for organizing the entire tour.
I think we did a good job, and heard some great comments from people at the events. Right now we’re starting to think about 2013, and I wanted to get some open-ended feedback here from people on what they’d like to see. Whether you attended a SQL in the City or not, give us your opinion.
What do you want from a one day training event?
I’m curious what you hope to get out of an event, what formats do you like, what types of content, and at what levels? Does the size, location, or any part of the venue matter? Do you need tables for a laptop or are you coming to listen and get ideas that you’ll work on in the future? Give us your feedback on what you think makes a worthwhile day of training for you.
There are any number of ways we can do things in the future, all with various tradeoffs. We can do smaller events in more cities, larger events with more staff, or something in between. I can’t promise anything specific, but your opinion and feedback do help us to try and deliver the community service and education that will help you get better at using SQL Server.
There’s always more than one way to get things done.
This editorial was originally published on Feb 13, 2008. It is being re-run as Steve is returning from the PASS Summit.
I have come competition here with my daily editorial. If any of you get the SQL Server Performance newsletter, you might have noticed that Peter Ward is now writing daily over there. He’s a nice bloke from down under, a fellow MVP, and he has some interesting things to say. He’s a bit more focused than I am and if you have the chance to hear him speak, it’s worth it. He gives some nice presentations.
I noticed the other day that he was talking about how he uses ALT-X to execute SQL queries, while most other people click “Execute” or click CTRL-E. I can’t help but think even more highly of Peter since I’ve used ALT-X for over a decade and know it’s the best way to do this. 🙂
His discusion wasn’t on executing queries, but on the fact that there are many ways to accomplish tasks in SQL Server. For example, you could audit by placing triggers on every table, capturing the inserted/deleted information, and storing it in a table. Or you could run Profiler forever and ever, making sure you have a system in place to manage those files. Or you could rewrite your code to use the OUTPUT clause. Or wait for Change Data Capture in SQL Server 2008.
All of these are valid, and there might be good reasons why you’d pick one over the others in your environment, but which one should you look to? What about many of the other possible ways of doing things in SQL Server? Should you look to rebuild or reorganize indexes by default? There are any number of tasks you need to accomplish and, often for people less experienced in one area, it seems either of a few ways is acceptable.
I certainly wouldn’t want to remove options and limit a DBA to developer to one way to do things in most cases. I think the richness of SQL Server, which is always growing, allows it to scale and handle a wider variety of tasks than ever before.
However I think that often the overwhelming number of choices can lead to a paralysis of choice for some people. Or confusion about what to do or what makes sense. Present my daughter with 2 candy machines and she can pick the one she wants in an instance. Take her into a 7-11 with an aisle of candy and I’m still standing there 15 minutes later waiting on a decision if I haven’t forced the issue.
I think that one area of improvement that could really help with many of the possible choices for completing a task is to have some strong guidance from Microsoft (based on customer pilot testing and the reasons for implementing a feature) that would explain what the defaults should be. And would have those defaults set up as the default choices or actions for the product.
Those of us with reasons to make changes can, and will do so. But those that don’t know what to do would have more guidance right out of the box.
Steve Jones
BTW – No offense intended to the cat-people out there.
The Voice of the DBA Podcasts
The podcast feeds are now available at sqlservercentral.podshow.com to get better bandwidth and maybe a little more exposure :). Comments are definitely appreciated and wanted, and you can get feeds from there.
Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.
