Voice of the DBA

Tag: big data

  • Too Much Data

    Big Data is getting lots of attention. Will that mean more resources for managing it.

    Many data professionals these days feel like they are working with too much data. Queries run slow, storage is constantly a concern, and our hardware is outgrown on a regular basis by the sheer amount of data that we need to manage. It’s a good thing that “Big Data” is in the news since it focuses more attention on the issues of data management and growth in our systems.

    Some companies are taking the idea of managing their data more seriously. Boeing and Nike are talked about in this piece about coping with large amounts of data. With the vast amount of growth in the data stores these two companies work with, they had to find ways to ensure that business people could effectively find the information they needed in a sea of data.

    It’s an interesting read and one that you might pass on to your management. The solutions aren’t new and innovative; they’re back to basics ideas. Have a central warehouse, use a data strategy that includes strong models and enforces data quality. Integrate your systems to allow a better view of your entire enterprise. These are the ideas that many data professionals preach, but are rarely found in many companies.

    Sometimes a new perspective helps, and articles like the one above can help you convince managers of the need to raise your data quality and enforce some standards.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • What is Big Data?

    big data visualization
    Big Data visualization of “water” being spoken to a child.

    This industry is full of some very smart people. We tend to get some very strong opinions, and some rather snarky comments at times about the way the technologies that we use are presented. Karen Lopez (@datachick | blog) recently made headlines with her comments on big data. “What the heck kind of definition is that?” she asked when looking at the Wikipedia definition of Big Data. If “big data” is data that is “awkward to work with” then I know lots of people that have had to deal with big data anytime they had to work with outer joins.

    While I’m not sure big data is well defined, I do know that there are large data sets which can overwhelm relational databases, both in scale and the speed at which the data is collected. Microsoft added StreamInsight to SQL Server to help deal with the speed problem, but I’m not sure that relational databases can handle the size issues at the same time. Some large computing and analysis problems don’t lend themselves well to non-relational soluttions, and other technologies are needed.

    I do agree with Karen that big data, whatever that means to you, is not really in conflict with relational databases like SQL Server. There are lots of data sets that work well inside of a relational database, especially when you have built a solid data model and referential integrity. Many problems with relational databases come about because of poor code added to the platform, not a problem with relational platforms in general.

    Relational database systems are not “obsolete”, despite what some vendors of other databases think. I have complete confidence that SQL Server as a relational platform will be used for years to come.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.