Voice of the DBA

Tag: databases

  • Data Quality on the Open Web

    Input from customers like this must include data quality checks

    I used to hear that one of the strengths of Linux was the thousands of volunteers that would help you get a patch or a fix in record time when you reported an issue. That worked well, but not well enough for many companies that really wanted a company to stand behind patches. A few companies, like Red Hat, sold support agreements with the “free as in beer” OS that ended up costing companies almost as much as a regular license of another OS. While Linux is a great system, it hasn’t taken over the world like many people thought.

    Lately there’s been a different flavor of open-ness on the Internet. It seems that so much of what we read and is pushed out to us as news or information is based on the crowd-sourcing of what’s popular. Facebook shows a “most active” view, Twitter has trending topics and re-tweets, and many news sites like Reddit use a crowd voting mechanism to help determine what you see first on their front page.

    However there’s a downside to using these open systems. There’s the potential for abuse when a group of people get together. Google started using the open model on it’s map services to allow people to add businesses to maps. A very handy feature, butthe addition of a “mark this as closed” button allowed people to abuse this privilege. Whether it was competitors, vandals, or some criminal element isn’t known, but apparently the quality of data Google is providing on maps isn’t necessarily accurate. With many people using maps on iPhones and Android devices, this could damage businesses that add themselves to the mapping service. I think Google is playing a little fast and loose with their crowd voting on data points, but with so many companies looking to capitalize on the social networking phenomenon, I’m not surprised it’s being abused.

    Whenever we build systems that take input from users, we have a maxim: garbage in, garbage out. Essentially we aren’t responsible for bad data, but many companies won’t feel that way. They will still feel that we ought to be better policing the data quality and not showing bad data in reports or downstream systems. As more and more companies look to incorporate data from customers into their systems, it becomes more important that data professionals incorporate automated scans and manual workflow checks before data moves from staging areas to prevent incorrect data from affecting our production systems.

    Steve Jones

    The Voice of the DBA Podcasts

  • The Growth of Data Types

    What data types will use the most space in the future?

    In the relational databases we have all different types of data that we store. We have the basic types of integers, character data, dates and times, and more. As the RDBMS platforms have evolved, we have also added spatial, XML, and other types of data that build on these base types. By classifying them as different types, even if we are storing numbers, or characters, we can more efficiently work with these data types.

    As the world changes and expands, I think we will end up with more and more data that takes advantage of different data types, and perhaps even adds more. This week, I wanted you to look forward, and predict what changes we might see in our data storage in the future.

    What types of data do you think will occupy the most space in our future databases?

    Will we see more video files in our databases? More audio, spatial, or will plain numeric data continue to dominate the majority of the space we use in our data files? I used to think that numeric data would dominate, but the sheer sizes of some of the binary data types, along with the increasing use of these formats in applications makes me think that some type of binary data will dominate in the future.

    Steve Jones

    The Voice of the DBA Podcasts

  • Data Matters

    A very interesting book about technology and business – from inside Google.

    I recently finished reading In the Plex, which is a great look inside Google and how they operate. For a company that has had so much secrecy around it’s internal workings, this was an amazing look at how the company operates. If you’re interested in Google, or even in how a technology company might grow and operate, or even just curious about the decisions that might affect our lives with regard to digital information, this is an interesting book to read.

    There are lots of things to talk about in there, but one thing in particular seemed to stand out to me. I n many cases, the justification for getting some project approved, or moving forward, is different than in many companies. It’s not the personal relationship with the person in charge, or because it seems like a good idea, but because the data matters. That’s a quote used quite often in the book.

    Those of us that work as data professionals know that data is important. We spend lots of time and effort to cleanse, standardize, and organize data in a way that allows it to be used as information for better decisions. That’s the fundamental idea behind business intelligence and data warehousing: extracting information from data and using it to run your business better.

    However I suspect that many companies don’t have management that really understands or truly appreciates the value of the data stored. They don’t often devote the resources, especially time, to allows data professionals to build better systems that people can use. We struggle to purchase enough storage and struggle to make writing better T-SQL or tuning the systems a priority. Most importantly, we often don’t have the time to spend just working with data in different ways to try and build patterns.

    Data is important, but without having the ability to turn it into information, most of that value remains locked away.

    Steve Jones

    The Voice of the DBA Podcasts

  • The Career Path

    Which career path would you like?

    Most people seem to wander through their careers, taking any job that comes their way. Others have a tight focus, often with a larger goal of ending up as the owner of a company or perhaps the CIO of one.

    However for the technical person, someone that wants to remain employed for a long time, but doesn’t want to oversee others, there isn’t a clear path. So many DBAs, developers, and other people have no interest in moving into management, but they still would like to progress in some fashion to other jobs.

    This Friday I wanted to ask what that progression might be like if you had no restrictions from HR and could design your own set of jobs:

    What would you like your technical career path to look like?

    I am wondering if you think there ought to be some way to advance to a senior person, maybe with multiple levels after your title. Perhaps some of you would like to move into an architect position or even be seen as some technical fellow after some amount of time.

    Think about the way that a technical career might advance, without going into management, and let us know in which direction you wish you could move.

    Steve Jones

    The Voice of the DBA Podcasts