Voice of the DBA

Tag: Database Weekly

  • Saving Hard Dollars

    This week I ran across an analytics piece that talks about how Big Data and analysis can do more than help marketing, it can also create hard dollars for your company. Companies are using sensors and technology for mundane tasks, but tasks that can save the company substantial, hard dollars.

    You may have heard of UPS and FedEx optimizing their routes, and tracking the actions of drivers. Those companies (and others) have learned that a little programming work to analyze and create instructions for their workers can reduce wasted time and fuel, which ultimately means less wasted money. Other companies have reduced their heating or cooling costs with more data and small investments.

    I don’t know how many additional cost reductions can take place in most companies, but there are likely changes none of us have thought of that could be easily implemented with technology. It seems like a no-brainer that technology can build better routes, but early on, it wasn’t considered as an option because it didn’t replace a formal manual process. It wasn’t worth paying someone to design routes each day for a truck.

    As computer power increases, and more data becomes available, I predict that we’ll find many ways to analyze that data and change the way we run businesses. Many of the ideas will be unexpected, and some may be counter-intuitive, but those of us working with data will have a front-row seat to the ways in which data analysis can transform our world.

    Steve Jones

  • Watson Freemium

    I’ve been intrigued by the Watson project from IBM. It was quite a coup to see the platform win on Jeopardy. That’s an amazing accomplishment for machine learning, though it does seem like the investment and effort to set up the platform and experiment with it was more than most people could afford.

    However this week I saw an announcement that IBM has introduced a freemium version of Watson, aimed at people looking to work with analytics. This product allows users to upload some data and ask natural language questions, which Watson will answer.
    I have no idea how this will work, but I like the model they’ve taken. We can upload limited data and experiment with it. If it works well, we can subscribe, pa some money and upload more data and get more complex analysis. As an IT person, I think this is great. End users can play with it, and I don’t have to mess with a proof of concept. If it works, I can get involved, help automate ETL, query structure, tuning etc.
    It’s a changing way of working with analytics, and one I welcome. To me, as business people play with technologies like this, they become more savvy and willing to invest in technology. And as they demand more, they create more opportunities for IT people to help them, not less.
    Steve Jones

  • The Flash Database

    I caught this piece on the impact of flash storage on database engine design this week. It’s interesting as there’s been debate for years as to whether SQL Server should alter its behavior if it detects SSDs being used for storage instead of physical disks. It doesn’t, and perhaps that’s fine, though the article makes me think there are performance gains to be had if behavior changes.

    The article really looks at a few of the NoSQL products, though the design changes aren’t necessarily specifically limited to those products. I particularly thought that two ideas in the piece were interesting: indexes in memory with data on disk and the realization that threading can be he bottleneck with SSDs. I’m not sure if Windows and/or SQL Server could use these ideas, but they are interesting.

    I do wonder sometimes if a little more control of indexes would be helpful in SQL Server. Imagine if I could limit a large slice of memory to strictly non-clustered indexes and then have other data on SSDs. Would there be a way to tune SQL Server to run better for some workloads? Perhaps the algorithsm that choose query plans would change if they knew a scan of an NCI could be completed in a fraction of the time that a seek took place on a CI? Maybe we’d be willing to perform more seeks on in-memory indexes before performing lookups on disk.

    The idea of more concurrent operations, requiring more threads, also seems to be an area where I could suspect that both Windows Server and SQL Server could benefit from SSDs. If the systems changed their read and write algorithms and used many more threads with SSDs, could we get more throughput? Should our systems be more aware of how many controllers and paths might be on a system? I wonder, especially as some of this hardware becomes cheaper and cheaper. I could certainly see more organizations looking at using lots of smaller SSDs for a few servers that require high performance than a SAN.

    However it’s not as though SQL Server isn’t trying to take advantage of technology changes. The In-Memory OLTP system  and Buffer Pool Extensions in SQL Server 2014 are designed to take advantage of more memory and SSDs to dramatically improve performance. I don’t know what else might be coming in the next version of SQL Server, but I do hope that as new ideas emerge, SQL Server considers taking advantage of them.

  • Data is Not Always Enough

    As data professionals, our jobs deal with ensuring that there is data available, accurate, and relevant to the organizations in which we work. However we often go beyond the simple maintenance, gathering, and manipulation of data to help our clients and customers understand the information contained in our systems. In many of my positions, whether as developer or DBA, I’ve been tasked with working closely with business people to make decisions based on data.

    I’ve always been of the opinion that more data is better, though I certainly understand the problems of Big Data and the potential to reveal false patterns or overwhelm systems with conflicting information. The better I understand the domain, and the better educated the business analysts are, the better we can work to extract information from databases.

    However that’s not always enough to help an organization act. I read an interesting piece this week on data in sports, which talks about one of the NHL team’s managers using data to make decisions on how to restructure his team. It’s interesting to see that additional data gives the general manager more insight, and leads him towards a decision, but the rest of the organization can’t follow through. In this case it’s a matter of money, but resources constrain our efforts in many decisions. I think this is a place where more flexible, and perhaps more in-depth, BI-type analysis of what-if scenarios can be more helpful.

    The other piece that caught my eye had to do with real estate, and housing prices, with the author bemoaning the lack of data in determining the value of a house. It’s interesting, but to me, it’s flawed. More data won’t help because houses aren’t like many other commodities. One house is not fungible with another one, and the market is both fast moving, and inherently full of friction. We rarely buy a house without visiting it, a task that consumes time, and slows the movement of information through the system. I’m as frustrated as the author, but I don’t know that more data would help in this case.

    Data is important, and it should be a part of our decisions, but we should remember that data isn’t necessarily going to make the decision for us. We need to be ready to incorporate our own knowledge and judgment into a data set to help us decide on a course of action.

    Steve Jones