Voice of the DBA

Category: Editorial

  • More CPU Means More Data

     

    sailors
    Better modeling requires more CPU and a lot more data.

    Moore’s law has held true for computer resources for a long time. I don’t know how long it will continue to work in the future, but I am interested to see if it can continue to work for the rest of my career. I don’t know that it will, with our newer CPU designs approaching the limits of physics, but I’m sure that research is under way to find ways to vastly improve our processing capabilities. Look how far we’ve come from the Apollo guidance computer, which landed 3 men on the moon, and had a fraction of the power of almost all of today’s cell phones.

    I ran across an article that talked about modeling sailing conditions to help sailors design faster and more stable boats. Just like many other types of modeling, this is a way to use computer simulations of various real world events to test designs of new products. This can save lots of time and money as fewer prototypes can be built and many more design ideas can be tested before a final design is chosen. More intensive modeling means more processing power is needed, but it also means more data is needed.

    We see this more and more in business. We want to get answers faster, often by running more complex business simulations. To get our answers faster we need more processing power. However as we upgrade our machines and enhance our CPUs, we also add more and more data to our databases. The additional data, while beneficial to making better decisions, also slows down our queries. The additional data is also a challenge to load, manage, and store, which is something we need to ensure we are learning to do better.

    Processing costs are going lower and lower. These days with the large clusters in the cloud that anyone can rent for a few thousand dollars, extremely complex simulations can be run. The code to run them, however, is still something that we need to produce. Just as the scientists  have developed complex models that accurately test new designs, each of us must learn how we can write more efficient queries that analyze our business data and take advantage of the constant improvement we see in processing capabilities. Otherwise the advances in CPU power might be negated by rapid data growth.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Creativity

    down tools suggestions
    How many ideas would your group come up with in a month of creative thought?

    At Red Gate we have “Down Tools Week” a few times a year. This is the same idea as the ShipIt days at Atlassian or the 20% time at Google; it’s a chance for employees to have free reign to work on some project, approved as being semi-business related, with the chance to present their work to the company at the end. I think it’s a really cool idea, though I’ve yet to come up with an idea that I’d want to take a week off to work on.

    When I look at companies and think about the time spent on various projects and efforts, I think much of it is useful, but I’m not sure it’s critical. All too often we get caught up in the idea that getting project X done or finishing a piece of software will make or break a company. The reality is while the item might be important, most of the time it will be later than scheduled, and the company will survive. I’m sure there are exceptions, but they’re rare. Most of the time things run late, which makes me question whether or not managers should be pressing so hard to get every project finished on time.

    What if you shelved all work from your developers (or markers, business analysts, etc.) for a month? What if you let them think about what they could do to make the company better, or improve operations, or maybe just find a way to make their job easier?  Would they innovate or just waste time? 37 Signals did just that, taking the month of June off to let people that weren’t running day to day operations work on whatever they wanted. They got a lot of ideas, some of which they are going to implement.

    It sounds crazy, but if a small company can do it, one where everyone wears multiple hats, couldn’t you do with with your department? I’m sure you could. Now if you can only convince management that “thinking time” can be just as important as the rest of your daily job.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Mining and Algorithms

    Amazon
    The recommendation engine is the result of some complex algorithms

    When we publish articles that look at the performance of T-SQL code, the pieces are very popular. It seems that people are always willing to learn how to write better code, and there is no shortage of people willing to discuss the methods, the results, and the implications of various techniques. It also seems there is no shortage of argument and debate that take place in the discussions as to the best way to solve problems.

    However we don’t really understand what will resonate with people, or what things they’d like to learn. What we would like is a system like Amazon’s recommendation engine that might help us improve traffic, and also bring you information that is more relevant and interesting to your career. If you’ve ever purchased something from Amazon and then browsed their site, you might notice the recommendations appearing all over the site. If you’ve received an email from them, you might see some interesting items suggested to you.

    Amazon knows their engine is valuable, and it’s constantly being refined. We don’t know exactly what they use to build the recommendations, but they are constantly analyzing the results and tweaking their system. That makes sense for their commerce oriented business, since they can easily measure the results by looking at sales, but it might also make sense for other businesses. Many of us have reports, and other queries used to perform analysis on our data. Do we regularly analyze the performance of those algorithms and look to tweak them?

    I’ve found many employers in the past were content to develop a system and leave it alone until some new requirement arose. One company didn’t like that approach, and regularly looked to tweak our systems to try and get more performance out of an inventory analysis application. It was frustrating to regularly rewrite a series of reports, but also challenging to try and find ways to solve a complex problem.

    As more and more data professionals begin to work in the analysis and business intelligence areas, learning to build algorithms, and tweak them over time, will be more and more important. It will be challenging, but it should be fun if you are excited by the challenge of improving your algorithms over time.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • A Cloudy Future

    crystal ball
    Can you divine what’s important for your career in the future?

    Those of us in Information Technology are constantly learning to do more with the tools and platforms we have. There are no shortages of new systems to manage. Since staffing levels never increase as quickly as servers, we must find ways to manage and maintain more and more systems all the time. That’s fine since tools like Powershell or MultiScript allow us to extend our reach to many machines as easily as we work with one machine. That’s not quite accurate since the effort to build a repeatable process is greater than performing a task one time, however I have found that most of the time if I need to perform a process once, I’ll need to do it again.

    Over time I’ve learned that investing in my own skills, and learning to work smarter and more efficiently, has paid dividends for me. Becoming more familiar with the business my employer is involved in has helped me to return more value to the company, as well as ensure that my employment is appreciated. As companies look to virtualize systems, move to cloud services, or find some other more efficient way of dealing with computing resources, it’s up to each of us to find new ways that we can provide value. That’s what system administrators need to do, and I suspect DBAs need to be thinking along these lines as well.

    The world is getting more complex from an IT perspective. These days companies have their pick of candidates for many jobs, and the person that has a variety of skills in multiple areas with a history of solving problems is desirable. Even when companies have few candidates, they seem to be waiting to hire a candidate that is a good fit rather than picking the first person they find if that person lacks some of the skills the company needs.

    As you continue along the path of your professional development, think about adding a variety of skills in different areas, skills that improve your efficiency, and allow the business to get more of their work done every day.

    Steve Jones

    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.