Voice of the DBA

Category: Editorial

  • The Need for DRE

    “Today’s database professionals must be engineers, not administrators”

    That’s a quote from an interview at O’Reilly on Database Reliability Engineering. I don’t want to quibble about whether those of us working in technology are engineers in name or not, but in function, I do think the world is changing to one where we need to do more than be caretakers of systems. We need to be actively involved in ensuring we have very reliable, well-designed systems that use the best practices and patterns of highly available and reliable database systems.

    There is advice to learn and work closer with developers. I like that, and I do think the idea of learning to script and automate tasks is important, perhaps more important all the time, for data professionals. Some of the advice to look towards the quick moving, limited lifetime of many software components today doesn’t make sense. A database, at least some parts of a database, don’t fit within that model. There are valid reasons why a datastore must provide some stability and persistence for an application and won’t fit within a model of multiple systems. Or at least not practically within a model.

    However, there is good advice in the piece to learn more about different types of database services and patterns, and consider other ways of implementing datastores other than a RDBMS. I wouldn’t necessarily abandon a RDBMS for some NoSQL store just because developers think it’s easier, but I would consider whether I actually need to very tight coupling between different parts of the database and complete consistency. I think there are a fair number of domains where a CQRS pattern or some distributed store would work well. The move to microservices might be an enabler for your business if you consider the advantages for OLTP type transactions.

    The flip side of all the power that many stores provide is that many also make reporting and aggregating information more difficult. I would venture that in many cases, a data warehouse (perhaps even a RDBMS-based or columnar store) is necessary, along with the ETL process necessary to keep it up to date. These aren’t simple processes, and take resources to build. For most of us, because we don’t work at extreme scales, I’m not sure it’s worth leaving a relational platform, but I do think that we can learn to evolve and enhance our relational databases faster with DevOps ideas and techniques.

    Steve Jones

     

  • Your Job Description

    In every job I’ve ever held, there’s been some scope creep in my duties. No matter what I think my job is, I end up handling tasks in very different areas. While bartending, I’ve had to cook food a few times. While managing a network, I ended up performing some software development. As a DBA, I’ve assembled network cables. As a manager, I’ve had to arrange and setup dinner banquets for late working employees. Even at SQLServerCentral, I used to have to make sales calls when I expected to be more of a technical worker.

    That’s the sign of a good employee. Someone that pitches in and gets the work done as needed, doing their best if the task is out of their skill area. It’s what I’d expect from most employees, though I’ve never worked with a union. Rules might be different for union employees, though I’d hope most would pitch in in a crisis. I also think this is something I look for when I interview people. I hope to get the feeling from a candidate that they’d be willing to learn to handle tasks outside of their primary position.

    I was thinking about this as I read a post about job descriptions. It talks about the mess that many job postings contain, with a wide variety of requirements and duties. I think many of these descriptions, especially in technology are the result of too many chefs working on the final product. From what I’ve seen, most of the descriptions need a mythical candidate in order to meet all requirements. Companies will never find this person and instead compromise their choice based on who applies. So, always apply if you meet 50% of the requirements.

    This made me wonder about the descriptions that are out there. Some of them are really crazy, but I think some of the issue is there are people that want to make their jobs sound more important than they really are, just as they may do with their job title. Keeping this in mind, what would you say is your job description? Is that what you’d list in a job posting or would you add more or take away some of the items?

    I used to want a more important title, and a more complex job description as I was growing my career. These days I’m less concerned with pumping up my job, but I think that’s part of the cycle of a career. I’m curious where you are in your career and how you might describe what you actually do to others.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Securing the Things

    I would venture to guess that a lot of the data that will be produced in the next few years will come from various “things” that are implemented by companies, users, and governments. We tend to refer to the Internet of Things (IoT) as those computing devices that are small, specialized, and connected computing device that sends data elsewhere. Sensors, special purpose devices, and more make of the IoT spectrum, including many industrial and manufacturing hardware.

    Computers grew by leaps and bounds from approximately 250 worldwide in 1955 to one million in 1980 and 30 million by 1986. In 2015, the world saw 238 million computers sold worldwide. If we add smartphones, we had around 120 million sold in 2007 and 1.4 billion sold in 2015. Tablets add to this, but really, the large market is going to be IoT devices. 15 billion connected in 2015 with an estimate of 75 billion by 2025. I would wager that is going to be a low prediction.

    One of the major concerns with these devices is security, and with good reason. There have been hacks against many of these devices, and few have been designed or sold with security in mind. Quite a few devices assume a strong network perimeter inside of an organization, but history has shown us that these barriers aren’t always secure. What might be more disconcerting is that many of the hacks originate inside of the network boundary, whether from insiders or compromised devices. There’s a nice write up on the need for security with IoT. Microsoft recently proposed that we regulate privacy and security for the IoT devices.

    Many of these devices will generate data that we will use as data professionals. In fact, I expect a tremendous amount of data to be generated. If devices are hacked, we may end up with lots of suspect data. Even devices are secure, they may fail or have intermittent errors. How to we detect errant data and remove it from our data sets? How can we be sure anomalies aren’t just problems or failures in the device (or network)? These will be challenges for us moving forward.

    I don’t know that I think specific regulation is required, but I’d like to see some sort of framework devised. I think encryption should be required, as well as guidelines for the security of connections and data management. Perhaps a list of valid technologies could be used, growing and changing over time, including removing older items that might be outdated. For example, DES shouldn’t be allowed in any new deployments, and it really should be removed from old ones, though with some grace period. That means a regular investment in software development and upgrades that evolves our systems over time. Especially with regard to data security.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Software Has Bugs

    Every once in awhile I have someone I meet complaining about software quality, usually with regards to some piece of Microsoft software. I’ll invariably hear how there shouldn’t be bugs in SQL Server or Windows, and Microsoft isn’t testing their software. This person will claim that they write software with no bugs. I can’t remotely believe this is true, aside from small, somewhat trivial applications. All software has bugs, but there’s no way to disprove anyone’s assertions unless you can get to their software and examine it.

    I do think that some people think their software is more bulletproof than it really is, perhaps because no one really does test the limits of their work. It’s easy to assume that there aren’t problems when no one has ever probed deeply in the different ways the code might execute. After all, this is one of the reasons that developers turn in code that they think works, but in which others find bugs. They haven’t explored enough different inputs against the software. This is also why I think unit testing needs to be constant and evolutionary, growing and changing as you find new issues and bugs in your code.

    Recently a vulnerability was found in the Samba software, and apparently it has existed for seven years. This means lots of software was vulnerable, and everyone should be patching as soon as possible. I don’t know how many people have port 445 open to the Internet, or even open locally, but plenty of people might now know about the issue, so patching makes sense. Scans of the Internet have turned up hundreds of thousands of devices, some of which are running versions that can’t be patched.

    This isn’t meant to complain about Samba, but point out that this is code that is open source, lots of people have looked at and developed against, and I would guess no shortage of security people have checked this for potential vulnerabilities. And still a bug slips through.

    Security isn’t hopeless, but it’s also hard. Software development, with few boundaries limiting the choices developers can make, allows new interactions among the building blocks of code that have never been seen before. Building code is much, much more complicated than any physical structure because we don’t have any of the inherent limits of the physical environment. This means that we will, and do, struggle to test all of our software.

    I don’t think we’ll ever eliminate all bugs, but we can vastly improve quality (as many companies have), by moving to a more DevOps development style where we seek to constantly improve quality as a part of software development. We also have more and more developers also seeking to improve their skills, learning new and better patterns throughout their careers, and hopefully, building better software.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.2MB) podcast or subscribe to the feed at iTunes and Libsyn.