Voice of the DBA

Category: Editorial

  • Security Decisions

    How many of you have written code that results in a security issue or data breach? It’s possible many of you have, but are unaware of the issues. I’m sure a few of you have been hacked, have had to clean up data or code, but often it’s not your code. Maybe it’s something you inherited. Maybe it’s code your team wrote together. In any case, the feedback loop between your action (writing code) and the result (a security incident) isn’t tightly coupled.

    I ran across a post from Bruce Schneier on how people learn about cybersecurity. The piece links to a research paper, and it’s an interesting read. It turns out the researchers see most non-experts learning from news, friends, and websites, often with biases towards actions that have had immediate negative consequences, but not necessarily those that are more serious.

    That has me wondering about us, as supposed expert, or expert-ish, developers and DBAs. How many of us get security training, or get this training updated? How many of us learn from friends, or websites, and re-use this knowledge over and over in our work, not necessarily understanding, or unsuring, that we are building in strong security into our systems. I suspect many of us just try to get by, writing the minimal level of security that “works” for us, not really understanding how there might be flaws or holes in our system.

    Our code, our configurations, our systems have much farther reaching impact than ourselves. In some sense, I think that a fundamental broken concept of information technology is the lack of security practices and architectures being built into our platforms and applications from the start. While convenience may get the boss off our back, or allow greater profit for our companies, it’s not helping our industry, or even our companies in the long term.

    I just wish I had some idea on how to change things.

    Steve Jones

  • How Much Eventual Consistency Do You Have?

    There’s a never ending debate about NoSQL v RDBMS systems that seems to polarize the people that prefer one technology over the other. In fact, it seems that every time I talk to someone that dislikes NoSQL technology in general, the topic of eventual consistency comes up. The idea that not all data in our system might be up to date seems to be one of the concepts that scares many DBAs most of all.

    I was chatting with some people recently about complex SQL Server configurations, and the topic of replication came up. While this is a technology with so much potential, it seems that replication has been a bit neglected by Microsoft and is both amazing and brittle in its implementation. However, if you think about it, replication results in data that isn’t consistent across the systems.

    I’m sure many would argue that this isn’t an issue, but how many of you have businesses that make decisions or have processes built on data in replicated databases? I’m sure plenty of you, and most of the time, the data is consistent enough for use by our clients.

    There are plenty of ways in which we implement data movement across our databases that results in potentially inconsistent results for our clients. In fact, I’ve had no shortage of discussions with clients that can’t understand why two reports run minutes apart show different results.

    Today I’m curious how many of you have systems that your businesses depend on where the data is eventually consistent because of some technology that moves information from one database to another. Perhaps you might even share some of the tricks you use to ensure that delays or problems in your transfer process are detected and fixed before your clients realize just how inconsistent their data might be.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Are We Engineers?

    This is probably a topic that is regularly debated in programming forums, but it’s still one that I think is worth revisiting regularly as our industry evolves. For those of us that write code, that develop software, are we engineers?

    That’s the topic of a piece in the more mainstream media. It’s a bit of an attack on the software industry in general, noting that engineers are typically certified and regulated. They are subject to continuing education and apprenticeship. While certainly many of us continue to learn, it’s very informal, and not structured. Our education certainly isn’t certified in any meaningful way, though we often aren’t even really liable for our work. The cost of bad code is usually a job, with another one often easily found.

    Engineering is defined as:  the work of designing and creating large structures (such as roads and bridges) or new products or systems by using scientific methods. By that definition, I think we are engineers. We use scientific methods, primarily the hypothesis, test, (usually) fail, change the hypothesis. The hypothesis we use are lines of code, which are often wrong and need work to get fixed. A additional definition includes: the design and manufacture of complex products <software engineering>. That’s us. Even the most trivial application today is likely seen as complex by layman. If most of us bothered to dig into the libraries we use, we might see them as very complex.

    We certainly aren’t engineers in terms of the formal knowledge and liability that exists in our profession. There is little, often because we can fix problems quickly. Or maybe because even if our systems fail, issues can be undone by additional work. That’s not always true, but it does seem as though those systems which are costly when they fail (medical, financial systems), have some recourse available to those affected. Not always, but there isn’t always recourse in the analog world.

    I wonder if we’ll get to the point where we require some evaluation or measurement of skills (or code) from software engineers for certain disciplines. My guess is not, as the world of software changes too quickly, and allows for too many different scales of acceptable performance to really certify anything. My guess is the world will continue to struggle with determining the liability and responsibility of software failure as we depend more and more on computing infrastructure in our world.

    Likely this is an argument that will never end.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.1MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • An Inconceivable Scale

    One of the talks I give on SQL Server deals with unstructured data. I start out this talk looking at the scales of data we deal with and was been amazed by the research I did about how much data we humans have created. What’s even more interesting is that the growth is outracing predictions made just a few years ago.

    When I started working with computers, we talked about kb of data, thousands of characters. That’s an amount of data we humans can easily comprehend. In fact, we used to talk about floppy disks and the number of average sized books that could be stored in kb, or single digits of MBs. As humans, we can comprehend that scale. Most of us have seen hundreds or thousands of books in a library.

    When we move to GB, things get harder, though at 4GB for a DVD, many of us can conceive what multiple GBs can mean. However terabytes? Can we conceive the scale of data? Sure. A TB is about 40 Blu-ray disks. While we might not appreciate how much data that is, we can picture it.

    A PB? That’s 41,000 Blu-ray disks. I can’t even conceive of what that looks like, much less imagine the billion MB sized pictures.  That’s a scale that has no reference. However as humans, we will create multiple exabytes of data this year. As individuals working with data, few of us will work with EB in our organizations, but some of us will. I read recently that Paypal processes 1.1PB of data regularly. Regularly processes, not just has in cold storage.

    We have zettabytes and yottabytes, but who could possibly conceive of what those mean? There’s not frame of reference I can imagine, though that may change. I expect that we will become used to PB at some point, just like a TB is no big deal right now. In fact, I really think I’ll see a TB on my phone sometime before the end of this decade.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and LibSyn. feed