Voice of the DBA

Tag: rdbms

  • The RDBMS is Often the Right Tool

    I’ve spent most of my career, actually most of my life, working with relational database systems. I tend to agree with this piece on NoSQL systems being the wrong tool for many applications. I do think that’s the case, though to be fair, I haven’t done anywhere near the amount of work on key-value stores, document databases, graph databases, etc. as I have on relational systems. I admit I’m biased, but I think the author is as well. He works at MemSQL and used to work on SQL Server.

    The piece notes that relational databases didn’t deal well with certain problem domains and various NoSQL classes databases evolved to handle the challenges. The good things about this is that relational databases were forced to grow and adapt, incorporating new features and functions. I do truly wonder if that is the reason we’ve seen rapid changes in SQL Server, both in capabilities and the pace of new releases.

    Many of the non-relational systems are built to incorporate transactions, but usually within a very limited range. For example, many document databases only allow ACID compliant transactions within a document, not across documents. The same might be true in graph structures. Many of us working with relational systems know there are often a decent number of queries that need transactional consistency across entities, and sometimes across different databases. Our applications demand this, and while it might be overkill at times, it’s a capability that is definitely needed.

    There is also a discussion of schema-less features. I do understand the attraction for application developers. Adding or changing the way data is stored is much easier when there isn’t a set schema for writes. This truly does make development easier. What it doesn’t do, however, is remove the technical debt of having a schema. Instead, this debt is pushed to the reader, and often, means the application must maintain the ability to apply some schema for reads to make sense of the data. Does that mean everything needs a hard, strongly defined schema? No, but I think it means we need some schema.

    Ultimately I think that there are problem domains best solved by different platforms. In most cases, I’d argue a relational data store is the better platform, but I am certainly willing to entertain the discussion of where a graph, document, or other data platform may work better. Just be sure you argue for the problem being solved. Most of us aren’t operating at the Google/Facebook/Spotify/Netflix scale, and we have different problems to solve. We need something that works well for our problem, not the one another organization is facing.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.9MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • MoSQL

    Google is doing more SQL, or at least shifting towards relational SQL databases as a way of storing data. At least, some of their engineers see this as a better way to store data for many problems. Since I’m a relational database advocate, I found this to be interesting.

    When Google first started to publish information on BigTable and other new ways of dealing with large amounts of data, I felt that these weren’t solutions I’d use or problems that many people had. The idea of Map Reduce is interesting and certainly applicable to the problem space Google had of a global database of sites, but that’s not a problem I’ve ever encountered. Instead, most of the struggles I’ve had with relational systems are still better addressed in a relational system.

    Google feels the same way, and in a blog, they talk about choosing strong consistency where possible. This is a post that promotes their relational SQL database (Cloud Spanner), but there is a good discussion of why consistency matters and where moving to popular NoSQL models, like eventual consistency, cause a lot of problems. Both for developers and clients.

    This quote caught me eye, and I may use this with developers that look to avoid RDBMS systems: “Put another way, data stores that provide transactions and consistency across the entire dataset by default lead to fewer bugs, fewer headaches and easier-to-maintain application code.” I think that’s true as I think many of the advantages promoted in non-RDBMS systems are often placing a greater burden on the application developer than they realize. A burden that grows over time as the techniques used cause more technical debt.

    I think more SQL based systems are the way to go for many problem domains. Google agrees, and if you read more about the Cloud Spanner team, you might agree as well. You’ll also find them to be incredibly smart people that think deeply about the problems that are both raised and solved by relational systems.

    So go ahead and promote more SQL Server databases. Google thinks they’re good for many applications, and that’s good enough for most developers.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.1MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Defending the RDBMS

    A few weeks ago I ran across an essay from Randolph West called, Relational Databases Aren’t the Problem. This was a response to another essay that made a case for relational databases being bad for many businesses. I thought that both pieces were interesting for different reasons. Certainly I don’t believe the the RDBMS is perfect, and it certainly can be hard for developers to build software that interfaces with a relational system.

    The original complaint about the RDBMS is somewhat rambling and deceitful, in my opinion. It is an excellent study of how to use a few concepts to confuse and create doubt in a casual reader. If I weren’t reading closely, I might fall for a number of the issues that exist with relational databases. However, in my mind, part of the issue is that quite a few of the issues that are discussed aren’t problems with relational databases, but often the issue with poorly developed software or design of the entities and relationships. I find myself even more disappointed that the author hasn’t really addressed any comments, but rather just pasted a link to his followup article.

    I do think that the defense from Mr. West does a good job, though it also misses some of the primary issues we struggle with relational databases. There are problems with the knowledge of how to build a well performing database, both from application developers that view this as a necessary evil as well as experienced database developers that don’t regularly improve their skills and try new design techniques.

    I also think that both of the pieces fail to address the issues of gathering and working with multiple rows of data. The second discussion of “doing without databases” really implements its own database management structure, which may work well, but is fraught with issues such as the concurrency issues of multiple users searching and scanning through data without having indexes. While indexes are overhead, they are necessary as hash buckets aren’t necessarily feasible for all the properties in a class. Also, if you end up building them for multiple properties, you’re building an index. There’s another good defense of some of the issues here.

    I do think that keeping more data in memory and synchronizing access to structures sounds great, but scaling that out to multiple systems, and ensuring consistency at high volumes, not to mention potential loss of data issues from crashes are a problem. Having a write ahead log in SQL Server does a wonderful job of ensuring we can handle redo/undo on system restart. The method presented doesn’t necessarily ensure this, though perhaps accepting some data loss from high concurrency changes is OK for many applications.

    I will say that the idea of all data in memory is interesting. I had to stop and think about how many databases really have more than 1TB of data. If we throw out indexes, does this cover most data stores? I bet this does, though that doesn’t mean that there aren’t issues with using in memory array structures, with widely varying data sizes.

    Would I use an in-memory data structure for software? It’s tempting, but honestly, I wouldn’t. The value of data is too high, with potential issues from poorly implemented ACID control structures. Plenty of issues have been found with different RDBMSs over their years, and even some in NoSQL systems. Thinking that I could avoid any issues and protect data is something I wouldn’t even try. After all, if there is some error, I’d prefer it from a system that many people use, rather than one I tried to emulate for no good reason.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 6.2MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Losing Rows

    There has been a lot of hype in the last 3-4 years around NoSQL databases. Note that NoSQL isn’t Not SQL, but rather Not Only SQL, implying that the SQL language still has a place here. In fact, a number of companies that make NoSQL products have layered a SQL-like interpreter or interface on their products to allow familiar SQL language querying.

    One of the big reasons companies look at NoSQL is that the various types of databases have lent themselves to scaling easier than traditional RDBMS systems. In the CAP Theorem, these systems tend to be more in the AP range, sacrificing some consistency for scale and performance. That seems crazy to RDBMS people, but if you really think about your application, consistency that is delayed on the order of seconds or less isn’t that bad in most applications. Even seconds might not matter in many reporting scenarios.

    The thing is, databases aren’t perfect, and that goes for NoSQL systems. I ran across a post where MongoDB sometimes loses consistency within a single node. Now, this isn’t within a single document, so changes there are transactionally handled well (as we RDBMS people think of them), but for a query, you could have rows drop out and reappear according to some criteria, sometimes within a few minutes or seconds of each query.

    That is disturbing. In fact, I’d be terrified of this, mostly because I’d spend a lot of time trying to explain why and getting yelled out. Do you know how many times I’ve had business people run a report and then someone else run the report a minute later and try to compare things? If I had whole documents dropping out of one or the other, that would be maddening.

    Now, there are ways to fix this in a MongoDB database, and I’m not disparaging MongoDB over this. It’s a fine system and there are some good applications built on MongoDB that work well. There are some fine applications running on Neo4j, and on Couchbase, and other NoSQL systems. Some of these database work better than an RDBMS in managing particular workloads and problem domains. Some applications and workloads would do well with a NoSQL or a traditional relational database.

    The important thing to understand is how your system works. Your organization should understand what allows a database to scale and what doesn’t, what things limit HA/DR capabilities, which items might impact consistency across nodes or systems. In the SQL Server world, if you use Availability Groups, you should certainly understand what consistency means for the different databases in the same AG.

    NoSQL databases aren’t better or worse than relational databases by themselves. Finding a good fit for a particular database platform and your application requires some knowledge and planning. Ensuring a successful application is often less about the technology, and more about the people that architect and code the solution. Worry more about the latter than the former and you should be successful at building software.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.7MB) podcast or subscribe to the feed at iTunes and Mevio .