Voice of the DBA

Tag: NoSQL

  • Learning NoSQL

    It’s a good idea to learn more about different technologies. I’ve been amazed throughout my career, and even now as someone working for a software vendor, how many different technologies I run into. I’m also amazed at how little bits of knowledge comes in handy, either to help me understand a problem or to help guide others with a recommendation of how to proceed.

    Many of us have experience with relational databases and tabular data. We are comfortable with it, even if we might struggle with a packing problem or using a tally table. However, do you understand how a NoSQL database might be different? Can you even name a few types of NoSQL databases?

    I find it valuable to know a bit and experiment a bit, as your devs may ask you if it’s suitable, or even how to work with one of these databases. They might even start using one and expect you can take responsibility for managing it, backing it up, securing it, and more.

    I wonder how many of you know MongoDB? What type of database it is or how it stores data?  What about Cassandra? Neo-4J? CosmosDB? I haven’t done much work with any of these, but I’ve learned a bit about them. I even spent six months digging into graph databases and learning how they work, how to model data, and how to query them. Part of this was SQL Server’s inclusion of graph structures, but part of this was a customer looking to add graph capabilities to their application. I learned a lot, and even delivered a few presentations on how they work.

    I’m not an expert, but I can have a rational, reasonable, informed debate about the good and bad things with graph databases. Enough to help provide an opinion on whether they might suit a problem. They solve some complex things well and I think they might be a good platform to add to some applications. I wouldn’t get rid of the RDBMS, but supplement it.

    This year I’m going to dig more into MongoDB and Cassandra. We have more customers asking about these platforms and we’re working on Flyway support for both of them.

    If you have any thoughts on NoSQL, or experiences, or even want to write a bit about them, let me know. I’m always looking to learn more, and I always need good authors.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

  • A Case for Document Storage

    One of the challenges for both database developers and administrators is doing more, often with less. Many companies continue to grow their database estate, both in width with more platforms, and in depth with more instances of the platforms they have.  Some companies will look to shrink their staff, especially when adopting a cloud platform, while others may add more databases, but not increase staffing to match the additional load.

    In either case, what many have found over the years is that the cost of labor is high. Both for developers that write code against databases, and administrators that manage those platforms. While licensing can seem to be a large number, compared to the cost of labor, it isn’t usually a significant number.

    Often it seems administrators would prefer more of the same database platform. Developers often seem to ask for new types of database platforms, often some type of NoSQL data store. I ran across an article that makes a case for adding in document storage data stores to your environment, instead of just choosing am RDBMS. Labor is one of the big reasons for doing this. The other one is that for a given workload, the hardware cost is lower.

    The article opens talking about the object/relational mapping problems. There is some truth to the time and effort to map an object in an application to a table (or set of tables) in an RDBMS. There is some knowledge required to do this, but I also think it’s an important skill for many developers. The same type of object mapping to a serialized JSON document is shown as being easier, and it is.

    However, if you add or change your object, the application code to handle the document from the data store gets complex. Over time, you will have lots of “new” fields that don’t exist in older documents. How do you handle those? It’s not hard, but labor is required to write this code. And this code has to be maintained over time.

    The other argument is that less hardware is needed, made by noting all the data you may need can be co-located with your object. This is what we would call denormalization in an RDBMS and leads to data duplication? Whether that is a problem or not depends on the amount of duplication. Certainly the structure of an application that often works to send or retrieve singleton rows is easier in a document database.

    However, non trivial queries, which the author postulates are hard to write for developers, are likely hard to run for a document database. The load of querying across lots of rows, or updating them, is much higher in a document database. Depending on how often you update data, this can be an issue, and require more hardware.

    Which is better? The classic “it depends” applies here. Database modeling is important in both cases. As I’ve worked with people that move to NoSQL databases, I find they struggle to model in that world as much as many of us struggle to model in the RDBMS world. I also find that a NoSQL database often is going to require some sort of data warehouse or other structure that is built for reporting across documents.

    I’m not against the various types of NoSQL databases, but I also don’t think they are a panacea of any sort that magically makes building and operating an application easier.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The General Database Platform

    It’s been a decade-plus of the Not-Only-SQL (NoSQL) movement where a large variety of specialized database platforms have been developed and sold. It seems that there are so many different platforms for data stores that you can find one for whatever specialized type of data you are working with. However, is that what people are doing to store data in their applications?

    I saw this piece on the return to the general-purpose database, postulating that a lot of the NoSQL database platforms have added additional capabilities that make them less specialized and more generalized. I’ve seen some of this, just as many relational platforms have added features that compete with one of the NoSQL classes of databases. The NoSQL datastores might be adding SQL-like features because some of these platforms are too specialized, and the vendors have decided they need to cover a slightly wider set of use cases.

    It also could be that some of the features of the relational database, which are certainly a general-purpose data store, are demanded by customers. I personally think that many problems are solved well by the RDBMSes out there,  and if you need something specific, then add on a NoSQL document, graph, time-series, or other type of database for some part of your application.

    A lot of organizations have moved to using multiple data stores in applications. I see it more and more from customers I deal with, often because there is a specialized need, like Redis for caching, that just outperforms the main database for a certain function. As long as it’s easy to integrate, why not use a second database platform in your application?

    I don’t know that you want to keep adding specialized systems, however. As the article notes: “the overhead today of having to learn and interact with multiple databases has become more burden than boon.” That can certainly be the case when there are a lot of platforms and your staff changes on a regular basis. We’ve seen this become a burden when there are too many versions of a database platform. Supporting these is hard and learning about all the features is a challenge for developers. Building expertise in many areas is sometimes very difficult for an organization.

    My guess is that the future will continue to be applications that use one data store, but the trend is away from this. Especially as more organizations start to use the cloud, I suspect a lot of companies will use a specialized data store for some part of their data. They’ll also be more likely to use a second platform for a data warehouse or analysis platform. With cloud vendors building easier data transfer capabilities, like Synapse Link, I do expect more customers to take advantage of these and use a second, specialized platform where it makes sense.

    I like the idea of specialized platforms and systems. I do think that there are some problems that are not well suited to being solved by relational structures. However, I also think that for a lot of critical transactional systems, it’s hard to get away from a relational store.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Is CosmosDB the One?

    The question asked of Keanu Reeves in the first Matrix by many of the characters is whether he is the one. THe one that can save the world, and make a difference in their war. It’s a fun movie, and one I’ve enjoyed many times. Keanu doesn’t really believe he’s that special until the end. That’s in contrast to the products that most vendors build. Most of the times I think developers, marketers, salespeople, or all three think that whatever they’ve recently released is the best thing ever.

    CosmosDB is mentioned in a piece as the potential one database to rule them all. The article talks about some of the basis and value that come from the platform, which Microsoft continues to push as the database that can store any kind of data and meet the needs of any application. Whether you believe that or not, I find many articles and blogs from developers that are experimenting and building systems on top of CosmosDB.

    I do think the consistency models for CosmosDB are interesting, and having a variety of these models across a distributed RDBMS table would be great. I’d love to be able to insert data into a table, have it not available as a live value until committed on other nodes, and avoid locking my table. I realize I’m asking for something that seems like a bit of magic, but I bet Microsoft could add this to the SQL Server relational engine.

    CosmosDB is very interesting to me, and it’s an item on my list that I want to learn more about. I think there are certainly domains of problems where CosmosDB would fit very well. If any of you are working with CosmosDB, I’d be interested in knowing why and how it fits for your particular environment.

    There was a quote in the article that I thought was interesting. A Microsoft Technical Fellow said, “No data is born relational. In the real world, nobody thinks in terms of schemas — they think graphs or maybe JSON document if you’re an IOT device…”

    That’s interesting. If I produce a JSON document, am I not thinking relational? Do I not expect that most strings, values or arrays will exist in most of the documents? Am I not thinking schema, even ragged schema, from the beginning? I’d think most data relational. It’s just not easy to get it into that format when you deal with many instances of data.

    Of course, dealing with many instances of data is hard for most humans. That’s why we built relational databases.

    Steve Jones