Voice of the DBA

Tag: NoSQL

  • The NoSQL Rise and Fall

    I’m not sure this blog that seems to talk about the problems of NoSQL databases in general makes a lot of sense. If you read the comments, you certainly see lots of complaints, but I also think the post isn’t well written. It lists the problems of RDBMSes as possibly deleting all tables while changing a key or being unable to add a column easily. While I don’t know about all RDBMSes, I don’t know any that could lose all data with anything less than DROP DATABASE.

    At the same time, the NoSQL complaints and problems seem to be generally presented, which isn’t good. The various NoSQL flavors of databases vary widely and the way you look at a columnar or graph database is much different than a document database. Really, the piece ought to be separated to look at a certain class of NoSQL database compared with RDBMSes.

    That being said, I do see less hype around NoSQL in general. The class of problems handled by various types of databases vary, and I am sure that lots of people using NoSQL databases are quite pleased with them. At the same time, I’m not sure a lot of the RDBMSes did get replaced or continue to get replaced. RDBMSes have a lot of value and we have a lot of experience, both as developers and administrators.

    There might be a shortage of NoSQL (pick a type) administrators for operating the databases. Even in some of the famous companies that use them, I find developers often explaining a RCA or the troubleshooting of a particular process. While that might be a choice that an organization makes, a lot of developers that I know don’t want to be supporting systems very often, especially at night and on weekends. A couple of bad incidents usually has developers wondering why there isn’t an expert on the Operations staff.

    I do think that many of benefits of different types of NoSQL databases are a bit over-hyped. Not that a graph database can’t handle your e-commerce system, but it likely takes a lot of code, and likely a lot of mistakes in getting it to work well. I also feel that some of the downsides of RDBMSes are minimized. They are a pain for application developers to work with, and the SQL language isn’t very sensible.

    That being said, we have a lot of experience with these databases, and that should not be discounted. Productivity is a big advantage in trying to build new software in a flexible manner. I am all for experimenting with different platforms to understand where they work well, but also learn to use your existing RDBMS platform well. Write better code and you might be surprised just how scalable your system proves itself to be.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Where Does NoSQL work well?

    It seems not too long ago that NoSQL was the hottest thing on the market. MongoDB was leading the charge, with tremendous growth as a company and many developers looking to adopt their platform. I remember years ago going to visit a number of customers in New York City, where Mongo had just grown their offices. Every company was considering migrating their RDBMS to MongoDB, trying to calculate the cost and return.

    Now, I certainly hear about companies adopting database that are in the NoSQL family, but not always (or even often) to replace a RDBMS. Instead, it’s for a new application or alongside an existing RDBMS. While there are applications that are well suited for a non-RDBMS data store, there are plenty that perform worse on those platforms. This might be because the platform doesn’t handle their workload, or because the developers aren’t writing good code, but I know I’m not worried that RDBMS usage is going away, or even down.

    I ran across an article earlier this year that wonders if NoSQL is a good choice for your application. The article covers a bit of the types of these stores and how they work. It’s a basic look, but it makes sense and it addresses the benefits of NoSQL, like scaling. It also notes there are issues, which is something to think about. Not that I see a lot of developers thinking about the issues. Often they think the can code around them.

    The piece notes that for some problems, these databases work great. However, for a side project or a simple site, a relational store is likely better. NoSQL classes of databases are suited to scale well, but with quite a few tradeoffs.

    Are there places you’d use NoSQL? I’m curious. If your organization has adopted these stores, or rejected them, it would be good to share some information.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Mongodb Encryption

    The last decade has seen a number of new database platforms get implemented in a variety of organizations. Most of the newer popular platforms have been NoSQL (Not Only SQL) products, and one of the first that became quite popular was MongoDB. I first ran into a few customers using this platform around 2012/2013. At the time some customers inquired whether we had any products to help with development. We did, somewhat, as we invested in 3T.

    MongoDB has enjoyed a lot of popularity and set some standards for how to store and retrieve data. CosmosDB (and other products) has API compatability with MongoDB, precisely because of its popularity. While I’m not sure MongoDB is the replacement for your RDBMS, I’m sure it has places and problem domains where it shines. I’ve also watched the product improve and grow to meet the requirements and desires of more customers. One recent addition was field level encryption.

    MongoDB has had storage level encryption, similar to TDE in SQL Server. It also has had transport encryption (think SSL network protection). This new feature is more like Always Encrypted (AE) in SQL Server. Clients can perform the encryption and decryption, assuming they have the keys. This isn’t quite like AE as it appears to need code changes, but it does protect the data from system administrators, which is a concern for some applications and industries.

    How well does this work? There certainly is a challenge with managing keys, distributing them to clients, revoking them, and more. All of these are the same challenges more encryption technologies have to deal with, including AE. This is a good addition to all data platforms, which ought to enable encryption, but having, or even using, the encryption, isn’t a panacea for the problem of protecting data. We need lots of other things to be designed, implemented, and operated securely.

    I’m not a big fan of MongoDB, but it’s a fine platform, and it likely works well for some organizations. I’m glad that it continues to enhance its security capabilities, and I hope that anyone implementing it enables all that they can. Too many installations in the past haven’t had good security, and that’s a problem. If you manage data, secure your system, and be sure you continue to monitor the platform for security issues and keep learning more about how to best protect your data.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher or iTunes.

  • The Age of Multiple Databases

    I can still remember learning to store data in files, reading one record at a time, testing for end of lines and end of files. It was slow and cumbersome, but for a teenager, it was amazing. We built applications that could help run Dungeon and Dragons games, and the ability to store information about our games was amazing. I even adapted some of these routines to store data about various chemistry experiments in school, making my report writing (and data fudging) must easier.

    In time I stumbled upon the basic database structure of dBase, thinking this was vastly superior. I could control the structure of each table, but have the power of quick access and indexes. Of course, I learned to make sure I opened all the indexes with the data file, otherwise I might “lose” some data over time.

    Those first database structures felt great, and even superior to SQL Server when it was just one person. Initial queries were much quicker under dBase type applications (Clipper, FoxPro, etc) than under SQL Server, though that quickly changed as the number of users grew. For the last two decades, I’ve been entranced and thrilled with the power of relational systems. They solve many problems well, and I find them fairly easy to develop against.

    Not everyone feels the same, and there have been many different types of data stores built in the last decade, each designed to solve different types of problems. I ran across a post about the different specialty databases available in AWS from Werner Vogels, Amazon’s CTO. The piece is interesting as he notes that one type of database can’t meet all the needs of modern applications, at least, it can’t meet the needs of very large scale systems like Amazon’s eCommerce site. Instead, there are situations and problems better solved by specialty systems.

    Amazon built a key-value, extremely fast database to back a system that mostly needed those types of simple lookups. Over the years, in response to their needs or those of their customers, they’ve added other types of databases in their AWS cloud. Mr. Vogels lists six databases in AWS, including relational stores. He talks a bit about where each can be used. He also cites some customers that have had success in different applications with different data stores.

    Microsoft has done similar things in Azure and their data platform. Most of us know SQL Server as the relational engine, but we have an in-memory component and with 2017, we also get graph capabilities, albeit rudimentary ones. In Azure, they have CosmosDB, which include a number of different types of data stores (key-value, document, graph, etc.). I know Google has multiple databases available as well.

    As we build larger systems that reach out to more people, I wonder sometimes if we ought to be architecting to use multiple data stores from the beginning, choosing the type that works best for the problem and not choosing to just use one database for all parts of an application. I already have some customers that supplement their SQL Server with Redis, Elastisearch, AWS or Azure queues, cloud storage buckets/containers, and more.

    I’m a practical person, who likes to pick the solution that works well for the problem I need to solve. I have no issue with using multiple technologies, and as my career as advanced, I think it’s easier than ever to do so. If you haven’t considered other data stores, perhaps you want to learn a bit about them and see if they could help improve your system. Keeping an open mind and considering alternatives is good, and I recommend it. What I don’t think you should do is choose another technology because you like it or are interested in learning more about it. Pick the solution that works best.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.8MB) podcast or subscribe to the feed at iTunes and Libsyn.