Voice of the DBA

Author: way0utwest

  • The RDBMS is Often the Right Tool

    I’ve spent most of my career, actually most of my life, working with relational database systems. I tend to agree with this piece on NoSQL systems being the wrong tool for many applications. I do think that’s the case, though to be fair, I haven’t done anywhere near the amount of work on key-value stores, document databases, graph databases, etc. as I have on relational systems. I admit I’m biased, but I think the author is as well. He works at MemSQL and used to work on SQL Server.

    The piece notes that relational databases didn’t deal well with certain problem domains and various NoSQL classes databases evolved to handle the challenges. The good things about this is that relational databases were forced to grow and adapt, incorporating new features and functions. I do truly wonder if that is the reason we’ve seen rapid changes in SQL Server, both in capabilities and the pace of new releases.

    Many of the non-relational systems are built to incorporate transactions, but usually within a very limited range. For example, many document databases only allow ACID compliant transactions within a document, not across documents. The same might be true in graph structures. Many of us working with relational systems know there are often a decent number of queries that need transactional consistency across entities, and sometimes across different databases. Our applications demand this, and while it might be overkill at times, it’s a capability that is definitely needed.

    There is also a discussion of schema-less features. I do understand the attraction for application developers. Adding or changing the way data is stored is much easier when there isn’t a set schema for writes. This truly does make development easier. What it doesn’t do, however, is remove the technical debt of having a schema. Instead, this debt is pushed to the reader, and often, means the application must maintain the ability to apply some schema for reads to make sense of the data. Does that mean everything needs a hard, strongly defined schema? No, but I think it means we need some schema.

    Ultimately I think that there are problem domains best solved by different platforms. In most cases, I’d argue a relational data store is the better platform, but I am certainly willing to entertain the discussion of where a graph, document, or other data platform may work better. Just be sure you argue for the problem being solved. Most of us aren’t operating at the Google/Facebook/Spotify/Netflix scale, and we have different problems to solve. We need something that works well for our problem, not the one another organization is facing.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.9MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Should the Data Lake be Immutable?

    There’s a concept in computer science of immutability. At a high level, this means once something is set, it isn’t changed. Various computer science languages do this with variables, where values don’t change, though variables can be destroyed and recreated.

    In the PASS keynote, Dr. Ramakrishnan pointed out that we have silos of data, often in disparate systems where we keep our information. We want to query this together, so we transfer this to a data warehouse or data lake (the future view) and that items in the data lake are immutable. They aren’t allowed to chang in the way that we update values in our relational databases. We should just read the most recent version of any data, and if there is an update, just add a new set of data.

    That’s an interesting concept, but not sure I agree. I think that while we might often want to use a simpler process, there are cases where we do need capabilities to edit. Imagine I had a large set of data, say GBs in a file, would I want to download this and change a few values before uploading it again? Do we want a large ETL load process to repeat? Could we repeat the process and reload a file again? I don’t think so, but it’s hard to decide. After all, the lake isn’t the source of data; that is some other system.

    Maybe that’s the simplest solution, and one that reduces complexity, downtime, or anything else that might be involved with locking and changing a file. After all, we wouldn’t want queries that could potentially read the data in between us deleting a value and adding back a new one.

    If you’re a data warehouse or analysis person, what do you think? Does it make sense to keep the data lake as immutable and reload data that might not be clean? Let us know today.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Reader’s Choice Awards are Taking Nominations

    If you’re using products, you might want to nominate a vendor or product for a database tool that helps you. The Database Trends and Applications Reader’s Choice Awards is taking nominations for the next few weeks.

    At Redgate Software, we try to build useful products that help you solve problems and get work done. If you think we help you, I’d appreciate you nominating us in one or more categories.

    Enter your nomination today.

  • Git, GitHub, GitLab, BitBucket, What’s the difference?

    I was watching Brent’s webcast session on GitHub recently. I’ve struggled to explain this to users in the past, and it’s always good to see what others do when they do similar sessions.

    You can watch Brent’s session, and I’ve got some content on git. Need to do more for Redgate, so look for some content to appear on the Hub or our YouTube Evangelist playlist.

    In any case, what’s the deal with Git, et al?

    Git is a version control system (VCS). If you’ve ever used TFS for code, or Visual SourceSafe or Subversion, Perforce, CVS, etc., they are all methods of tracking versions of files. Usually for code, but could be for any files. Binaries don’t work as well, but any text, and plenty of people use a VCS for book projects.

    While git (or Git) is a VCS, there are various companies that provide git hosting services. GitHub is the most well known, now owned by Microsoft. This means that they provide essentially a remote file service for git projects. Since git is a distributed VCS, this means that you can have multiple copies of your git repository (folder) and sync back and forth. Like DropBox, but with programming code versions.

    BitBucket, GitLab, Beanstalk, Codebase, etc. are all hosting services. You could duplicate the git portion of hosting with a VM in Azure, AWS, etc. However, these services provide a few other things. They include some features like security, clients, and pull requests. A pull request is a method of notifying people to review your code before it merges, and it’s a fantastic team feature. It’s in git, but it’s cleaner with a service.

    I tend to use Azure DevOps for my git hosting, but I do have repos at GitHub as well. I’ve tried BitBucket, and really they all feel the same. I get used to a GUI or CLI push/pull of code and then it doesn’t matter.

    If you’ve never tried using git for code, give it a try today. I’ll get more content and videos up soon, so watch for those.