Voice of the DBA

Category: Editorial

  • The Cloudy Future in 2021

    Ignite took place recently, and the keynote from Satya Nadella was interesting, and if you have time, interesting to watch. If not, there’s an excerpt in written form. In any case, I like his passion and excitement about the future. I’m slightly sad that he’s not on stage somewhere, and instead in a studio of some sort, but that’s the world today.

    There were five pillars of the keynote which related to the future of the cloud. One of these was the idea that we are at “peak centralized computing” now, but we need to get more decentralized. The idea here, I think, is that we will have more of the computing services taking place outside of data centers and server rooms, and in more devices. We already see some of this, with mobile devices, cars, sensors, and more handling some computing loads.

    The Edge is something Microsoft continues to push, and I think the adoption of this paradigm will continue, with some pre-processing or filtering occurring in devices or intermediate servers, and then less, or limited, data being loaded into our database platforms. I also expect that we’ll see more data loaded into a database somewhere, and less ETL into another platform and more just in time querying from a client or other data store.

    I was also glad that trust was a pillar, though I think this was an attempt to corral a number of disparate ideas into one topic. However, the idea of data protection, and individual rights such as privacy, are important, and I do think that Microsoft recognizes this. Even if they are just trying to build a product to profit, I appreciate the work they are doing in this area, adding new capabilities to allow companies to better control, limit access, and protect data in flexible ways. Now, if we can just get the companies to respect the individual’s data they hold, I think the world will be better.

    Lots of what Microsoft presents in their keynotes and conferences is aspirational, with limited experience or understanding of whether customers really will adopt these ideas. Every year I see some visions that don’t ever get widely used across the next year and aren’t mentioned the next year. However, some of their ideas are good, and some will evolve in different ways to help improve our world. This year, I loved seeing the remote ways HoloLens 2 is being used.

    I never know what will catch on, but I do like seeing the visions that others have for our world.

    Steve Jones

  • The Ease of Containers

    I first heard about containers in 2012. I went to a conference on software delivery and talked with a gentleman that was using them in his Java application, mostly because the Java app had version issues and would leak memory. He could run hundreds of instances of the app on each server and handle a load, allowing each to be built, run, and then get destroyed in minutes.

    Since then, I’ve come to appreciate containers as a way to isolate workloads of specific applications. They became very popular a few years ago, and many developers and companies were looking at them. Since then, the hype has cooled, but I find that the used of containers is still growing, and certainly my use is growing as well.

    In the last year, I’ve experimented with a few different pieces of software in containers. Rather than try to install a number of dependencies on my system, I’ve downloaded a container, mapped a volume, and been up and running much quicker than I would have been otherwise. My experiments with Jekyll were one of the recent examples.

    I also got a demo from someone at Redgate recently that included a container file, allowing me to use VS Code, but develop and run in a container, avoiding dependencies on my machine. I was up and running with this software in minutes. I was truly impressed with the ease of getting started, and the simplicity for me to actually run and debug code.

    I still don’t see a lot of database container work in production, or even in development. We get requests from customers, but often it’s a wish, and they aren’t even sure how they’d get started. While I think database development with containers is fantastic, you do need to have a good dataset available that you can use inside the container to keep resource usage low and make this a viable environment.

    I still expect the future of database development to be in containers, especially as we start to have more and more applications connecting to multiple data sources. That’s going to take some time, but I still think learning about, and experimenting with, containers is a great skill for you to have. It’s also an impressive topic on which to have a few stories ready for your next interview.

    Steve Jones

  • Distributed SQL Databases

    One of the challenges that SQL Server, and many RDBMSes, face is scaling out. While SQL Server can grow to handle a large workload on one piece of hardware, eventually there reaches a point when a single machine cannot handle the workload.

    Microsoft is trying to handle this with some new tricks. We have the Hyperscale editions of Azure SQL Database and Big Data Clusters, on premise. I don’t know that either of these will end up providing us with scale out for our OLTP databases, but they are options.

    There are other options. Google has Spanner, which is a distributed SQL database and CockroachDB has attracted some attention. I caught an article that talks about some of the reasons you might look at CockroachDB, which is a cloud system, but one that gives you scale, and perhaps more importantly, lack of lock-in with a cloud vendor.

    I don’t know all the ins and outs of a distributed, scalable SQL database, but I do know that scaling one table, with lots of concurrent access, can be tricky. There are improvements in bandwidth and technology that might allow things to scale well, but I can’t help but think that conflict resolution will be a challenge if many people need to write to the same rows.

    I also know that often we aren’t writing to the same rows, but to the same table. Distributing that is easier, but when we start to look at lots of tables with referential integrity, I wonder how scalable this is. After all, the data needs to get committed and then written to multiple places for this to work well.

    The one advantage in the cloud is throwing hardware at problems like this, often in a more cost effective way than an organization can do in their own data center. Whether this works well remains to be seen, but certainly there are customers and investors that see this as a part of the future.

    Steve Jones

  • Disclosing the How

    Many of us have written a computer program of some sort. Maybe just a script to complete a task, maybe complex software the clients interact with. For all of us, I would guess that we’ve been asked to do something, or we’ve asked ourselves to build a program, and we use that request as a specification for our code.

    When we do this, how many of us get this right the first time? I’m sure some do, and the simpler or narrower the task, the more likely we get it right. However, we also sometimes make mistakes and realize that our specification wasn’t correct or that we interpreted the specification incorrectly. Even when I write code for myself, sometimes I realize I haven’t described when I need clearly enough, even in my own head.

    That happens when I control the entire process. When someone else is my client, there are plenty of ways in which we can make mistakes, especially in decoding the specification. As we use more and more software to make decisions for us and automate our world, I wonder how often we’ve assumed the software works in a way different than it actually does. With machine learning and artificial intelligence programs, I think there is an even greater chance of a misunderstanding.

    Recently I saw that in New York City there is a proposal to disclose how software is used in hiring decisions. Humans are barred from discriminating by law, but what about software? What if there are bias that can come out in the software based on how someone enters data?

    I think there are certainly potential issues in having software evaluate individuals, though I do think software will do a better job than many people, once we’ve worked out the bugs and we know how we want it to work, and then we can compare that to how the algorithm produces results.

    These are tough issues, and difficult projects. I also think it’s important that we analyze the data that is used in testing software, to be sure that it doesn’t contain its own biases. Ultimately I think having public data sets that are evaluated by many individuals is important. I could see a reference set of data and the expected results, or some range of results being required to ensure software meets the goals we want as a society. This, along with the disclosure of how algorithms work will make our use of software better over time.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.