Many years ago I was in the Denver airport, waiting to board a plane. At the time I was consulting with a small startup company and they called me in a panic, having had an issue with their database deployment and needing to restore the previous version. However, they were getting the “The media set has 2 media families but only 1 are provided. All members must be provided.” error. Fortunately, I knew what was wrong and verbally walked them through the process. When I landed, I logged in and verified things were working and that they’d followed my instructions.
A few years later I was shopping for furniture with my wife one Saturday night and I got a call from a fellow Operations person at JD Edwards. They had some security issue with a SQL Server, and since I was the SME (subject matter expert), they called me for help. Sitting in a comfy lounge chair (which I never purchased), I helped them solve the issue. Later, I verified things were working.
In these cases, I was using remote hands from trusted individuals at an organization. However, these days, many people call cloud vendors (or other hosting companies) for the same type of remote hands support, but without necessarily having the accountability from that staff. Sure you can check if they solved an issue, but are you sure they didn’t read some data or change anything else?
It’s a concern, but it’s one that I think can be solved with help from vendors. Microsoft recently announced there’s an audit capability for Azure SQL Servers, allowing customer to determine what actions Microsoft support engineers might take on your system. Since we may not necessarily see what is happening when someone else is working on our PaaS or IaaS systems, it’s good to have something tracking their moves. I don’t know this gathers all queries run, but it certainly should.
I’ve seen some presentations on how customers have to authorize engineers in VMs with Customer Lockbox, and the technical details are interesting. These are time limited and audited access controls to ensure that customers are aware when a Microsoft employee touches their systems. I don’t know if this is comprehensive enough, but it is a good start.
Over time, more and more of us have to delegate some work to others, either inside or outside of our organization. We need strong controls and good tools to ensure that we can determine what happened, especially in the event of some incident taking place. Building these controls is important, and really, they should be easily enabled in all our systems. While I don’t know we have a lot of rogue employees accessing things, we do have some, and we have more that make mistakes. Auditing their actions should be something we can easily do.
Steve Jones
Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.
Voice of the DBA 