Voice of the DBA

Category: Editorial

  • Should the Data Lake be Immutable?

    There’s a concept in computer science of immutability. At a high level, this means once something is set, it isn’t changed. Various computer science languages do this with variables, where values don’t change, though variables can be destroyed and recreated.

    In the PASS keynote, Dr. Ramakrishnan pointed out that we have silos of data, often in disparate systems where we keep our information. We want to query this together, so we transfer this to a data warehouse or data lake (the future view) and that items in the data lake are immutable. They aren’t allowed to chang in the way that we update values in our relational databases. We should just read the most recent version of any data, and if there is an update, just add a new set of data.

    That’s an interesting concept, but not sure I agree. I think that while we might often want to use a simpler process, there are cases where we do need capabilities to edit. Imagine I had a large set of data, say GBs in a file, would I want to download this and change a few values before uploading it again? Do we want a large ETL load process to repeat? Could we repeat the process and reload a file again? I don’t think so, but it’s hard to decide. After all, the lake isn’t the source of data; that is some other system.

    Maybe that’s the simplest solution, and one that reduces complexity, downtime, or anything else that might be involved with locking and changing a file. After all, we wouldn’t want queries that could potentially read the data in between us deleting a value and adding back a new one.

    If you’re a data warehouse or analysis person, what do you think? Does it make sense to keep the data lake as immutable and reload data that might not be clean? Let us know today.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Employee Target

    We hear regularly about consumer backlash from data breaches. There are usually legal repercussions, and often hard dollars (euros, yen, etc.) being spent on things like identity protection or credit monitoring. In addition, in the last couple of years, there are plenty of people that try to limit, or cease, doing business with organizations that lose their data. That isn’t always possible, especially when the government is the one getting hacked, but more and more people are taking an active stance against poor internet security by organizations.

    It’s not anything that I think will happen soon, but I bet it will happen. Airbus had a breach that lost employee data. I wonder when we’ll start to see employees initiating lawsuits or other actions against their employers. My suspicion is that this will happen with former employees first, but with the GDPR, current employees in the EU might feel emboldened, and with good reason. Employers have a lot of data about us, and that ought to be well protected.

    There have been hacks to lower a stock price, or affect a company. Why not hacks to attack employees? I wouldn’t have thought of this until I talked with a forensic analyst about other possible second order attacks. Could there be individuals that might seek to attack IT staff through personal information and blackmail or otherwise extort them to copy data? What about partners and spouses of employees at big companies? I don’t worry about Redgate as a target, but what if you work for Google/Apple/Microsoft/Facebook? We already had a potential sleeper employee at Twitter.

    Unfortunately, I see no depths to which criminals might sink. Across the last decade there have been stories of actions that I never would have contemplated. While I hope my fears are unfounded, I worry that sensitive data about employees might be on the radar for some nefarious individuals.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

     

  • A Good Benefit?

    I’m wondering what you think. Is the idea of trading some paid time off (PTO) for student loan debt relief a good idea? That’s what Unum, an insurance company, is offering. They give their employees 28 days off, which is quite generous. I don’t know many people that get than many days off. If employees wish, they can trade up to 5 days for the equivalent salary, which is used to give them money for their student loans, or their children’s.

    It’s a creative offer, and one that doesn’t force people to forego vacation, as they’d still have more than four weeks of time to take off. In the US, most people don’t take their entire vacation allowance when it’s over two weeks. This gives them a chance to pay down some debt with minimal impact on their work life balance. At least, minimal for me. I struggle to take my 25 days, so this is a trade I’d make.

    The downside, to me, is that there may be plenty of employees that don’t have student loan debt. They don’t get a choice here, which feels unfair. I’d much rather the company offer employees the chance to sell back up to five days and use that payment for anything. After all, plenty of people have debt that isn’t related to education. Why not give them a chance to reduce other obligations?

    I am not quite sure of what I think of companies becoming more involved in employee’s personal lives. We are in a more freelance, gig economy where people change jobs regularly and often may work remotely. For me, I tend to favor the approach from Basecamp where the company treats people fairly and create an environment where employees are respected and want to work for a long time. While I know this is hard for many companies, and certainly hard to change, I do think more organizations could improve their culture and treatment of employees.

    I’m very lucky in that I work for a company that’s amazing. Redgate Software is the kind of company I’d want to create if I built another organization, and I have a fantastic set of co-workers, managers, and benefits. We are growing and hire here and there, and if you’re interested, I can’t recommend this organization enough.

    Can you say the same thing about your employer?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Treat All Sensitive Data as Important

    We know that not all the data in our company is important. We have databases that contain orders or inventory or schedules, often much of which isn’t easily or directly related to an individual. At least, it’s not if you have a normalized database. If you use SQL Server to emulate Excel spreadsheets, it’s possible that most of the rows of information in your system contain sensitive data.

    In some systems, there is definitely some data that is sensitive and needs more care than other data. We know this, and with legislation like the GDPR, we must protect this data. We also need to ensure we know where this data is, and having a good data catalog is important. This is something that few of us have, though I expect this to be a more regular part of our job as data professionals. SQL Server is building data classification into the product, which I am happy to see.

    When data is sensitive, we need to treat it carefully, even if we don’t like the content of the data. Recently there was a data breach from B&Q, a home improvement retailer in the UK, where 70,000 names were lost. These weren’t customers, but rather people that had been caught stealing from the stores. Perhaps this was an honest mistake, on a data store with poor security. Perhaps no one thought this data needed security because these were criminals, or suspected criminals. Even if these were individuals that might be prosecuted by the company, their data still deserves the same protection as any other person’s data.

    I don’t know what the fallout will be from this breach, and certainly most people would have little sympathy for criminals, but who knows just how accurate the data might be. I certainly think this is a situation where there is a high likelihood of legal action against the company if the proper GDPR notifications were not followed. Wouldn’t that insult to injury? People caught or suspected of theft suing you because you leaked their personal information. I could certainly see management getting extra upset and terminating someone that forgot to secure these systems.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Libsyn.