Voice of the DBA

Tag: Database Weekly

  • Moar Azure

    There are a number of choices for working in the cloud, aka, renting resources, but there are three big choices: Azure, AWS, and GCP. These three dominate the cloud services, though arguably AWS and Azure are way ahead. All three have lots of services, and all three can likely meet your cloud or hybrid needs, even if you work in a regulated industry or government.

    This week there was a piece on the work Microsoft is doing to increase their capacity. Between the 0365 Office services, Teams for collaboration, and their regular PaaS/IaaS workloads, they’ve grown quite a bit this year with the COVID-19 pandemic. As they try to increase their resources, they’ve had people working round-the-clock to add new servers. I’m sure they are still using containers, but likely they are struggling to fill these with new hardware.

    They’ve also been adding bandwidth, which is a bit need between the data centers. I had no idea they have their own undersea cables, but apparently they do. I suspect Amazon and Google do as well.  There is a note that they’re also coding, trying to improve efficiency. After all, while you can throw hardware at a problem, many of us know that bad code can overwhelm those efforts. We see it in our jobs every day. I would hope that many of our organizations that might be depending on software more realize that better code is an advantage for us and our customers.

    I know Microsoft runs a lot of their own stuff on Azure, including development work, and they’ve had to reschedule their internal workloads as well as move other work outside of regions. I think that’s fine in many cases, but with different laws in different countries, I hope they’re extremely careful about what moves. Data sovereignty can matter.

    If you read through the article, it seems that Microsoft has learned a lot about adapting their work methods, both with people and systems, to meet the increased workloads. This is one area that I hope we see some detailed technical blogs, which might help others learn how to better manage their own resource challenges, or even learn to code better. While some of this might be proprietary, with the move to having many things open source, why not share ways to improve application resource usage.

    Steve Jones

  • The Social Impact of Data

    Let the data drive your decisions.

    This has been something of a mantra for many technical people, and even many business people, across the last twenty or so years. The allure of business intelligence is harnessing lots of data to make decisions that are rooted in some rational analysis of what has happened. Many companies use “data driven decisions” as a way of achieving success.

    However, what about when the data is flawed? When deliberate or inadvertent actions give us data that isn’t quite as pure as we expect. In the last week we have seen many protests and complaints about the ways that many people feel they have been unfairly treated by police. That brutality, particularly for African Americans in the US, has been a problem for decades. Some of that is due to human biases, beliefs, and more. However technology plays a part, and will for some time to come.

    I have watched as algorithms have been used in sentencing, and I’ve questioned their use, as have others. There is this idea that computers will be more fair, looking at inputs and making a decision that isn’t encumbered by human biases. The problem is that humans that program the systems might have some bias. Maybe more disconcerting is that the data used to train systems is likely biased as well.

    There is also the concern that as technology advances, it can be put to new uses, perhaps in ways that the inventor regrets. Oppenheimer regretted the violent use of his work, and I wonder if technology inventors will feel the same way. Surveillance technology is controversial. It can help retails companies prevent theft, but it can also be used in ways that might enhance and reinforce bias in police work. This can be controversial, and no matter how you may feel about the technology, there are moral questions of privacy and prejudgment that are worth debating.

    The last couple weeks have saddened, upset, and angered me at different times. I am also confused and concerned, unsure of how to discuss and debate these topics. I find my position moving slightly with different stories and different information, as I should. I learn more and my views grow and change, shaped by what touches me. I do worry about how we use data in the future, and how it can be abused. There are ways in which more data can help improve our world, but the potential for abuse is high, and I do believe we need governance, transparency, and an independent appeal process for those wrongly impacted.

    Steve Jones

     

  • The Build Book of News

    Every year Microsoft has held the Build conference for developers. I’ve been lucky enough to go a few times, and I was hoping to get the chance to attend again. With the pandemic, the entire event went virtual, and was held across 48 continuous hours. I was slightly disappointed as a few sessions I wanted to see were early in the morning or late at night, but I am sure most of these will be available on demand later, so I can watch them if I wish.

    One of the things Microsoft has done is produce a Build Book of News, a 55 page PDF book of all their announcements and changes. You can download it from Paul Andrew’s blog, or you can look at the announcements online at Microsoft News.

    I randomly watched Build on and off this past week, having some of the sessions in the background while doing other things. There certainly were some interesting sections, though not a lot of data platform focus, but I found a few items that you might like.

    We’re all remote now, or the majority of us are, and I can see that remote tools are
    important. I know there has been some remote code work in Visual Studio in the past, but I didn’t think many people would use it. Now in watching some of the remote code demos, I think these tools might be interesting, especially when a few people are trying to debug a crisis situation. Being able to see the code on my own machine, and seeing what others might change or think is a better (or worse) solution could be very helpful.

    The mobile GitHub tools for review are getting better, and that’s good. Since it’s likely more of us might be in different locations, on different schedules, the ability to approve a PR could prove really helpful. Codespaces are especially interesting. Maybe not so much for databases, but maybe. I wonder if we could easily get data provisioning in here.

    Azure CosmosDB is still something I want to play with, though I’ve been hesitant to set up a db because of cost. Serverless CosmosDB might get me to try something. I have no real excuse since they’re an emulator, so maybe not. I wonder how many of you might try CosmosDB out. There are also announcements about PostgreSQL and MySQL on as PaaS services getting feature parity with SQL Server in many ways.

    Lots of Synapse changes, which might be of interest to those of you with larger warehouses and looking to take advantage of the cloud offering. Quite a few AI changes as well, with Bots, speech, ML, and more. I especially liked the keynote on AI, which I think was interesting. It provides some vision of what could be possible.

    Tons of developer sessions, and I’d suggest that you browse or search the sessions to see what interests you. There aren’t a ton on demand as I write this, but likely more will appear this coming week. I hope the Cloud Skills Challenge will also remain up there, as I’d like to spend some time experimenting with services. If you haven’t worked with the Azure services, I might give that a try and get a feel for what’s possible.

    Build is always interesting, and I’m not sure if I love this virtual format, but I’m glad the event was still available and once again I’m impressed with all the work Microsoft is doing to help developers build solutions.

    Steve Jones

  • Automatic Refaction of PII

    Data privacy and protection has become a hot topic. Across the last few years, as I’ve worked with customers that deal with the GDPR or other laws, they’ve become more concerned and careful with their use of data. I find less and less resistance from developers about using sensitive production data in development environments, but still too much.

    Using data that’s stored in databases or other text files is one thing. What about data in less structured forms? I’ve dealt with a few customers over the years that recorded customer interactions for various purposes. Call center or financial organizations commonly do this, and sometimes deal with sensitive information. I know I’ve given my date of birth, credit card number, bank account number, and other data to representatives at various times.

    Those calls are often recorded, and the IT staffs have often had to ensure extra security is applied to these files. Not everyone has access to listen for various reasons, but certainly when there is sensitive information inside the audio (or video), this data needs the same protection we’d apply to data in other forms. Providing protection, or redacting the information, isn’t an easy task.

    I saw recently that the Amazon Transcribe service will now redact some PII information automatically. This service can be configured to automatically remove the information in text, which is fantastic. This is a great way to start to use technology in a safe way to ensure that we have less data  leakage when we re-use data. Certainly people might look at this data to better train reps, but it’s also likely someone wold look through transcripts to determine why customers are calling in and use that information to better design applications. In either case, there isn’t any need to expose PII data to them.

    This doesn’t protect against the data inside the audio, but perhaps companies can delete and remove those recordings sooner with transcripts available and more quickly reduce their potential attack surfaces. We’ll always have some liability, but reducing that and not unnecessarily creating issues is part of what we want to do when protecting data.