Voice of the DBA

Category: Editorial

  • Privacy Problems in Apps

    Software is a great equalizer, in that almost anyone can build an application and make it available for others. The app stores from Apple, Google, and others are great examples of where individuals or small teams have made some incredible applications that people all over the world use on a regular basis.

    This year, a pandemic changed the world. One of the main things that many scientists and governments have desired to help control the spread of disease is contact tracing. One would think this is a great situation where technology can help.

    It can, but many apps built for contact tracing aren’t respecting privacy. Researchers examined a number of apps in use around the world and found many aren’t using the privacy friendly notification systems that Google and Apple have built. There are a variety of permissions and practices in use, which is a little disconcerting. How can someone really understand the implications of choosing one over the other?

    As with any situation, it’s likely that some developers have good reasons to request additional permissions and collect more data. Lots of governments and universities built apps because they didn’t feel the Google/Apple frameworks collect enough data. It’s also likely some malicious actors built apps to collect data, and certainly a few have been built with advertising or some other money making scheme integrated with the app.

    If you choose to use some sort of app, you might spend a bit of time understand how it works, what data it collects, and if there are any policies or ability to control this. While we want to control and eradicate this pandemic, many of us don’t want lasting control of some data about us to be in the hands of others.

    Steve Jones

  • Rough Consensus

    The IETF has a document, RFC 7282, also called ” On Consensus and Humming in the IETF”, in which they describe how the body makes decisions. This is their philosophy:

    We reject: kings, presidents and voting. We believe in: rough consensus and running code.

    That is a process that I can get behind. Overall, I don’t like authoritative leaders. I don’t like to be one, though I do recognize as a manager there are times I need to make some decision. Overall, however, I like consensus.

    Someone was discussing how to choose approaches when developing software and brought my attention to that quote. With us being remote now, and unable to hum, this person talked about how to solve problems without humming.

    This individual talked about identifying solutions in a team, but not taking a vote, or rather, no immediate vote. Instead, they devised a rating system allowing everyone to look at all solutions and rate them from 5 (best solution ever) to 1 (this would be a terrible mistake). They then produce a histogram of the results for each solution.

    Once they’ve done this, they can then have people raise issues and debate them. I assume time limited, which is important, but often when we see different levels of support, we may rethink our rating or the merits/detractions of an approach. At some point, they re-rate the solutions and try to develop consensus.

    We won’t always agree completely with an approach, but I do think that we can often come to some decision we can support, especially with some rational arguments and reasoning. I’d like to think that I rarely had to overrule an entire group. Building a team means finding ways to discuss, debate, and decide together.

    Steve Jones

  • Building for Scale

    This year’s pandemic has moved a lot of business to the digital world. We see that with increased ecommerce from many companies, including lots of local business, like restaurants, that have embraced online ordering, pickup, and delivery. In many cases, the companies providing these services were able to scale and meet the additional load, which arguably wasn’t that high for any individual organization.

    However, some services aren’t able to handle the loads, especially government services. Recently the Florida voting registration website crashed under a larger than expected load. We’ve seen unemployment sites crash in the US, the contact tracking website in the UK, and plenty more.

    I can understand the challenges of government sites, many of which were built on older technology and rarely upgraded with budget restrictions. Investing in new technology is not only expensive, but it can be a huge challenge to ensure systems work through a transition, especially when many users are counting on services to be available.

    I do think that the future of much software development needs to be with a hybrid cloud architecture. There are great government secure options from AWS, Azure, and other providers. However, even if you build software that is intended to be in your data center (or a rented one), you ought to be thinking about how to scale it beyond the machines on which it exists, especially if you provide public facing access.

    Whether in government or private enterprises, we see the demand for systems grow and shrink rapidly. You can have incredible success, or face shrinking demand quickly. This isn’t like an investment in a large physical plant. The digital world allows us to easily scale up and down, using the cloud concepts and architectures developed over the last decade. Learn about the patterns, and ensure your systems can scale as needed.

    Steve Jones

  • Scaling Serverless Azure

    I enjoy reading articles and blogs about how people solve technical issues. The
    Trainline blog has had some good posts over the years, and the Netflix tech blog always gets me to think, even thought I don’t use many of their technology choices. There are others of course, but for the Microsoft tech, I run into some neat posts at the Microsoft Devblogs.

    Recently I saw one from Davide Mauri on streaming at scale using Azure SQL Database. I really like the platform, but it still feels like something that needs maturity. Seeing this post with metrics, testing, and evaluation of how the system works with data gives me more confidence that this might work for a scenario I hadn’t considered.

    I love that the post includes lots of references to code and other explanations, making me think I could set this up and test it. Certainly I’d like to know more about the actual costs of the test, which is something missing from too many discussions of cloud implementations, especially PaaS ones.

    An exception might be Troy Hunt’s series on Azure and how he’s built HaveIBeenPwned. This is a scaling post, though you can find others if you search around. He’s disclosed the costs of things in some of his posts, which help me to understand how important choices are to both performance and your budget.

    If you are looking to build systems in the cloud, I urge you to search out and read as many experiences as you can. Most of them boil down to write for the cloud, write good code, and test, but do so in a way that takes advantage of the cloud. Don’t expect to just have things work for you, no matter how good a developer you are. You need to understand your tools, and the cloud is just another tool.

    Steve Jones