Voice of the DBA

Tag: software development

  • Learning from Microsoft

    I know that many data professionals out there are skeptical of the cloud. Certainly there are problems with reliability, security, and more that prevent many of you from migrating to some online database. However that doesn’t mean that everything being done in platforms like the Azure SQL Database is a bad idea.

    One of the interesting things that has occurred at Microsoft is a completely new engineering effort for SQL Server. It seems there is one SQL Server development team that works on the Azure SQL Database. This is (I guess) the main branch in version control, and what we see with SQL Server 2016 is a subset of the code being built for the on-premises, boxed product.

    Certainly there are some features that exist in the box that aren’t in Azure, but relatively few of them. Those features, and many new ones, have to be included in the version control system, but they can’t be activated. Microsoft must be making extensive use of feature flags, allowing their developers to build and test features in the Azure cloud, but not make them available to customers, or potentially even other parts of Microsoft.

    If you really think about the effort to build a platform like Azure SQL Database and allow it to be a development environment as well as a production environment, that’s an amazing engineering effort. From the outside, I’m amazed at how far the SQL Server team has come in the last few years. I’ve seen a number of features released in the cloud first, and then appearing in the SQL Server 2016 CTPs.

    I think there are some amazing lessons here for software development. Certainly I think many of us could learn from how feature flags and multiple versions of software can co-exist in systems. That would certainly help many of us deploy new software in a way that minimizes interruptions to customers and clients. Or manage those multiple clients with different requirements from the same codebase.

    I hope that Microsoft will continue to evolve, as well as share their knowledge with the world. There are other companies that seem to be accomplishing amazing things at scale, Netflix, Amazon, Google, and more. Some of them share their techniques, but I truly hope that the methods they all use for building software become spread throughout the industry and help all of us build better, more secure applications.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.3MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • The Dangers of Algorithms

    I ran across an interesting piece in Wired recently that talks about algorithms and the power they hold in the modern world. I don’t think it’s a great piece, and it starts to mix code, data, algorithms and other aspects of modern software in a way that seems disjointed to someone that works on software. I think there is come confusion about how data and software differ.

    I don’t deny that there are potential negative impacts of the ways in which we process data. Certainly we have bugs in software that cause issues, but we also end up with implementations of specifications that may not quite match up with the expectations of our clients. Even in simple applications, if I’m not constantly interacting with a client in a detailed way, I find that the software can operate in a subtlety different manner than others expect. Of course, there are certainly people that deliberately program systems to intentionally help, or hurt, others, but I think we have just as many issues from small communication mistakes. That’s one reason why DevOps, CI/CD, and frequent releases are valuable. We get feedback quicker on the implementaiton of the design.

    Does it make sense to disclose algorithms? Make source code available for examination? I definitely think there are intellectual property issues to consider, but when there are legal challenges or arbitration, I would expect that the actual code of a system can be examined. We definitely need to ensure that code examined in these cases is not part of the public record. However there needs to be open examination in such a way that the groups involved, and perhaps even other interested parties, have confidence that the algorithm performs as expected.

    This becomes easier in databases, since our code is often easily accessible, but more complex in applications. Many of us realize that subtle changes in application code can cause issues for data, which is part of the reason we like declared referential integrity in our databases. That way we’re not affected if an application makes a mistake in enforcing our data integrity rules.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.7MB) podcast or subscribe to the feed at iTunes and LibSyn. 

  • Forcing Strong Passwords

    It seems that every week we have some sort of data breach that results in emails and passwords for users being released. That alone is a good reason why you should not have passwords shared across sites or systems. I can’t stress this enough, and if you do this, please go get a password manager today and stop. Really, I mean it.

    However is it just the users that create poor passwords? In many cases, sure. However as the people that develop software, we can help. In fact, we should. Security should be on the mind of everyone that writes software.

    I ran across a piece on The Hacker News that says far too many sites don’t require users to choose strong passwords. I think this is changing, but far, far too many sites don’t check a user’s password for strength, or reuse. Some even force users to choose less than ideal passwords. I’ve run into plenty of sites that don’t accept the characters in my strong passwords for some reason. Often the length or addition of any special characters if prohibited.

    I’ve said it before, but I’ll note it here. Most of us aren’t competent at writing authentication routines and should just use one that’s already written and been vetted. More importantly, we should be sure the code, application and database, is modular enough and can be replaced if we find that our application needs stronger security. Because it will. We are constantly racing Moore’s law to implement better security.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 1.9MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Reasonable Timeframes

    Many of us hear about problems with various systems on a regular basis. We report them in Database Weekly far too often, and I’m sure a few of you have been informed (or found) bugs, nagging issues, even vulnerabilities in your systems.

    What’s a reasonable time to fix a system after an audit?

    It’s a simple question, but it’s not very simple to answer. After all, most of us don’t have a lot of slack development time built in to fix issues. Unless the issue is a broken application that doesn’t work, the items disclosed in an audit need to scheduled in among other work. After all, most of the time the audit finds something that no one is aware of, or no one has wanted to fix. This is work that no one really planned on completing.

    I ran across an interesting piece about the Employment Department for the state of Oregon hasn’t fixed a number of issues after an audit last year. While some strides have been made, there are still outstanding issues, the sum total of which it is estimated will take a decade to complete. That’s a long time, but in large systems, especially ones where the entire application cannot be rewritten because of resources, it’s not unusual. I’ve worked in a few places where we had large scale systems that we knew had issues, but we couldn’t easily re-design and implement fixes in any reasonable length of time. Often this was because of downstream dependencies, but certainly culture and management hadn’t made change a priority.

    I sympathize with those people dependent on mainframe systems. The power and reach of those systems, the poor documentation, not to mention the complex training required to change clients’ habits is huge. I would hope that the government groups using these large scale systems would work together to jointly proceed on development, with politicians also involved to help standardize the requirements across state lines (or countries’ borders) and simplify the software needed.

    However, no one ever makes software simpler, especially when it’s being designed.

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.