Voice of the DBA

Tag: software development

  • Do Less

    When I read this blog post, I was transported back to simpler times, when I learned to develop applications on 8088 processors. Back then, to make the system go faster, we learned to do less. It’s certainly true that the less work a system performs, the faster it can complete the limited set of things it must do. Working within C code, most programmers I knew back then tried to optimize algorithms by reducing the amount of work that our CPUs needed to perform.

    In T-SQL, we don’t have the low level of control we might have in other languages. In fact, our language has a relatively small number of instructions that we we use to build queries and applications, though that number continues to grow and expand over time. I am grateful for that, but also realize this brings with it additional chances for developers to write code that does more work than necessary.

    For much of our work, optimization consists of finding ways to group our operations together. Building a set-based solution takes advantage of the way in which SQL Server most efficiently processes instructions. Far too often there are developers that write inefficient code because they apply techniques they’ve learned from programming applications in some other language, or even from some other relational database.

    For a given system, there is a known amount of work that can be done in a period of time. There are also restrictions on the various resources involved (disk, CPU, network, etc). Reducing the load for each of these resources, especially in multi-user systems, can dramatically increase the performance of your system. SQL Server is built to manage concurrency in a variety of ways, but poorly written code is a great way to overwhelm the system.

    Do less. Get more performance.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and LibSyn. 

  • Building Better Software

    One of the ways in which many software development teams have tried to improve is by examining what has worked in other organizations and projects and experimenting with ideas. Across the decades that we have been using computers, there have been quite a few models, the most well known of which is the CMMI Model from Carnegie Mellon University.

    The CMMI Model contains lots of good ideas and practices, much like Six Sigma, ISO and other frameworks. However we (as a group) often don’t follow many of the ideas. It seems that so many developers (under pressure from managers) work to just complete tasks, without regards to the well known, valuable techniques of measurement, improvement, and more.

    However there isn’t much magic in using any of these frameworks. They all cover similar concepts, which are based on measuring what you do, taking that feedback to alter your process and get better, as well as then working within a structured process that is repeatable. Do more of what works, and do less of what doesn’t. Adjust your way of working regularly.

    The problem for most of us is getting a group of people to actually follow a process over and over. It seems that we continually want, at least as a large group, to cowboy code our way through development cycles, without learning from the past. Of course, I meet plenty of people that use an ALM (or DLM) process to build software, and they all espouse the benefits, but they do note that it takes a long time to adopt a new process throughout an organization. A year of investment isn’t considered uncommon.

    Perhaps that’s the problem? Can management actually make the investment time to build out better processes? The media hype says they want to, with all the DevOps hype. Plenty of companies, including Redgate Software, are selling tools that companies use to help themselves adopt a process. However I wonder how successful organizations are. Is there enough continuity in the technical staff and management to actually make a change? It took Microsoft years to change. Are you moving in that direction? Is there any chance you will?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Learning from Microsoft

    I know that many data professionals out there are skeptical of the cloud. Certainly there are problems with reliability, security, and more that prevent many of you from migrating to some online database. However that doesn’t mean that everything being done in platforms like the Azure SQL Database is a bad idea.

    One of the interesting things that has occurred at Microsoft is a completely new engineering effort for SQL Server. It seems there is one SQL Server development team that works on the Azure SQL Database. This is (I guess) the main branch in version control, and what we see with SQL Server 2016 is a subset of the code being built for the on-premises, boxed product.

    Certainly there are some features that exist in the box that aren’t in Azure, but relatively few of them. Those features, and many new ones, have to be included in the version control system, but they can’t be activated. Microsoft must be making extensive use of feature flags, allowing their developers to build and test features in the Azure cloud, but not make them available to customers, or potentially even other parts of Microsoft.

    If you really think about the effort to build a platform like Azure SQL Database and allow it to be a development environment as well as a production environment, that’s an amazing engineering effort. From the outside, I’m amazed at how far the SQL Server team has come in the last few years. I’ve seen a number of features released in the cloud first, and then appearing in the SQL Server 2016 CTPs.

    I think there are some amazing lessons here for software development. Certainly I think many of us could learn from how feature flags and multiple versions of software can co-exist in systems. That would certainly help many of us deploy new software in a way that minimizes interruptions to customers and clients. Or manage those multiple clients with different requirements from the same codebase.

    I hope that Microsoft will continue to evolve, as well as share their knowledge with the world. There are other companies that seem to be accomplishing amazing things at scale, Netflix, Amazon, Google, and more. Some of them share their techniques, but I truly hope that the methods they all use for building software become spread throughout the industry and help all of us build better, more secure applications.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.3MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • The Dangers of Algorithms

    I ran across an interesting piece in Wired recently that talks about algorithms and the power they hold in the modern world. I don’t think it’s a great piece, and it starts to mix code, data, algorithms and other aspects of modern software in a way that seems disjointed to someone that works on software. I think there is come confusion about how data and software differ.

    I don’t deny that there are potential negative impacts of the ways in which we process data. Certainly we have bugs in software that cause issues, but we also end up with implementations of specifications that may not quite match up with the expectations of our clients. Even in simple applications, if I’m not constantly interacting with a client in a detailed way, I find that the software can operate in a subtlety different manner than others expect. Of course, there are certainly people that deliberately program systems to intentionally help, or hurt, others, but I think we have just as many issues from small communication mistakes. That’s one reason why DevOps, CI/CD, and frequent releases are valuable. We get feedback quicker on the implementaiton of the design.

    Does it make sense to disclose algorithms? Make source code available for examination? I definitely think there are intellectual property issues to consider, but when there are legal challenges or arbitration, I would expect that the actual code of a system can be examined. We definitely need to ensure that code examined in these cases is not part of the public record. However there needs to be open examination in such a way that the groups involved, and perhaps even other interested parties, have confidence that the algorithm performs as expected.

    This becomes easier in databases, since our code is often easily accessible, but more complex in applications. Many of us realize that subtle changes in application code can cause issues for data, which is part of the reason we like declared referential integrity in our databases. That way we’re not affected if an application makes a mistake in enforcing our data integrity rules.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.7MB) podcast or subscribe to the feed at iTunes and LibSyn.