Voice of the DBA

Tag: software development

  • The Downside of Tool Autonomy

    I am a big believer in letting teams of people work in a way that allows them some autonomy to solve a problem. I think this is especially important in the software world, where developers need to experiment and learn from the new ideas that they investigate. They need to be able to try new techniques, technologies, and platforms. Yes, that includes NoSQL data stores.

    At the same time, this can lead to developers becoming captivated by new tools and techniques, all of which are good for advancing skills and code maturity, but can also lead to a very disparate environment where there are no best of breed solutions, or no standards for writing features. As the number of developers grow, this also raises the bar for each individual to know and understand more languages or technologies.

    While I do think it’s easy to pick up logic and syntax, when writing production code, we want to ensure we’re optimizing the way we work. The way in which we might write some code when we start with a new API, SDK, language, etc. is often not the way we’d write that code after a year or experience. At least, I hope it’s not the same way. There are nuances, tips, and tricks in how to better structure code that we learn as our experience grows.

    When I was at the Redgate Software office last year, I noticed that there was a short article posted on our internal blog. This noted that there is a set list of tools and technologies to use when building our products. This isn’t intended to be a long term immutable list, and the post noted that other choices are possible, but we want to avoid an unending growing list of tools when we have identified there is a good tool in a space. There is an approval process to propose and new tool and explain why it really is needed.

    At Redgate, we’re pretty flexible and open, so I’m sure this isn’t a tough process. However it does prevent every developer from implementing their special little code into a project. We’ve had people use various different data stores, different types of web technologies, and it becomes a maintenance issue over time. Even the VB.NET component we used for years at SQLServerCentral caused headaches as there wasn’t any experience in the company in this language.

    I think it’s worth having some light overview that ensures an organization isn’t adding technology just for the sake of making one or two developers happy. There is a finite amount of time to spend learning and growing, and while it’s important that technology workers do this, we also want to developer more experience and expertise in a few areas. Limiting tools and technologies helps preserve a balance that allows for growth, but ensures the expansion isn’t unchecked.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Measuring Effectiveness

    I’ve never been the guy that produced code the fastest, or the most code in a day. I haven’t been able to show that I work the most hours, or that I even keep great hours. I’m often late or erratic in the times I come and go from work. I don’t show that I am closing tickets in consistent times as I often will be slow to close new issues. I haven’t ever been the employee of the month.

    What I have been is effective. I get things done. I leave the organization and the job better than when I found it. I work to deliver value for the salary I’m paid. That isn’t something you can easily measure with many traditional metrics that people use, but there are ways to decide that I’m a good employee. It’s a little ephemeral and sometimes uncomfortable to have an employee that doesn’t seem to fit the same model as most other employees, but that doesn’t mean you wouldn’t want me working on your staff.

    This article on measuring DevOps tries to explain how you can determine if your DevOps process is working, which is a similar ephemeral way of working. When we implement DevOps, we don’t have large project plans, we don’t look forward to completing a system. We undertake a set of work, knowing that we don’t have an end date. We just keep doing work and getting things done that are needed. In this case, how can we measure if a project was on time, on budget, and finished?

    I’d argue, as would the article, that those aren’t good measurements. Those are the ways we would traditionally look at work, but in software, those aren’t often the things we want to look at. If we deliver software that meets some requirements, but users struggle to use it or complain, is it done? Most management would call it done, just like the airlines mark the plane as leaving on time when it pushes back from the gate. Standing on the runway for 45 minutes doesn’t count against that target, even though I’d call that leaving late.

    The article talks about picking things that bother a manager or team members about the software or their process. Some of these are easy, like the time to deliver a new server or recover from a failure. Others that I’ve found to be useful are the time to assemble a release or deploy software to an environment. While we can sometimes play with what these numbers measure by moving to smaller set of changes or altering our process, we’ll quickly find out if we have a lot of overhead in our work by adding these measurements.

    What about the software itself? Can I actually compare the time to deliver report a vs. report b? One might be much more complex than the other. That’s true, but we ought to average out delivery estimates over time, and we can certainly separate out query writing of the complex logic against the time required for formatting. In fact, it might be good to start to measure different parts of software delivery to find out if certain people are better at some parts, or if requirements from clients are causing unnecessary delays. I know I’ve certainly had some formatting complaints require more time to get right than the entire rest of the software.

    It can be hard to develop metrics that have outcomes and are truly actionable, but it’s a better way to determine if your team is improving. Relying on simple and traditional metrics is lazy, and allows for lots of argument and debate over whether a team is doing well or not. Tackling those specific items that are irritating to management or customers might take some work, but you’ll end up with a list of things that can be targeted for improvement and show progress that actually means something.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.8MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Porting SQLServerCentral

    Like a few of you, I’ve been working with WordPress as a blogging platform. Over the years I’ve tried a few different pieces of blogging software, but I really like WordPress. I’m not alone as there are estimates that 20-30% of all websites run on this platform, including a few you might not expect. I thought UpperCup and Krispy Kreme UK are sites that don’t really look like they’re powered by WordPress. Those make my own blog and T-SQL Tuesday look pretty bland. Maybe I’ll do a little design work at some point on those. That’s after SQLServerCentral moves over to the platform.

    SQLServerCentral started as a custom ASP site many years ago, then upgraded to ASP.NET at some point. This was a joint effort from the founders to build in new functionality and features as we needed them, purchasing components (like the forums) where we found a suitable product. This first evolution of the site lasted for many years until Redgate Software acquired the property. We then underwent a second platform shift to NHibernate, which has been underpinning the site for a decade. We now move forward with our third evolution.

    We have a project underway that is porting our site to WordPress, for a variety of reasons. Like many of you, I struggle to get resources assigned from my employer for the projects that I’m passionate about if they don’t rise in importance above other things being worked on. There are only so many resources available, and they must be shared by the company. While Redgate values SQLServerCentral, we have a site that works well, and has worked well for many years. Thus, it’s not the same priority as some of the other projects in the company. Since we have some requirements around better mobile support thanks to Google, we had to move in some direction.

    We have struggled with skillsets over the years as most of our web developers aren’t well versed in NHibernate as we’ve moved many of our other web projects to WordPress or more basic technologies like React. Building all the various features from scratch would be a big project, not to mention a constant maintenance headache, so after reviewing some responses to our RFP, we decided to go with WordPress, under Project Nami. This is an open source project that replaces MySQL with SQL Server. While I run MySQL on T-SQL Tuesday, one of our key requirements was that we use SQL Server as a database, and Project Nami allows us to do this. Since there are numerous people with WordPress skills, and lots of plugins that can be easily added (or removed), our view is that WordPress will allow us to grow and change the site over time with fewer resource constraints.

    The last few months have been a long, drawn out project as we needed a number of custom plugins written, or existing ones adapter for some of the functions on the site. At its heart, SQLServerCentral is a rather unique publishing platform, and we needed to preserve much of this functionality. As with most projects, we’ve run over time and budget a bit, but we’re now getting close. I don’t have a date yet, but I anticipate we’ll add more user testing in January and then make a switch sometime later in the month.

    I hope that you’ll find the new platform to be very similar to what we have now. Our goal was to change relatively little in terms of functionality and minimize the look and feel changes. There are some, but I don’t think they are too disruptive. However, we will be looking for feedback and make decisions on what things we’d like to change or adapt for the future. Keep an eye out for more announcements and fingers crossed that everything goes smoothly during the deployment.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Better Static Code Analysis and Security Scans

    I was listening to a talk from Stefan Simenon on their CI/CD transformation within ABN AMRO, a large financial company. One of the interesting things he noted was that they consider open source to be less secure, possibly with more vulnerabilities than in house written software. Their build pipeline will fail if a developer starts using new OSS components.

    I find that interesting, as the DORA 2018 State of DevOps report sees more use of OSS software in companies that are adopting DevOps. In general, I think that having many people able to view the source and find errors makes companies feel that open source is more secure. I think that’s likely more true, though it’s a bit of a philosophical argument. We can look at some data, but it’s hard to prove that one or the other is empirically more secure.

    The thing I agree with is that using new components without some review is not a good idea. Whether this is written in-house, copied from an Open Source project, or purchased from a vendor, we need to perform some testing and analysis of the code or component.

    This is also true in database code. When we get a query from a developer, it’s often easy to determine what is happening, but when the size of code grows, or there is a large stored procedure, we often don’t perform a detailed analysis. What’s worse, we don’t have good static code analysis tools for database languages. As much as I like what Redgate Software has done with SQL Prompt, I know this is rudimentary and is built to avoid code smells. There isn’t any detailed look at whether the code is secure, or if there might be unintended effects.

    There aren’t really any good tools I’ve seen, though I’m not even sure what I’d want here. How can a tool tell me that querying 4 tables and updating 3 more is OK, but an insert to some other table in a separate database is bad. That insert to the other database might be what a malicious actor wants to copy data elsewhere. The best thing to me would be some analysis of what objects are being touched and how, which could help alert developers to potential issues.

    Building static code analysis tools for database languages is hard, but it’s something that our industry needs to do. This is even more true when we start to have more programmability features, like the ability to execute other languages inside of our database engines. In those cases, not only do we need to ensure the code for another language passes test, but that we understand what types of interactions our database code has with those modules.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.5MB) podcast or subscribe to the feed at iTunes and Libsyn.