Voice of the DBA

Tag: software development

  • Code Supply Chain Security

    There have been a number of attacks in the last few years on source code. In fact, I saw a new one this week for an e-commerce WordPress plugin. This time hackers got access to the distribution server for the company, Fishpig, and altered the plug-ins that their customers download.

    A few years ago this was big news, with the SolarWinds exploit. There was also an attack on PyPy, a popular Python package that many people include in their code.  There have been no shortages of problems in npm packages as well. I’m sure this has happened in other software packages, which is scary. In the days of DevOps where we publish code from a repository, an exploit against your developers might go unnoticed. Then again, maybe not.

    Would any of us notice new code in a file share or a folder on our system? We might just compile a large project without realizing it. At least with DevOps, we have the opportunity to include security scans and code analysis checks, some of which could look for known patterns of exploits. I know some companies use these, and often compromised or vulnerable packages are stopped by the automated pipelines.

    In the US, various security agencies have released a set of recommendations, as has the Open Source Security Foundation. Both of these are designed to help developers secure their supply chain against attacks. This is likely going to be a continuous problem for software vendors in the future as it’s much easier to attack one vendor whose software many people use than each individual company. I shudder to think about what happens if someone manages to get a ransomware package into a vendor’s codebase.

    Ultimately, there will still be problems. Many new projects begin with poor practices precisely because they’re experiments and the authors don’t know if others will find the software valuable.  While we can have good templates and security controls, I’m not hopeful. To me, the best solution for stopping code is to have patterns detectable by security checks in the pipeline. Checks that can be expanded and enhanced as new issues are determined.

    Of course, that means the makers of security software need to ensure their supply chain is protected as well.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Assembling Your Own Toolbox

    Last month Brent Ozar posted that September was community tools month. He had a number of posts that highlighted tools and a number of other people (Francesco, Erik, VladDBA ) did as well. I didn’t do any blogging because, well, it was a busy month for me. I was gone from home for 21 days, so a bit harried in life.

    As a part of this effort, I saw Ken Fisher write about being a Script Magpie, which is an interesting analogy. A magpie is a bird that collects various shiny objects. Ken collects scripts in the same way. Things that catch his eye are added to his repo of scripts that he keeps handy.

    I used to do something similar. I didn’t have a repo back then, though I should have. However, all VCSes in the past lacked some of the ubiquity of Git. Instead I had a folder that I kept synched on various flash drives that I carried from job to job. I’ve also usually had a share inside companies where a team of developers or DBAs could share SQL scripts.

    I’d hope that most technology professionals would share scripts, especially when they can be tokenized or added as SQL Prompt snippets. Reusing work is a big part of working with computers and why many people have embraced tools and utilities that they didn’t write themselves. I certainly think having a toolbox is important, especially tools that integrate well with an existing environment.

    I don’t think it’s worth rewriting many tools inside a company. I work for a software vendor, and I’ve seen the effort put into building tools. Both free and paid tools can save you a lot of time and effort inside your job, not the least of which is maintaining the tools over time as versions and editions change.

    What are the contents of your toolbox? Which things are those that you’d bring to a new position if you moved organizations or groups? Let us know today.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • The Challenge of Early Tech Decisions

    When we design software or databases, we have to make decisions right now. If you work in as many environments as I have, this means I’m often picking a tool, platform, version, technology, or something else without a complete set of information. In some cases, I’m actually writing code (SQL or C#/Java/etc.) without completely knowing the business problem I’m solving. I do know what someone described, but as many of us find out, those descriptions often prove to be lacking.

    Over time, we refine the code with feedback from users, which is why DevOps has become so popular. We write the best code we can and then quickly alter code if we haven’t hit the mark. We constantly adjust by making small changes and improvements.

    However, changing the code is one thing. Changing more fundamental technology choices is harder. There is an interesting post on the disproportionate impact of your early tech decisions from one of the developers at Stripe. One of the examples given is for the database platform, which was Mongo. The piece notes that once the database platform is chosen, most groups never switch platforms. I think that’s been true in many places I’ve worked. The same thing for languages and cloud providers as well, as once a choice is made, rarely does a team, much less an organization, switch.

    Why is this? Is the cost of switching that high? I think it is. I’ve seen a few organizations want to migrate from a licensed relational platform to one that is free and OSS. However, the time it takes to rewrite code, the time to build new expertise, learn the tips and tricks for a new database (or other platform technology) is significant. A team can make no shortage of mistakes during this process. Is that worth the cost of licensing?  Perhaps. Perhaps not. It’s a hard thing to decide. With the high cost of labor, I think it’s hard to make a good argument to rewrite.

    In many cases, I’d argue that core changes to your technology stack ought to be considered, but understanding that no tech will solve all your problems and you will make lots of mistakes in the process of changing. Does this mean you need to make a great decision when you start working on a project? In my mind, no. Make the decision to use what your team is most familiar with and then live with it.

    And be open to using targeted technology for specific needs. Lots of transient data in a busy workload? Add Redis to your database stack. If you need full-text search, consider ElasticSearch. Maybe add a graph database (not SQL Server) if you need queries in this problem domain. Use what works, and learn to use it well.

    That last sentence might be the most important. Any technology can likely support most workloads, but you need to write good code and learn to use the platform well.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Developer Optimism

    Developers, in general, are very optimistic about the code they write. This is likely one cause of their estimates of the time required being low, as well as the various bugs that slip through because of corner cases that appear for the problem being solved. Often developers think they’ve considered the various ways this code ought to work and covered all the possibilities. Usually we find they’ve not thought about the problem from other perspectives and need to adjust their code.

    They often also feel that their code is superior to others, and that they can examine a problem in a new way. One of the reasons that I think many developers want to rewrite systems in some new technology or new way, embracing the Not-Invented-Here way of looking at other people’s code. They want to write their own solution.

    Of course, many developers these days don’t eschew all code from others. Usually, it’s just other people in their organization, as a developer will embrace some random open source project they’ve discovered, sure that using this library/language/whatever will make their code better. While I do think there are examples of some new tech that is better than what’s around, I don’t know that any of them are a panacea. While F# is praised in many circles as better than C#, it’s not widely used, at least according to surveys of the top programming languages in use (IEEE, SO). It must not be better enough to get more projects to use it.

    What is widely used is SQL. In fact, if we look t0 other database languages, nothing else is close. While I’m sure plenty of those C# and Java programmers use LINQ or some ORM to produce the SQL code, I don’t know many competent or highly regarded developers that can’t write some SQL code or don’t regularly write SQL.

    There are some interesting ruminations on the optimism of developers in this piece. It made me smile, and I like the practical ending. While you can debate and discuss things, ultimately we need to ship code. And we are almost always better off using a technology we know rather than searching for a perfect new one to solve our problems.

    When working with databases, relational or even most of the popular NoSQL ones, this means knowing SQL. And when trying to solve our database problems, in many cases, this means learning to better write SQL and build relational entities, not abandoning our platform for some new shiny one that we think will make everything run smoother.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.