Voice of the DBA

Tag: software development

  • The Scientific Method

    What’s the greatest invention in the history of humankind? We probably each have some ideas, and we could certainly debate the answer, but Dave Farley says it’s science, and the scientific method. Many of us probably learned about the scientific method. Look at a problem, develop a hypothesis, test it, alter our hypothesis based on results. Repeat. Eventually come up with an explanation that’s proved by the evidence.

    This sounds like a good way to actually build a piece of software. However, is this what most of us do? Or do we continue to pile additional caveats and restrictions to our hypothesis and try to force the system to work a certain way.

    Is the scientific method something we need more of in software? I was watching the talk from Dave Farley, and he mentioned this. He talked about the need to examine a problem, guess how to solve it, try implementing your guess, and then examining the result. If it’s not correct, then you change something and try again. It’s an agile process that tends to work well when there are fast feedback cycles.

    However the one thing in the talk that was the most telling to me is that something isn’t finished until a customer can see it and give feedback. Until then, it’s really in limbo. I would guess that if more development efforts focused on making this happen quicker, we’d end up with better software, written more quickly.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Software Estimates

    We really stink at estimating the amount of time that we’ll spend on building software. However I’m not sure that we’re much worse off than many professions. If you ask a doctor how long it takes to get better, do they know? They do in some small, often seen, well documented issues, but in many cases, they don’t really know. Can lawyers determine how long it takes to solve a case beforehand?

    It seems the more accurate estimates typically come from places where the problem is fairly simple and very well known. However that’s probably true in software as well. If I asked you how long to write a new blog site, I bet many developers could come up with a close estimate.

    The idea of telling a computer to do x seems more bounded and contained than other professions, but it’s really not. This is often because developers don’t get a complete statement of all the requirements from clients. However this also isn’t really any different than many professions. How many of you know exactly how you want a house built, or legal matter settled, or the specifics of medical treatments and effects?

    I read an interesting piece on the idea that developers should just make small estimates, small changes, and move forward. They should be willing to undo, or redo their efforts as needed. That’s one of the benefits of Version Control Systems (VCS) and the great developers take advantage of the possibilities.

    While there are certainly problems with moving too fast, there should be plenty of reviews, tests, and other steps in the process that keep the quality of code high. Letting developers push forward, experimenting, and building sections of code quickly can help continue to push your development effort quickly. As long as they’re willing to undo and redo their efforts as they get closer to the end product.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • No More Foreign Keys?

    I struggle with removing FKs as a general idea. It’s presented in a piece from a developer. It appears to me that if you do this, you are merely moving the problem, and potentially causing other issues. Apart from Grant’s response on performance, which is very true, there are other challenges with microservices and separate databases for each service.
    Let’s say we are doing something trivial, like a simple e-commerce site. I could have a separate database for products, one for orders, one for shipping actions, one for customers, etc. However as I scale, I’m not using intra-database queries or FKs/joins/other techniques to verify information for an order. Now I’m sending messages. At small scale, these are trivial. At large scale, my network traffic and messaging is vastly growing and potentially a bottleneck.
    Also, since there is a “products” database, and an “orders” database and a “customers” database, I still can’t necessarily scale out these microservices beyond their own machine. Perhaps I can scale them higher in terms of each being as large a machine as I can have now with a single Oracle/SQL Server/MySQL/etc. box, but I’m still have a scaling issue. I also now have a new messaging problem I need to manage and architect. If I lose one database, how gracefully can my application degrade or does it start to fail with unforeseen interactions? Do I create more frustration when customers cannot place an order because we can’t find the customer db or because the site is down?
    Certainly there are domains of problems that would work fine here. Spotify is probably a good example of this. There may be e-commerce systems that allow this, perhaps being willing to notify customers after some time (hopefully minutes, perhaps hours) that a product is out of stock because the message from the orders database didn’t get to the application. There are certainly ways to copy and replicate some  data, assuming that “fresh enough” works for your application. However that’s a business decision more than a coding decision. This also means that you also have a new load on your system to move data. I also think the more you work with singleton rows and single objects in an application context, the less you need RDBMS capabilities.
    Ultimately I think FKs and strong RDBMS systems work very well in many situations. They work less well in many situations and your domain may qualify for those areas where a NoSQL, a multi-db, or other architecture works. We should certainly investigate and write about where things work and don’t work, but I wouldn’t assume the RDBMS with it’s PKs and FKs isn’t a valid, well thought out and incredibly useful architecture.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.5MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

  • Why Use Version Control?

    I’ve spoken with quite a few people in the last few years that don’t see enough value in version control to actually implement it. Most of them agree using a VCS is a good idea, but they aren’t doing it. Maybe this analogy will help.

    I was listening to Ike Ellis talk about VCS and he said something like this. Imagine you had a ten foot board to walk across and it was laying on the ground. You’d be willing to walk, skip, dance, maybe cartwheel across it. After all, what are the consequences?

    Now imagine this board is 50 feet off the ground. How do you cross it now? Many of us would be more cautious. In fact, if you’re like me, you’re likely laying down on the board, gripping tightly as you move inch by inch.

    However, what if there were a net below you? And if you fell, this net would toss you right back onto the board where you’d fallen so you could continue on from the last place you’d had success.

    That safety net is your VCS, and when you’re working along coding, you’re crossing a board that’s quite a distance from the ground.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( MB) podcast or subscribe to the feed at iTunes and LibSyn.