Monoliths and Microservices

Posted on March 24, 2025 by way0utwest

I have run into a lot of people in the last few years that love decoupled software and microservices. It seems many people are aiming to move their work in this direction, and while I see some appeal, I also see tremendous additional complexity that has moved out of the software into the operations and debugging space. As I read the book Observability Engineering, I found myself thinking the complexity of setting up more logging and instrumentation in an observability framework as well as the costs of managing a system start. That caused me to think this is overkill for most software.

To be clear, I don’t think that Uber could have been built as a few monoliths, and there are other examples of such systems, but most of us don’t work at that scale. There are lessons to be learned about large, real-time software systems, and certainly Google, Amazon, Spotify, Netflix, etc. can help us understand how certain techniques work better at scale, but most of us don’t work at that scale.

Scale to me is thousands of connections and terabytes of data. Those companies tend to work at a couple orders of magnitude above that. Those are also companies that must have real time systems up to survive. The vast majority of companies I’ve worked at might suffer a loss with an outage of a system, but honestly, we could survive a day or two of recovery.

Heck, look at all of the companies that have had portions of their digital infrastructure knocked offline from ransomware the last few years. Some failed, but most didn’t. The incident sucked for IT staff, but those companies survived. I still remember the SQL Slammer worm forcing us to take our entire network offline for multiple days at a large software company. We still ran sales, and support, and other systems independently or from paper. Granted that was 20 years ago, but I’m sure Redgate and many other organizations could survive a few weeks with no network.

Uber on the other hand, that would be a disaster for them. They would likely survive, but at a high cost. How many people would jump to a new service and never look back?

I was reading a piece on coupling and complexity, which I’ll discuss a bit in another article, but it got my thinking of how often I see people overcomplicating their work by trying to move to a decoupled world instead of hiring and training people to adopt better architectures and communicate better. After all, moving to microservices isn’t going to avoid the training issue. You’ll still have to teach people more.

And while you’re at it, force every developer to pass a SQL test every year. That might help you build better systems as well. SQL isn’t going away and better SQL coding in will result in much better applications everywhere.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Posted in Editorial | Tagged | Comments Off on Monoliths and Microservices

Friday Flyway Tips: State-based deployment with Flyway

Posted on March 21, 2025 by way0utwest

I was asked about state-based deployments in Flyway, so I decided to show how this can work with a quick demo. This post walks through the process.

I’ve been working with Flyway and Flyway Desktop for work more and more as we transition from older SSMS plugins to the standalone tool. This series looks at some tips I’ve gotten along the way.

A Steady State

I have a Flyway project setup already. I’ve refreshed the screen and as you can see here, there aren’t any differences between my database and my project in version control.

2025-02_0349

In SSMS, I’ll create a new table in my FWState_1_Dev database. This is where my project is pointed in Flyway Desktop (FWD).

2025-02_0350

I’ve created a new table and it exists. If I  check my QA database, FWSTate_3_QA, I don’t see this object.

2025-02_0351

If I refresh my project in Flyway Desktop (FWD), I’ll see my change. I can pick it and see the code, and then I’ll click “Save to project” in the upper right.

2025-03_0106

Once the code is saved, I see the confirmation and I have the code ready for commit to a VCS. I won’t do that here, though you should.

2025-03_0107

What I want to do now is go to the left bar and select “deploy”, which is the rocket icon.

2025-03_0108

When I do this, I see a blank screen, as I don’t have a target. I can select that on the right if I’ve configured one.

2025-03_0109

If I pick the drop down, you will see my dev and QA environments. I made changes in the dev environment, and want these changes to go to QA. I can also click the “Manage environments” button. Let’s do that.

2025-03_0110

When I do that, I see a list of environments. I use this a lot to double check that I’m deploying to the right place for this project and where to look for changes. If you work across multiple projects as I do, this is handy.

You can see below I’ve selected QA with the green checkmack in the radio button. Once you pick your environment, you can click “Confirm”. If you need a new target, click Configure.

2025-03_0111

I return to the deploy screen, and this time I see the changes that have not been deployed. In this case, that’s one change, but if there were more, I would see them. I can click any object to see the changes, and once I’ve selected what to deploy, I can click deploy.

2025-03_0113

When I click Deploy the script is generated and I see a preview. I also can see that this is from my schema model on disk being deployed to the Qa database. That’s handy as this might take a few minutes and who knows what distractions might appear in my life.

I also can execute this as a transaction or not, and copy the script if I want to examine it in another tool (or format it like SQL Prompt).

2025-03_0114

When I click Deploy, I need to confirm my deployment. Always a good step, though I think if you do this often, your muscle memory will be ready to just click confirm.

2025-03_0115

Once the deployment runs, we see a message this was successful.

2025-03_0116

If I now check the QA database, I see the object.

2025-03_0117

Summary

This post has shown how you can perform manual deployments of changes in a state-based method using Flyway. This is a quick way to move your changes from one machine to another.

Flyway can do much more, and for smoother automation, check out Flyway Enterprise.

Flyway is an incredible way of deploying changes from one database to another, and now includes both migration-based and state-based deployments. You get the flexibility you need to control database changes in your environment. If you’ve never used it, give it a try today. It works for SQL Server, Oracle, PostgreSQL and nearly 50 other platforms.

Video Walkthrough

Here is a video of the process below.

Posted in Blog | Tagged , , , | Comments Off on Friday Flyway Tips: State-based deployment with Flyway

More Supply Chain Attacks

Posted on March 21, 2025 by way0utwest

The latest code supply chain attack isn’t a direct attack, but a failure of a system designed to be efficient. There is a Go (Go-lang) module that had a malicious module inserted into it years ago. Someone caught this and removed the module from GitHub, but Google had cached this and has kept it alive for the last three years.

This attack worked because it relied on the Go Module Proxy services, which prioritized caching. Even when the source changed, the cache wasn’t invalidated or reloaded, which seems like a major ovesight or even larger design flaw, but I know everyone trying to maintain software archives at scale tends to cache a lot. After all, so many developers might load modules in their daily work, in CI, etc. that caching matters.

However, the bigger issue is that criminals are getting more enterprising and aiming for supply chain attacks where possible. One would hope that any PRs (pull requests) are carefully examined, but the truth is that a lot of people might depend on their unit tests passing in a CI build. They are not necessarily looking to see if anyone has actually entered malicious code or even poorly written code. I can see some people worried more about code structure and naming (or tabs/spaces) than what the code does. Imagine seeing a change that looks innocuous at the top of a file, all tests pass, and you merge the change, but didn’t notice there was a few new functions added below in the file because they’re not obvious in the UI.

Someone maintaining a popular code repo as a side project might be fooled here, but if this were in a corporate repository we might be even more susceptible. We have more reasons to trust the PR isn’t a problem if the code passes tests inside of an organization. After all, who things criminals might insert code into their corporate repo? Not many people do, but we’ve seen quite a few successful supply chain attacks in the past. Who knows how many more we don’t know about.

Security is a hard business, and when it’s extended to the code we write, it might even be harder. I know there are security scanning solutions you can integrate into your codebase, but those detect what they know about and criminals keep finding new ways to attack us. Ultimately I think we depend on code maintainers carefully examining PRs from outside their circle of trusted individuals, and even then, things can slip through.

Some days I think it’s truly a mad, mad world.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Posted in Editorial | Tagged | Comments Off on More Supply Chain Attacks

Advice I Like: The Main Thing

Posted on March 21, 2025 by way0utwest

The main thing is to keep the main thing the main thing. – from Excellent Advice for Living

We can summarize this advice as focus on what’s important. I too often see authors and others get distracted from the main point on which they are writing. I see this in meetings when people start to speculate about something related (or even unrelated) and forget the purpose of the work.

This becomes an even bigger issue when someone goes off to work on a project for a period of time and drifts away from their original goal or focus. You might think you don’t do this, but I bet most of you do. I know I do. I sometimes start to change my focus, thinking the current thing I’m working on is more important than the main goal.

An example is that I might be working on a new presentation. I’ll start to outline the talk, add some bullets and notes, maybe think about demos, etc. At some point, I get distracted with which pictures make my point and I start to waste time on choosing pictures, rather than remembering that the point of the talk isn’t a picture, it’s to teach something. The picture might help, but it’s not the most important thing and it’s not where I should spend a lot of time.

Today’s interruption culture of having notifications, slack, email, etc. available on devices makes this worse. The large amount of things we often engage in at any one time also makes this hard.

Focusing on what you want to achieve and keeping that in mind is important. For presentations, I often put my abstract in the first slide notes, so that I remember what I’m focusing on. For coaching, we often list a few things we want to work on, and as we run drills or evaluate the practice we can look back and see where our focus is. What was the main thing.

Keep the main thing the main thing.

I’ve been posting New Words on Fridays from a book I was reading, however, a friend thought they were a little depressing. They should be as they are obscure sorrows. I like them because they make me think.

To counter-balance those, I’m adding in thoughts on advice, mostly from Kevin Kelley’s book. You can read all these posts under the advice tag.

Posted in Blog | Tagged , | Comments Off on Advice I Like: The Main Thing