Distributed Monoliths

Posted on December 6, 2024 by way0utwest

I was watching a video called Microservices are Technical Debt. In it, the person being interviewed said that a lot of people really have a distributed monolith. That caught my eye since I’ve worked with a number of customers who are trying to adopt microservice architectures for their applications. I think this is less a performance/scaling choice than a reworking of their software development teams, and I’m not sure they will end up with a better system.

What is a distributed monolith? I am not an expert, but this appears to be a place where all the services still depend on each other. For example, I might have a service getting user profile info that an app calls, with another service getting previous orders, and a third service that returns inventory. In a monolith, if any of these are down, the others don’t work. In a distributed monolith, these might be built independently, but perhaps the core app/web page still requires all these to be working to show the user something.

That’s how my gym app seems to work sometimes. It can’t get my profile and the list of classes and the things I’ve registered for, so it hangs. This is very annoying when all I really need is my ID QR code, which never changes, and is validated by another system.

In a microservices architecture, if any of these were down, the others would still be working, and the app would continue functioning. If inventory is down, I can still update my profile. If previous orders are working, I can view them even if the current inventory doesn’t work.

Things are loosely coupled for the end user and for the development teams. Not for just one of those groups.

I’m not sold on microservices, especially when they include databases. There is a lot of power, and efficiency, from being able to pull together different data sets in a database efficiently rahter than doing so in an application, which might require different API calls, separate db connections, and the delays that come with returning more data than needed to let the client join it together. At the same time, I can see how separate services could potentially scale much higher over time.

The thing that I tend to see is that most of us never need to really scale as high as the Google/Facebook/Spotify/Netflix/etc. organizations that espouse these microservice architectures. They have different problem domains and different needs than most of us. Even Uber has realized that one size doesn’t fit all and is moving some things from microservices to macro ones. I’m not against microservices, but they need to be used in appropriate situations, not all of them.

What most of us need is a little more help from Brent or Erik or Paul or someone else in the form of a little consulting that tunes our code to run more efficiently. And if we use them, we should learn a few things from them so that we write better code the next time.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Posted in Editorial | Tagged | Comments Off on Distributed Monoliths

A New Word: Fardle-dun

Posted on December 6, 2024 by way0utwest

fardle-din – n. a long-overdue argument that shakes up a relationship, burning wildly through your issues like a forest fire, which clears out your dry and hollow grievances and reminds you that your roots run deeper than you think.

I have had a few fardle-dins. Both in my personal life and at work. The private ones are private, but let me tell you about a work faux-pax on my part.

I had complained to an individual at a previous company that the way their team approached some software work was incomplete. In my mind, the team often simplified things too often and looked at the software for only a simple use-case, not examining common situations outside of the core use case. At the same time, the teams thought I wasn’t focused enough on different problems to ensure they were well designed.

After seeing this a few times, I publicly criticized their approach and offended the manager of that team. I received a private tongue lashing for this, with the manager pointing out that they do a lot of different work to consider different situations.

We ended up working through our differences, with me understanding their view, but the team also starting to understand why their approaches each tended to be over simplifications and created tunnel vision. I apologized publicly and we burned through our issues, creating a better relationship for the long term.

It’s never a pleasant process when you encounter a fardle-dun, but it can be cathartic and build something better.

From the Dictionary of Obscure Sorrows

Posted in Blog | Tagged , | Comments Off on A New Word: Fardle-dun

FIRST_VALUE vs. Min: #SQLNewBlogger

Posted on December 4, 2024 by way0utwest

I had mentioned some new T-SQL functions for SQL Server 2022 and a commenter asked about the difference between Min() and First_value. This post looks at a few cases.

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

The Scenario

Let’s set up some data and examine these functions.  First a table and some data:

SET ANSI_NULLS ON

GO

SET QUOTED_IDENTIFIER ON

GO


CREATE TABLE [dbo].[MonthlySales](
     [TransactionDate] [date] NULL,
     [SalesAmount] [decimal](10, 2) NULL

) ON [PRIMARY]

GO

Some data for 2024:

insert MonthlySales (transactiondate, SalesAmount)
  values
  ('2024-01-30', 100),
  ('2024-02-28', 200),
  ('2024-03-30', 300),
  ('2024-04-30', 400),
  ('2024-05-30', 500),
  ('2024-06-30', 600),
  ('2024-07-30', 700)

Now, a query and some results. This query runs the same over() clause for first_value, min, and sum. The sum just shows totals.

2024-11_0108

It seems that first_value and min do the same thing. Let’s add one item to this. I’ll add two columns that remove the first_value and min from the total sales. Let’s assume we want to get rid of the first month’s sales for some reason (they lag).

2024-11_0109

OK, this looks good.

The Difference

Now, I’ll add some 2023 sales with this code:

insert MonthlySales (transactiondate, SalesAmount)
  values
  ('2023-01-30', 10),
  ('2023-02-28', 20),
  ('2023-03-30', 5),
  ('2023-04-30', 40),
  ('2023-05-30', 50),
  ('2023-06-30', 60),
  ('2023-07-30', 70)

When we re-run the query, we see different results for lines 3-7. This is because above, we had increasing sales in every line, when ordered by date. In 2023, we have a dip in sales, which happens, so the min value after Mar is 5, but the first_value is still 10 (from Jan).

2024-11_0111The same sort of issue can come with last_value and max, as the ordering might not match the sorting of values.

At first glance, these might seem like duplicate functions, but that really depends on the use cases you have for windowing functions and your data. You might often order by a numeric used in an aggregate, in which case they can be the same. However, there are plenty of cases where these work differently.

Another quick example: in my baseball database we can see Barry Bonds HR counts by year, and the min differs from the first year in SFN.

2024-11_0113

SQL New Blogger

This post took me about 15 minutes to setup and write. I’ve practiced telling a story, but I bet most of you could produce this in 30-45 minutes of work. Easy if you spend 1 hour a week on your career branding.

Showcase your skills and set up a blog today.

Posted in Blog | Tagged , , | Comments Off on FIRST_VALUE vs. Min: #SQLNewBlogger

Reducing the Cycle Time

Posted on December 4, 2024 by way0utwest

There are lots of software development methodologies. This page lists a few, among them waterfall, agile, iterative, rapid, and more. What’s been interesting to me is that the process of deciding what to code and then whether it works doesn’t change much between different ways of building software.

Instead, the cycle time between when we ask a client what to do and when we deliver it changes. The more agile/lean we are, the lower the cycle time. The more waterfall-ish, the larger the cycle time. I guess that analysis and breakdown of problems into work also changes, as the scope in modern DevOps styles of development is smaller (more contained) than in waterfall.

However, we seem to follow the same steps. In the database world, we might do similar things if we think about how we build data models and code systems. We could get all the requirements and build the entire model, or we could get some requirements in an area, build that, and then ask for more. The former is more of a waterfall approach and the latter more agile/DevOps-y.

Is one better? Not really. I would say they are both situational. In some domains, waterfall might work better. When deciding to build a system to launch rockets, most of the problem domain is known and not changing often, so waterfall type approaches likely work well. Certainly we still went some level of decoupling to take advantage of changes that do occur, primarily in the hardware, but the overall problem remains the same.

However, in many of the business or software tooling places I’ve worked, no one has a good grasp of the entire domain. Heck, I think in most businesses, people roughly know how business runs, but they forget the myriad of exceptions that ensure our environments look chaotic to software, and they often constantly refine (or re-direct) the way the business works in pursuit of their latest goals. At times I’m amazed business runs smoothly, though I think this just shows how much we tolerate variance in business processes that we think are more set and defined than they are in reality.

I am a big believe in loose coupling and accepting uncertainty. I hope for the best, but plan for the worst, or at least, plan for things to change. I like NULLS in databases, not everywhere, but in places because have unknown values. I like agile/DevOps approaches to software because we rarely know all the information about a problem. Heck, sometimes clients don’t know the entire problem or don’t spend time thinking about the entire problem when they create requirements or requests for changes. Therefore, I like short cycle times, with the flexibility to change directions as necessary.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Posted in Editorial | Tagged | 1 Comment