Author: way0utwest

  • SQL Bits 2023

    I’m honored to be heading back to SQL Bits 2023. I was selected to deliver one session, in the professional development track. With so many people submitting, I’m not surprised that this was the only selection.

    This year, SQL Bits is in Wales, at the International Convention Centre just north of Cardiff. I’ve been to Cardiff only once, and am looking forward to going back and exploring a bit.

    SQL Bits has been my favorite conference over the years. Others are great, but I have always loved this one. If you want to join me, start making some plans and register today.

    Hopefully I’ll see you there.

  • No Flyway Baseline No Migration

    In my experiments with the Flyway CLI (fwcli), I’m finding some interesting behavior, some of which is catching my by surprise.

    This post looks at the baseline command and the issues with not having one. I also cover a naming issue. This is a bit long, but I wanted to document what happens as I experiment. I’ll condense down what a baseline does in another post (or two or three).

    The Scenario

    I created a FWTest database and put a small table in it. I can see in my Object Explorer (OE) that there is just one table. Assume there are no views, functions, etc.

    2022-12-27 16_17_44-SQLQuery2.sql - ARISTOTLE_SQL2022.FWTest (ARISTOTLE_Steve (83))_ - Microsoft SQL

    My flyway.conf file points here, and if I run Flyway Info, I see the results below. The only important part is the “schema version”, which is empty, and the table, which shows one versioned migration called getone.sql.

    2022-12-27 16_18_35-cmd

    The migration is in my SQL folder, which is below the location of the flyway.conf file. The configuration file is in the smoketests folder and it has only two lines uncommented: 1 for the SQL Server connection string and one for the location of the migrations, which is the SQL folder. The relevant lines are:

    2022-12-27 16_20_52-● flyway.conf - flywaysimpletalk - Visual Studio Code

    That’s the basic start. I’ve got a database, and I created a new script. The script in the file is shown here:

    2022-12-27 16_22_20-V1__getone.sql - ARISTOTLE_SQL2022.master (ARISTOTLE_Steve (64)) - Microsoft SQL

    Running Flyway

    Now, I have a migration script I want to apply to a new database. What happens if I run flyway migrate. Will this create my procedure? Let’s see.

    The output shown below runs and give me a green line and a red like. One success, one error.

    2022-12-27 16_23_10-cmd

    The success is that the script was named correctly, so it passed validation.

    The error is that there is no flyway_schema_history table. This is where all Flyway activity is tracked inside the database. Without this, there’s nowhere to stick the data on script execution.

    The error does note that we need to run flyway baseline or set the baselineonmigrate option to true. The default for this is false.

    Adding a Baseline

    Let’s do the baseline thing. The documentation for baseline is very poor (as of Dec 2022) in my opinion. I’ve sent a few notes around the company, as I think this needs to be cleaned up.

    In any case, this will do a couple things. First, it creates the flyway_schema_history table (under dbo for SQL Server) as the place to store history for Flyway. Next, it will add a row as the baseline version for this database. I don’t have a baseline script, a “B” script, but that’s OK. I don’t need it for now.

    Let’s try this. I’ll run this on my database and we see the results below. I’ve captured the text, ignoring the licensing and connection string part. The results are really here:

    Creating Schema History table [FWTest].[dbo].[flyway_schema_history] with baseline ...
    Successfully baselined schema with version: 1

    This shows me we have created the table and added a baseline of version 1. If we look in the database, we see this: the new table, but no proc.

    2022-12-27 17_05_53-SQLQuery2.sql - ARISTOTLE_SQL2022.FWTest (ARISTOTLE_Steve (83))_ - Microsoft SQL

    If I run flyway info, I see these results (again, ignoring the licensing and connection stuff).

    2022-12-27 17_10_54-cmd

    What I see in here is that my database version 1 is baselined with no scripts. However, the getone.sql script is noted as a versioned script but ignored because of the baseline.

    The baseline is supposed to be a level that includes all scripts up to that number. It’s not well explained in the docs, but this means that any scripts up to the baseline are assumed to have been executed in the database.  This is the baseline. The baseline.sql script also is supposed to include the contents of all previous migration scripts, but as my test showed, I don’t need that.

    The Problem with Baselines

    The thing I have to know here is that this baseline version means no scripts at this version or lower will be executed. Since my script was a v1 script, it gets ignored, as you see below:

    2022-12-27 17_10_32-cmd

    Why this says 2 migrations, I don’t know. There’s only one script. I assume this is counting the non-existent baseline script.

    However, nothing migrated, and my procedure isn’t created.

    Be aware, that you want baseline scripts and other scripts to have discrete numbering.

    Summary

    Getting started with Flyway means we need a baseline to get going. We can do this without a script, but we do need to run flyway baseline, or set an option. I’ll look at those two items in a future post.

    I also need to be careful with naming of scripts, as a script that matches the numbering of the baseline will not get executed.

  • Forgoing Tech Investments

    The US was hit with a number of storms over the Christmas holiday weekend. This disrupted air travel for many airlines and their customers, but one of the worst hit was Southwest Airlines. They accounted for most of the cancellations, over half of their scheduled flights at one point.

    A number of places reported talking with Southwest employees who blamed the lack of tech investment by Southwest over time, noting this caught up with them. The Chief Operating Officer disagreed, saying that their scheduling system is the best in the world, even as the CEO noted that their scheduling software couldn’t keep up and they fell back to manual operations.

    Most of us likely have no idea of how Southwest software works or the scope of the problem. This airline does tend to operate differently than many others in that they mostly fly point to point, rather than using hubs. Possibly they have the best point-to-point scheduling software in the world, but it still couldn’t keep up with the storms covering much of the US.

    There’s an interesting perspective on Facebook, supposedly from a pilot with 35 years of experience with SouthWest. If you don’t want to click, his view is the hands-on CEO retired years ago and accountants were appointed as CEO and COO. They improved the money flow, but neglected investments in tech and weren’t aware of how the business really runs day to day. The infrastructure and software deteriorated, and they’ve had many small issues, but issues that were bigger than other airlines. They’ve started turning around with a CEO that is more hands-on, but they’re digging out of a hole.

    Like many of you, I’ve built and operated software over the years. I sometimes realize just how hard it can be to keep up with the demands of customers for adjusting how our systems work. I also know that it’s easy to slow your investment in a system that appears to works and limit your efforts to just maintenance work. Allan Hirt wrote about this.

    This does bring up the issue of investing in systems and maintaining them over time. I see why many companies would prefer to purchase software and let someone else manage the investment in ongoing development. I also know that for companies that see software as strategic, likely there needs to be regular investment, upgrading and refactoring code, as well as finding ways to scale higher and use resources more efficiently. Especially for databases.

    The battle between enhancing software and reducing technical debt is a constant one. I see this struggle being one that project managers and developers never agree on, but in the companies that seem to thrive, there is a balance. Perhaps it’s splitting the sprints, perhaps it’s allocating regular time during each development period, or maybe there’s another way.

    One thing is certain. We need to find a balance. Otherwise we might get into the situation where a complete rewrite or replacement of software is warranted; a situation that is almost always very costly.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Daily Coping 16 Jan 2023

    Today’s coping tip is to learn something new and share it with others.

    I do this all the time. It’s a part of my job, and I did this recently with a PostgreSQL container. I also did this with a piece of my coaching training, helping kids understand where to serve from /to for better chances of success.

    Sharing is a great way to ensure to engage with the world and give back.

    I started to add a daily coping tip to the SQL Server Central newsletter and to the Community Circle, which is helping me deal with the issues in the world. I’m adding my responses for each day here. All my coping tips are under this tag.