Tag: Flyway

In my experiments with the Flyway CLI (fwcli), I’m finding some interesting behavior, some of which is catching my by surprise.

This post looks at the baseline command and the issues with not having one. I also cover a naming issue. This is a bit long, but I wanted to document what happens as I experiment. I’ll condense down what a baseline does in another post (or two or three).

The Scenario

I created a FWTest database and put a small table in it. I can see in my Object Explorer (OE) that there is just one table. Assume there are no views, functions, etc.

My flyway.conf file points here, and if I run Flyway Info, I see the results below. The only important part is the “schema version”, which is empty, and the table, which shows one versioned migration called getone.sql.

The migration is in my SQL folder, which is below the location of the flyway.conf file. The configuration file is in the smoketests folder and it has only two lines uncommented: 1 for the SQL Server connection string and one for the location of the migrations, which is the SQL folder. The relevant lines are:

That’s the basic start. I’ve got a database, and I created a new script. The script in the file is shown here:

Running Flyway

Now, I have a migration script I want to apply to a new database. What happens if I run flyway migrate. Will this create my procedure? Let’s see.

The output shown below runs and give me a green line and a red like. One success, one error.

The success is that the script was named correctly, so it passed validation.

The error is that there is no flyway_schema_history table. This is where all Flyway activity is tracked inside the database. Without this, there’s nowhere to stick the data on script execution.

The error does note that we need to run flyway baseline or set the baselineonmigrate option to true. The default for this is false.

Adding a Baseline

Let’s do the baseline thing. The documentation for baseline is very poor (as of Dec 2022) in my opinion. I’ve sent a few notes around the company, as I think this needs to be cleaned up.

In any case, this will do a couple things. First, it creates the flyway_schema_history table (under dbo for SQL Server) as the place to store history for Flyway. Next, it will add a row as the baseline version for this database. I don’t have a baseline script, a “B” script, but that’s OK. I don’t need it for now.

Let’s try this. I’ll run this on my database and we see the results below. I’ve captured the text, ignoring the licensing and connection string part. The results are really here:

Creating Schema History table [FWTest].[dbo].[flyway_schema_history] with baseline ...

Successfully baselined schema with version: 1

This shows me we have created the table and added a baseline of version 1. If we look in the database, we see this: the new table, but no proc.

If I run flyway info, I see these results (again, ignoring the licensing and connection stuff).

What I see in here is that my database version 1 is baselined with no scripts. However, the getone.sql script is noted as a versioned script but ignored because of the baseline.

The baseline is supposed to be a level that includes all scripts up to that number. It’s not well explained in the docs, but this means that any scripts up to the baseline are assumed to have been executed in the database.  This is the baseline. The baseline.sql script also is supposed to include the contents of all previous migration scripts, but as my test showed, I don’t need that.

The Problem with Baselines

The thing I have to know here is that this baseline version means no scripts at this version or lower will be executed. Since my script was a v1 script, it gets ignored, as you see below:

Why this says 2 migrations, I don’t know. There’s only one script. I assume this is counting the non-existent baseline script.

However, nothing migrated, and my procedure isn’t created.

Be aware, that you want baseline scripts and other scripts to have discrete numbering.

Summary

Getting started with Flyway means we need a baseline to get going. We can do this without a script, but we do need to run flyway baseline, or set an option. I’ll look at those two items in a future post.

I also need to be careful with naming of scripts, as a script that matches the numbering of the baseline will not get executed.

In a previous post, I got Flyway installed as a CLI utility (command line interface). This post will look at the first connection to a database.

If you need to install Flyway on Windows, see my previous post.

Concepts

Flyway is a command line executable that takes various parameters to control what it does. Essentially, these are command verbs and decide what action is running. The previous post looked at the version command.

There are also parameters that can be included before the command. There are lots of parameters.

However, much of the way Flyway works is controlled by the flyway.conf file, which is inside a conf folder where you installed Flyway. If this file is found in the current folder, then Flyway will use configuration parameters from this file.

Note, the inline parameters should override those in the conf file.

Configuration

I find it much easier to use the configuration files. While you can specify this with the configFiles parameter, I often ensure that I run Flyway from within a particular folder that has a flyway.conf file. This has worked well.

Copy your flyway.conf from your install folder (under conf) to a new folder where you will experiment. For me, I set up a new folder called “smoketests” where I’m doing a little experimenting. As you can see, I only have two files in here:

The default conf file can get a little confusing. It’s full of many options, which are somewhat documented. My default looks like this when I open it:

Most everything is commented out, which makes it harder to figure out what to do.

The thing to remember is that you uncomment those settings that you want to set. For example, the drivers section is extensive. Some of theses are included, some you need to download. Since this are all Java based JDBC drivers, it can look strange to a SQL Server person.

The thing that gets lost in here for me is that I don’t comment out the line that starts with SQL Server. Instead, I need to copy and paste the jdbc part to the flyway.url= line. My valid connection would look like this:

Let’s try connecting with the info command. When I run Flyway info, I get asked for a user and password. Annoying, but not that bad. However, I then get an error:

This is a secure connection error. I do have TCP/IP enabled, which I know Java needs. This is more about a change the SQL Server team made, which requires secure connections.

Let’s add this to our connection string: ;trustServerCertificate=true

This gives me this string:

flyway.url=jdbc:sqlserver://aristotle\SQL2022;databaseName=FWPOC_1_Dev;trustServerCertificate=true

Now when I connect, things work. I still have to enter a name and password, but I can connect and I get back something.

I see some info on flyway versions. There’s a new version available. I’m also licensed for Flyway Enterprise.

Then I see the filesystem folder, sql, is missing. This is where the migrations are stored, so if I want to add migration scripts, I need to add a folder.

Next, I get the complete connection string. Lots of options here, of which most are defaults.

Then I see the schema is empty. Flyway works off schemas, which is what many platforms require. We get lazy in SQL Server and often just use dbo, but most other platforms want schemas set up.

Finally I see that there are no migrations run in this database, and I see an empty status table.

That’s a lot, and I connected to the database. Nothing changed in the database, and no flyway_schema_history table was added. This essentially let me know Flyway was working.

The last thing I’ll do is add this to my connection string: integratedSecurity=true

This lets get away from the name and password on Windows, but using Windows Auth for SQL Server.

That’s it. I’ll keep working on different commands in Flyway and getting to know the CLI.

One of the things I’ve been trying to do is dig in more deeply to the Flyway command line (CLI) as part of my work with Redgate. While Flyway Desktop is amazing, it’s a wrapper, and my inclination is to ensure I understand the underlying technology.

When I first downloaded Flyway, it wasn’t obvious what needed to be done to connect in different ways, so I decided on a short tutorial that would help me remember how and teach others a few things. This post goes over downloading Flyway and getting started.

Setup

The setup for Flyway is easy. The Community version is free, though there are also Teams and Enterprise editions.

Flyway Community works on many platforms. I’ll choose Windows below.

The Windows version is a zip file. Click this, and it will download.

Then unzip this. I have a c:\Utilities folder where I keep stuff and I’ve added a flyway folder there. I unzip the latest into this folder.

If you care about versioning, you might drop Flyway into a version named folder, but I typically don’t on this machine. For production deployments, I would be more careful.

You then need to add this to your PATH. If you are on Windows 10 or 11, then in the Control Panel (Windows+I), search for environment. Edit the variables for your account.

Once the new dialog appears, find the PATH variable and click Edit.

Now add an entry for the folder where you extracted the Flyway files. For me, this is c:\utilities\flyway.

That’s it, Flyway is installed.

We can check at the command line with “flyway version”.

If you have entered a license key, then you’ll see that message.