Typing Strange Characters–#SQLNewBlogger

I’ve had to type a few non-English characters lately, and this blog talks about how to do this.

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

The Need for Non-English Characters

Recently I wrote a blog on using Chinese characters. While you can copy/paste from Google, perhaps you want to actually type something regularly. For example, I have been learning French, and I sometimes need to

Another one I’ve used fairly often is the British pound symbol.

Typing Unicode characters

There are Unicode characters for these symbols. While some keyboards may include these, the US ones do not. I can switch keyboards, and I have a Japanese keyboard, but

A few commons ones for me are:

  • é (French e with acute (forward) accent) – 0233
  • è (French e with the grave accent) – 0232
  • ô (French o with the circumflex) – 0244
  • ç (French c with the cedilla,hanging thing) – 0231
  • £ (British pound symbol) – 0163
  • はい (Japanese yes, hai) – a little harder

To type these, I press the ALT key and hold it. Then I enter the number. So, holding ALT and entering 0163 gives me this: £

The Japanese is a little harder. There I needed to enable the Microsoft IME keyboard that lets me type the phonetics for Japanese characters. There are other ways to do this, but that’s what I did.

Learning to work with other languages and characters has been interesting to me, and it’s nice to be able to type São Paulo instead of Sao Paulo. Especially when I communicate with people whose names contain non English letters.

Posted in Blog | Tagged , , | 8 Comments

Securing All Your Connections

I read an interesting blog from the cyber security team at Microsoft, noting you should use TLS for your SQL Server connections. I would assume most professionals know that using TLS and secure protocols across the network is important. I would also assume few of us are willing to get real certificates for all of our SQL Servers, especially those in dev and test environments.

Setting up a certificate for a server instance isn’t hard, but it’s also not easy. It’s also something that I don’t know how easy would be to automate in many environments. I know that you can use APIs from somewhere like Let’s Encrypt, but integrating that into a server setup process would be something. What about integrating this into instances in containers? I don’t know that I think many SQL Server organizations have DNS integrated for most of their database servers, much less asking them to also get certificates set up.

I do think that Microsoft likely has better infrastructure in place, and better staff resources, than most of us. I can see this being something needed for Azure SQL DB and Azure MI. Heck, this might be something we want for all services that we get in the cloud. Since we don’t control much of anything other than the service, and Microsoft can automate the process of generating certificates, this makes sense.

I think this might be something that is a good idea for production systems as well, ensuring that when a DBA or system makes a privileged connection to a database server in production, the person or process is sure of which database server is at the other end of the connection. I don’t know that I’ve seen an exploit in the wild where someone impersonates a prod server and captures credentials in some man-in-the-middle attack, but I wouldn’t be surprised if it happens.

Do you think you also ought to avoid trusting the server certificate? I don’t think this is practical in many orgs, but I’d be curious what you think today.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Posted in Editorial | Tagged , | Comments Off on Securing All Your Connections

The Four Different Types of Flyway Files

This is documented, somewhat, but I wanted to put this down for myself, as the I don’t love the docs and they are hard to sort through.

Flyway is open source software owned and maintained by Redgate, my employer. There are also paid versions with additional features.

The Main Files

There are essentially 4 different types of files you can use for scripts in Flyway. These are:

  • versioned scripts
  • repeatable scripts
  • baseline scripts
  • undo scripts

I’ll discuss each of these in light detail below. This isn’t intended to replace the documentation, but give a short explanation of each.

For each type of file, there is a naming standard, which is the same for all. Essentially, there is are four parts. As an example, I’ll use V8.2__add_new_table.sql to explain this. The four parts are:

  • The prefix for the type of script, in this case, a V script.
  • The version. for my example this is 8.2.
  • The separator, always two underscores
  • The description, add_new_table, in my case.
  • The suffix, which is an extension

Versioned Scripts

These are the V scripts, which are named with a V to start. These are scripts which are run once on each target, and intended to be those that change the schema. The “V” is the default, but this can be configured. I wouldn’t change this.

These are typically the CREATE or ALTER scripts that you run to change objects. You can have as much SQL code in here as you’d like. Think of these as a SQL Compare deploy script. One or my objects changes.

The run once is nice because once you’ve deployed these, you don’t want to run them again. If you write idempotent scripts, you wouldn’t care but most people don’t do that well.

Repeatable Migrations

These are used less in the current Redgate paradigm, but these are repeatable scripts. They are designed to run every time you run “flyway migrate”. Again, the default is an “R” script, but this is configurable. Again, don’t change this.

These scripts don’t have versions because they execute over and over. This could be used for creating or recreating programmable code objects, like view/procedures/functions/packages. They could also be used for places you need to ensure data is always there or has certain values.

These scripts run after the migrate scripts, so be aware of that.

Baseline Scripts

By default, baseline scripts are “B” scripts. This can be changed. These scripts have a database version that you want to start with for some purpose. Like a new development effort or new deployment project.

These are run for new environments only, and usually contain all the base objects you might need. If you start a Flyway project on a database that has objects, put all the code to recreate this state (CREATE objects and add data) in a baseline script.

Undo Scripts

The undo scripts are the “U” scripts. This can be changed, and these should have a version that matches a versioned script. We can run one of these scripts corresponding to those versions to “undo” changes. Since you can’t really undo anything in the database world, these should include code that reverses the action of the V script.

Be careful with these, especially if you run these more than a few minutes after the V script. Flyway isn’t checking that you won’t lose data.

I’d only use these after extensive testing in a pipeline and then only if my deployment broke immediately. These are good for putting in a previous version of a view/proc/function and quickly rolling back.

Posted in Blog | Tagged , , | Comments Off on The Four Different Types of Flyway Files

Concurrency Challenges Around Schema Changes

I saw a great question on Twitter from Frank Pachot, a developer advocate of Yugabyte. He wrote: Without thinking how your preferred database deals with it, what do you expect if:

  • session 1 starts to reads table T
  • session 2 drops table T
  • session 1 continues to read

The choices in his poll were: session 2 waits, session 2 fails, session 1 fails, both fail. My first thought was SQL Server and the default need for session 2 to get an exclusive lock. In that case, session 2 would wait. Most people answered that same way, but then Frank posted a follow-up with a link to his blog. The answer for Yugabyte is that session 1 fails as it gets the message that the table was deleted.

Leaving aside the decision to drop a table, imagine this is some schema change instead. In the blog, some good points are raised about how to handle high concurrency changes, and the potential problems with having session 2 wait. On a busy system, this could cause lots of blocking as threads stack up behind session 2.

It’s an interesting read about the challenges of distributed system design and how to handle changes. In some sense, I get that this makes sense, but I wonder where this causes issues. If any schema change on the table by session 2 were to cause an error in session 1, that would be bad. However, does this mean that the database engine must now evaluate whether a column change impacts a query in flight? Then decide to send an error? What about evaluating views or procedures/functions that depend on

Does this mean that all nodes need to sync up the schema changes quickly, and at a higher priority than data movements? I don’t know exactly how Yugabyte distributes data, and if there are copies on multiple nodes, but I assume there are. This adds complexity to the communication between nodes, which is likely needed. Honestly, if someone drops a table and they should have, we probably don’t want clients getting results. If they do this accidentally, I’d like to know about it quickly.

The question is interesting, and there are multiple ways to look at this, but I found it fascinating to spend a few minutes thinking about the complexities of data in distributed systems and the challenges involved. This also made me think that the people who keep data safe and fix problems when they occur are invaluable in the modern world.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Posted in Editorial | Tagged , | 2 Comments