A New Word: Opia

opia – n. the ambiguous intensity of eye contact

The entry for this says “so much can be said in a glance, “ which I think is very true. Most of the time eye contact is amazing. I think we can express a connection, a welcoming vibe, a neighborly willingness to talk.

I try to smile and make eye contact often in my travels. I feel this creates a more pleasant view of the world, and it often gets others to smile or at least grin. The same reason I wave to people often. They wave back.

I also wave to dogs and horses, though they don’t smile or wave back.

I think eye contact is mostly good, but I know it can be creepy. Certainly too many men stare at women, and there are those that project ill-will or bad intentions. I think we can sometimes wonder about an intense eye contact and the meaning behind it.

Casual eye contact, however, is something I think helps create bonds between us.

From the Dictionary of Obscure Sorrows

Posted in Blog | Tagged , | Comments Off on A New Word: Opia

Friday Flyway Tips: Searching a Migration

This was actually a cool tip I saw internally from one of the product managers, when trying to find specific text in a migration.

I’ve been working with Flyway Desktop for work more and more as we transition from older SSMS plugins to the standalone tool. This series looks at some tips I’ve gotten along the way.

A Large Project

I’ve got a project in Flyway Desktop that I add to regularly. It doesn’t do anything amazing, but I keep slowly adding objects, with the idea that I want to build up a large project for showcasing different things to customers. You can look at the project here if you are interested.

In my list of migrations, I’ve got a number of migrations, 21 to this point. It is a little hard to read names here, but I’ve named each script to make searching easier. I’ve included a create or alter, a type, and an object name. This helps me use the search box at the top of FWD, which searches migration names.

2024-10_0002

Most of these are relatively small scripts, one or two objects. A few are a little larger. If I select a particular one (21 as shown below), the code scrolls off the screen. Even if I expand the code listing to take up the entire screen, I can’t see all the code.

More importantly, it can be hard to visually find something in the code. If you’ve ever edited a stored procedure of 100 or more lines, you know what I mean.

2024-10_0003

One of our PMs pointed out recently that you can use the standard browser CTRL+F if you’ve clicked into the script. I’ve done this below and the search widget pops up at the top of the code. I’ve entered part of an object name here, prod, and you can see this shows 6 matches.

The first one is highlighted in a pale yellow color in the code.

2024-10_0004

If I click the down (or up) arrows in the search widget, the code jumps around, as it would in a browser. You can see below I’ve jumped down to the 4th occurrence, with the 2nd and 3rd highlighted in an orange(ish) color, while the 4th also has the yellow background bar.

2024-10_0005

Many of us would perform work on this code in an IDE (SSMS, ADS, etc.), but while managing a project or perhaps evaluating something else, we might want to quickly find something in code in FWD. Perhaps reviewing a change I’m about to commit. USing CTRL+F might be a handy feature.

I do wish the search was more comprehensive at the top, and could dig through all the files in the project, but at least I can search within a file here. Having a good naming convention for migrations also helps me to find the high level purpose for a migration in the list.

Flyway Enterprise

Try Flyway Enterprise out today. If you haven’t worked with Flyway Desktop, download it today. There is a free version that organizes migrations and paid versions with many more features.

If you use Flyway Community, download Flyway Desktop and get a GUI for your migration scripts.

Video Walkthrough

I made a quick video showing this as well. You can watch it below, or check out all the Flyway videos I’ve added:

Posted in Blog | Tagged , , , | Comments Off on Friday Flyway Tips: Searching a Migration

Adding Row Numbers to a Query: #SQLNewBlogger

I realized that I hadn’t done much blogging on Window functions in T-SQL, and I’ve done a few presentations, so I decided to round out my blog a bit. This post will start with the ROW_NUMBER() function as a gentle intro to window functions.

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. This is also part of a series on Window Functions.

A Basic Set of Data

I’m going to use some fun data for me. I’ve been tracking my travels, since I’m on the road a lot. I’m a data person and part of tracking is trying to ensure I’m not doing too much. Just looking at the data helps me keep perspective and sometimes cancel (or decline) a trip.

In any case, you don’t care, but I essentially have this data in a table. As you can see, I have the date of travel, the city, area, etc. I also have a few flags as to whether I was traveling that day, if I spent a night away from home, and how far I was.

2024-09_0199

I have a travelID in here, which is a sequence, but what if I wanted to the trips I took in August 2024. I’d want a distance > 0 (not at home) and filters by dates. Adding that to my query, I’d run this:

SELECT
   TravelID
, TravelDate
, TravelCity
, Area
, Province
FROM travel
WHERE
   TravelDate     > '2024/07/31'
   AND TravelDate < '2024/09/01'
   AND Distance > 0
ORDER BY TravelDate;

This gives me these results:

2024-09_0202

There are 9 rows in here, but they have a weird ID number, plus these are different trips. I can just add a row_number to this data, and I’d see this result. Ignore the OVER and track I used, but you can see an incrementing number added to each row. The second column in the result set matches with the number added by SSMS on the side.

2024-09_0204

What if I wanted to see the separate trips with some row number for the day in each city?

That’s where a row_number() can help.

Creating a Window

The window comes from the OVER() clause, which is added to a number of functions, including Row_number(). The OVER() clause lets me set a window or rows on which the function works. I can set a partition and an order.

The partition is a column where we are essentially grouping data. For me, this would be the city. When I change city, I want to reset the number. Looking at the data above, I’d expect to see 1, 2 for the first 2 days in Minneapolis, then a 1 for a day in Fort Collins, and another 1 for day in Aurora

The ordering is what order is the data in the partition. In this case, I want to have the data in the window ordered by traveldate, so I’ll use that. I now have this code:

SELECT
   TravelID
, ROW_NUMBER () OVER (PARTITION BY TravelCity
                       ORDER BY TravelDate)
, TravelDate
, TravelCity
, Area
, Province
FROM travel
WHERE
   TravelDate     > ‘2024/07/31’
   AND TravelDate < ‘2024/09/01’
   AND Distance > 0
ORDER BY TravelDate;

And I get these results, where I can see that I essentially had 4 trips (all with number 1s), and these were the trips:

  • Minn – 2 days
  • Fort Collins – 1 day
  • Aurora – 1 day
  • New York City – 5 days

2024-09_0205

This shows how row_number() gives me a sequence based on the partition. The select null part in the earlier query just ignores the order by, which is required for row_number(). With no order, how do we know what sequence?

Let’s change this slightly. What if I partition by Province? Then I see this:

2024-09_0206

We put the data in date order, and ran through each province. In this case, my two 1 day trips around Colorado are bucketed (partitioned) together and I see one less trip. If I did this by country, I’d see all of this as one sequential list, since all my trips were in one country.

However, if I did countries for June, I’d see this, with the raw data on the left and the row_number() on the right. There’s a weird sequence in here; can you see it?

2024-09_0208

The weirdness is that my trips to England were broken up by a trip to Italy. So while my sequence looks good for the first part of the trip to English for 6 days, when I returned a week later, we get numbers 7, 8, 9. That’s because the data is grouped first by country, and the sequence added. The ordering of the sequence is by date, so the later days (June 13,-15) are marked with the higher sequence that continues on.

Hopefully this gives you a basic look at row_number() and some of the possibilities. I’ll examine it further in another post, along with various other window functions.

SQL New Blogger

Complex coding and finding weird situations are things employers want you to be able to do. If you work on algorithms or you’re learning new language elements, blog about them. That will impress people.

This post took me about 30 minutes, plus about 15 minutes or playing with code to set things up. You could likely do this in an hour if you’ve never blogged, though let someone proof things for you.

Posted in Blog | Tagged , , , | 5 Comments

Keep. It. Simple.

I get a tech newsletter most days, which has news that I enjoy, but interspersed among the news and ads are projects, frameworks, or repos, most of which I’ve never heard of before. I used to read these, but it seems that there is an endless list of these, which all have marketing descriptions that somehow claim this set of code solves problems that others don’t or that this code is easy to use and integrate with, or well, I don’t know what other promises. I’m usually turned off by the end of the first sentence.

The thing I’ve noticed is that there are so many projects out there. Even in the database space, if I happen to read a discussion on some aspect of databases, such as database deployment frameworks, I’ll see links to technologies I’ve never heard of in my life. Some are small projects, and some are small companies, but there is an amazing variety of solutions for any tech problem. I’m not sure most of them are much different from the others, but the Not Invented Here syndrome seems to be everywhere. These observations also remind me of just how vast the world is and how little I see of it on a daily basis.

I ran across an article recently on simplifying the way we build software. In this instance, the authors focused on the complexity of JavaScript frameworks, with a number of authors/developers/consultants saying they limit their use of frameworks and try to keep to simple technologies. There is a quote that I find myself agreeing with and disagreeing with at the same time from David Heinemeier Hansson: “The merchants of complexity will try to convince you that you can’t do anything yourself these days. You can’t do auth, you can’t do scale, you can’t run a database, you can’t connect a computer to the internet. You’re a helpless peon who should just buy their wares. No. Reject.”

I do think that many frameworks, products, etc. are very complex and there is a cost to using them. They can be overwhelming and hard to teach to others, and those costs can outweigh the benefits. The costs could be actual money or they could be the time it takes to learn, integrate, and teach the technology to others. Any change you make to your architecture will have impacts that you haven’t considered, and it will slow down your agility. Adding new technology does the same thing.

At the same time, some things are hard. Auth is hard. Lots of people struggle with this and Troy Hunt has made a good living at https://haveibennpwned.com/ pointing out all the poor decisions people have made. Including a bunch of them that use the auth provided by some tech stack. Running databases is hard. Can you run one? Sure. Will you run it well as your app grows? Experience shows me most of you can’t. Can you set up a simple backup strategy? Of course, but that doesn’t matter. You being able to restore things is what matters. There are plenty more examples of tech problems or failures that companies run into as they build and scale software. There are also plenty of examples where using a framework saves a bunch of time for development teams. Look at mobile apps, most of them are based on some framework because learning all the ins and outs of iOS and Android is too hard for most people.

For. Most. People.

DNN can do a lot of things that most people can’t. He’s very talented and smart. Lots of organizations have one or two very talented people, but they also have a lot of average people. They struggle to teach and train them things, and ultimately, they don’t want to spend time teaching everyone about how to manage CSS or auth a user from a service or deploy changes to a database. They want a framework that makes it simple for the ever-changing staff to get work done related to their business, which is more often than not building software. They need to build software, but they need the software built to serve their customers in some way. That’s their goal, serving customers.

I am all for keeping things simple. I want to minimize the number of technologies we use, and I want to limit the training time it takes to get people productive. However, I don’t want to build everything myself because I’m not sure I can, and I certainly know I don’t to go to that much effort.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Posted in Editorial | Tagged | Comments Off on Keep. It. Simple.