Category: Blog

  • A New Word: Opia

    opia – n. the ambiguous intensity of eye contact

    The entry for this says “so much can be said in a glance, “ which I think is very true. Most of the time eye contact is amazing. I think we can express a connection, a welcoming vibe, a neighborly willingness to talk.

    I try to smile and make eye contact often in my travels. I feel this creates a more pleasant view of the world, and it often gets others to smile or at least grin. The same reason I wave to people often. They wave back.

    I also wave to dogs and horses, though they don’t smile or wave back.

    I think eye contact is mostly good, but I know it can be creepy. Certainly too many men stare at women, and there are those that project ill-will or bad intentions. I think we can sometimes wonder about an intense eye contact and the meaning behind it.

    Casual eye contact, however, is something I think helps create bonds between us.

    From the Dictionary of Obscure Sorrows

  • Friday Flyway Tips: Searching a Migration

    This was actually a cool tip I saw internally from one of the product managers, when trying to find specific text in a migration.

    I’ve been working with Flyway Desktop for work more and more as we transition from older SSMS plugins to the standalone tool. This series looks at some tips I’ve gotten along the way.

    A Large Project

    I’ve got a project in Flyway Desktop that I add to regularly. It doesn’t do anything amazing, but I keep slowly adding objects, with the idea that I want to build up a large project for showcasing different things to customers. You can look at the project here if you are interested.

    In my list of migrations, I’ve got a number of migrations, 21 to this point. It is a little hard to read names here, but I’ve named each script to make searching easier. I’ve included a create or alter, a type, and an object name. This helps me use the search box at the top of FWD, which searches migration names.

    2024-10_0002

    Most of these are relatively small scripts, one or two objects. A few are a little larger. If I select a particular one (21 as shown below), the code scrolls off the screen. Even if I expand the code listing to take up the entire screen, I can’t see all the code.

    More importantly, it can be hard to visually find something in the code. If you’ve ever edited a stored procedure of 100 or more lines, you know what I mean.

    2024-10_0003

    One of our PMs pointed out recently that you can use the standard browser CTRL+F if you’ve clicked into the script. I’ve done this below and the search widget pops up at the top of the code. I’ve entered part of an object name here, prod, and you can see this shows 6 matches.

    The first one is highlighted in a pale yellow color in the code.

    2024-10_0004

    If I click the down (or up) arrows in the search widget, the code jumps around, as it would in a browser. You can see below I’ve jumped down to the 4th occurrence, with the 2nd and 3rd highlighted in an orange(ish) color, while the 4th also has the yellow background bar.

    2024-10_0005

    Many of us would perform work on this code in an IDE (SSMS, ADS, etc.), but while managing a project or perhaps evaluating something else, we might want to quickly find something in code in FWD. Perhaps reviewing a change I’m about to commit. USing CTRL+F might be a handy feature.

    I do wish the search was more comprehensive at the top, and could dig through all the files in the project, but at least I can search within a file here. Having a good naming convention for migrations also helps me to find the high level purpose for a migration in the list.

    Flyway Enterprise

    Try Flyway Enterprise out today. If you haven’t worked with Flyway Desktop, download it today. There is a free version that organizes migrations and paid versions with many more features.

    If you use Flyway Community, download Flyway Desktop and get a GUI for your migration scripts.

    Video Walkthrough

    I made a quick video showing this as well. You can watch it below, or check out all the Flyway videos I’ve added:

  • Adding Row Numbers to a Query: #SQLNewBlogger

    I realized that I hadn’t done much blogging on Window functions in T-SQL, and I’ve done a few presentations, so I decided to round out my blog a bit. This post will start with the ROW_NUMBER() function as a gentle intro to window functions.

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. This is also part of a series on Window Functions.

    A Basic Set of Data

    I’m going to use some fun data for me. I’ve been tracking my travels, since I’m on the road a lot. I’m a data person and part of tracking is trying to ensure I’m not doing too much. Just looking at the data helps me keep perspective and sometimes cancel (or decline) a trip.

    In any case, you don’t care, but I essentially have this data in a table. As you can see, I have the date of travel, the city, area, etc. I also have a few flags as to whether I was traveling that day, if I spent a night away from home, and how far I was.

    2024-09_0199

    I have a travelID in here, which is a sequence, but what if I wanted to the trips I took in August 2024. I’d want a distance > 0 (not at home) and filters by dates. Adding that to my query, I’d run this:

    SELECT
       TravelID
    , TravelDate
    , TravelCity
    , Area
    , Province
    FROM travel
    WHERE
       TravelDate     > '2024/07/31'
       AND TravelDate < '2024/09/01'
       AND Distance > 0
    ORDER BY TravelDate;

    This gives me these results:

    2024-09_0202

    There are 9 rows in here, but they have a weird ID number, plus these are different trips. I can just add a row_number to this data, and I’d see this result. Ignore the OVER and track I used, but you can see an incrementing number added to each row. The second column in the result set matches with the number added by SSMS on the side.

    2024-09_0204

    What if I wanted to see the separate trips with some row number for the day in each city?

    That’s where a row_number() can help.

    Creating a Window

    The window comes from the OVER() clause, which is added to a number of functions, including Row_number(). The OVER() clause lets me set a window or rows on which the function works. I can set a partition and an order.

    The partition is a column where we are essentially grouping data. For me, this would be the city. When I change city, I want to reset the number. Looking at the data above, I’d expect to see 1, 2 for the first 2 days in Minneapolis, then a 1 for a day in Fort Collins, and another 1 for day in Aurora

    The ordering is what order is the data in the partition. In this case, I want to have the data in the window ordered by traveldate, so I’ll use that. I now have this code:

    SELECT
       TravelID
    , ROW_NUMBER () OVER (PARTITION BY TravelCity
                           ORDER BY TravelDate)
    , TravelDate
    , TravelCity
    , Area
    , Province
    FROM travel
    WHERE
       TravelDate     > ‘2024/07/31’
       AND TravelDate < ‘2024/09/01’
       AND Distance > 0
    ORDER BY TravelDate;

    And I get these results, where I can see that I essentially had 4 trips (all with number 1s), and these were the trips:

    • Minn – 2 days
    • Fort Collins – 1 day
    • Aurora – 1 day
    • New York City – 5 days

    2024-09_0205

    This shows how row_number() gives me a sequence based on the partition. The select null part in the earlier query just ignores the order by, which is required for row_number(). With no order, how do we know what sequence?

    Let’s change this slightly. What if I partition by Province? Then I see this:

    2024-09_0206

    We put the data in date order, and ran through each province. In this case, my two 1 day trips around Colorado are bucketed (partitioned) together and I see one less trip. If I did this by country, I’d see all of this as one sequential list, since all my trips were in one country.

    However, if I did countries for June, I’d see this, with the raw data on the left and the row_number() on the right. There’s a weird sequence in here; can you see it?

    2024-09_0208

    The weirdness is that my trips to England were broken up by a trip to Italy. So while my sequence looks good for the first part of the trip to English for 6 days, when I returned a week later, we get numbers 7, 8, 9. That’s because the data is grouped first by country, and the sequence added. The ordering of the sequence is by date, so the later days (June 13,-15) are marked with the higher sequence that continues on.

    Hopefully this gives you a basic look at row_number() and some of the possibilities. I’ll examine it further in another post, along with various other window functions.

    SQL New Blogger

    Complex coding and finding weird situations are things employers want you to be able to do. If you work on algorithms or you’re learning new language elements, blog about them. That will impress people.

    This post took me about 30 minutes, plus about 15 minutes or playing with code to set things up. You could likely do this in an hour if you’ve never blogged, though let someone proof things for you.

  • T-SQL Tuesday #179: The Detective’s Toolkit

    It’s time for the monthly T-SQL Tuesday blog party. This month a longtime friend, Tim Mitchell is hosting and he’s got a neat invite. He’s asking us how to dig into a request and figure out th meaning in our database, our files, or something else. How do you solve a data mystery?

    If you haven’t participated in a T-SQL Tuesday blog party, start a blog and join us. If you’re written a post, host one of the months in 2025. Don’t get too caught up in the rules. At this point, I’m just trying to get

    Inspector Gadget

    2024-10_0157For some reason, the first thing that comes to mind is Inspector Gadget. My kids used to love this show and I can still hear the song and catch phrase in my head.

    Being a detective doesn’t require gadgets, but they do make things easier to use. As Tim asks in the invitation, how do you go about learning about your data?

    One of the main things I’ve done in the past is use Notepad++ to load a large text file and look at it. These days VS Code seems to work well for my scales, and I can get an idea of the structure of text files.

    If I’m looking for meaning in a database, often I need two things:

    1. An Extended Events session
    2. A business analyst or end-user

    Usually I only look at data in databases when someone asks me to find something or explain something. Often, this means relating something in an application to a data model that might not be intuitive, obvious, or even rational.

    The big way for me to figure out where data is stored is to get a user to show me something in the application, or tell me what data they see from their side. Then when they query something, I can trace what happened with xEvents and see what tables were queried.

    The does most of the detective work for me, though sometimes I’m doing some trial and error as any specific screen in an application, might query lots of tables.