Tag: SQLNewBlogger

  • Import a CSV with a Header Row using BCP–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Here are some hints to get started.

    I was demoing something recently and needed to show someone how to grab some data from a CSV text file. Since this was a task the person needed to do regularly, but with different files, they wanted to ensure this was programmatic from a command line call outside of SQL Server. They knew the basics of bcp, but weren’t sure how to deal with a header row.

    This is actually fairly simple as you will see.

    BCP Basics

    I have a simple file that looks like this:

    Time,System Production (Wh)
    08/01/2022,"58875"
    08/02/2022,"61260"
    08/03/2022,"60866"
    08/04/2022,"66395"

    I have a basic table of this structure:

    CREATE TABLE [dbo].[Stage](
         [ProdTime] [varchar](20) NULL,
         [ProdValue] [varchar](100) NULL
    ) ON [PRIMARY]
    GO

    This is just a demo import from this sample file. If I run a basic bcp command, I’d typically run this:

    bcp dbo.stage in export.csv -S Aristotle\SQL2017 -d way0utwest -T -t "," –c

    This runs easily, as you see below:

    2022-08-17 12_56_58-D__Downloads

    However, this is my data:

    2022-08-17 12_56_51-solarloading.sql - ARISTOTLE_SQL2017.way0utwest (ARISTOTLE_Steve (69))_ - Micros

    That’s not right. The first row is a header row, and while I can quickly and easily fix this, it’s better not to have to process this. Easier to fix this on the import.

    To do this, I need to look at the bcp documentation and include a flag. The –F flag is for the first row, which I want to set to 2. If I truncate the table and run this command:

    bcp dbo.stage in export.csv -S Aristotle\SQL2017 -d way0utwest -T -t "," -c -F 2

    I see these results:

    2022-08-17 13_02_04-solarloading.sql - ARISTOTLE_SQL2017.way0utwest (ARISTOTLE_Steve (69))_ - Micros

    No header row. There are still issues with this import, but this solves one problem, which is what the SQL New Blogger post is for.

    SQLNewBlogger

    This is a quick example of a post that is part of my daily work. I was showing a customer this, and I had to mock something up, so I grabbed a little sample data and did that.

    I spent about 15 minutes around other work getting this post written and screenshots taken. You could do this as well and show how you import data in a cleaner fashion.

  • Using a Regular Expression to Detect a Number–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Here are some hints to get started.

    I had a customer recently that was looking to work with Data Masker for SQL Server and had questions about how to handle some situations. In this case, they needed to detect a number type in a field that was overloaded with multiple types of data. Here’s an example of what they had in their “string” (varchar) field. Look at the stringvalue column below:

    2022-06-07 08_23_13-SQLQuery1.sql - ARISTOTLE.SimpleTalk (ARISTOTLE_Steve (58))_ - Microsoft SQL Ser

    If the string was a “nnn nnn nnnn” number value, then they wanted to change it. If it had other values, then leave it alone. This is really a query problem and a WHERE clause to structure.

    One would think this is where you use ISNUMERIC() and try that. If I run this, I get zero rows back.

    SELECT d.stringvalue
    FROM dbo.ddmdemo AS d
    WHERE ISNUMERIC(d.stringvalue) = 1

    This isn’t really a number, as the sequence has spaces. What if we try this:

    SELECT d.stringvalue
    FROM dbo.ddmdemo AS d
    WHERE ISNUMERIC(REPLACE(' ', '', d.stringvalue)) = 1

    It also returns no values.

    Really, this appears to really be a regular expression type of query, so I could do this, using LIKE.

    SELECT *
    FROM dbo.ddmdemo AS d
    WHERE d.stringvalue LIKE '[0-9]%'

    That, however, gives me two rows in this set of data. I see these results:

    2022-06-07 08_31_14-SQLQuery1.sql - ARISTOTLE.SimpleTalk (ARISTOTLE_Steve (58))_ - Microsoft SQL Ser

    The reason is that I am matching the first character only. The argument is a pattern and using square brackets implies a single character in a range. Since there are a lot of different patterns, and the “234223 Test” matches that, I ought to be more specific.

    This particular pattern from the customer is 3 numbers, space, 3 numbers, space, 4 numbers. Anything else is non matching. Since there could be trailing spaces, I’d really want this:

    SELECT d.stringvalue
    FROM dbo.ddmdemo AS d
    WHERE d.stringvalue LIKE '[0-9][0-9][0-9] [0-9][0-9][0-9] [0-9][0-9][0-9][0-9]'

    This returns my single row. It would match any row that is of the pattern “nnn nnn nnnn” where n is a numerical value from 0-9.

    There are other considerations here, and certainly this is likely to be a complex set of masking rules, but this shows a relatively simple way to detect a numerical pattern in a string.

    SQL New Blogger

    This was an interesting case. I initially thought  LIKE and an expression, but thought maybe there was a quicker way with isnumeric(). I didn’t find one, so I explained that and then the way that did work for me.

    To me, this gives someone who glances at my blog a bit of insight into how I think and what I considered. This might be how they think, or someone on their team thinks. This might get me an interview.

    Write about the problems you solve and how/why you do it.

  • Finding Memorial Day–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Here are some hints to get started.

    It’s Memorial Day in the US. A holiday, though I’m off on a trip to Germany today.

    I wanted to have a fun Memorial Day Question of the Day today, and I decided to write some code to calculate Memorial Day. This post looks at how the code works.

    The Algorithm

    Memorial day is always the last Monday in May. For me, I decided to find all the Mondays in May and then take the last one.  I started with a tally table to get a list of days. In any given year, we can find the first day of the year with this code:

    DATEADD (yy, DATEDIFF (yy, 0, GETDATE ()), 0)

    Now I use a DATEADD with my tally table to find the first 200 days of the year.  The end of May will always fall in this number of days. Here is the code for a list of the first 200 days of the current year:

    WITH myTally (n)
    AS ( SELECT n = ROW_NUMBER () OVER (ORDER BY
                                           (SELECT NULL))
          FROM
            ( VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) a (n)
            CROSS JOIN ( VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) b (n)
            CROSS JOIN ( VALUES (1), (2)) c (n) )
        , cteCurrYearDates (myDate)
    AS ( SELECT DATEADD (DAY, n, DATEADD (yy, DATEDIFF (yy, 0, GETDATE ()), 0))
          FROM myTally)
        , cteMay (Mondays)

    Once I have this, I now need to get the Mondays in May. I can use the DATEPART and MONTH functions to find this. Actually, I ought to use DATEPART to be consistent here, but my habit is MONTH() for the month.

    I also need to set the DATEFIRST for this code, otherwise the day of the week will be inconsistent. Here’s the code for this:

    SET DATEFIRST 7;
    WITH myTally (n)
    AS ( SELECT n = ROW_NUMBER () OVER (ORDER BY
                                           (SELECT NULL))
          FROM
            ( VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) a (n)
            CROSS JOIN ( VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) b (n)
            CROSS JOIN ( VALUES (1), (2)) c (n) )
        , cteCurrYearDates (myDate)
    AS ( SELECT DATEADD (DAY, n, DATEADD (yy, DATEDIFF (yy, 0, GETDATE ()), 0))
          FROM myTally)
        , cteMay (Mondays)
    AS ( SELECT cteCurrYearDates.myDate
          FROM cteCurrYearDates
          WHERE
            DATEPART (WEEKDAY, cteCurrYearDates.myDate) = 2
            AND MONTH(cteCurrYearDates.myDate) = 5
    )

    This gives me a list of Mondays in May for the current year. I want the last one, which isn’t easy to do in a result set. However, I can get the first one with a TOP 1 limit. The easy way to get the last one is reverse the order of the rows and then take the first one. I do this with an ORDER BY.

    Here’s the complete code:

    SET DATEFIRST 7;
    WITH myTally (n)
    AS ( SELECT n = ROW_NUMBER () OVER (ORDER BY
                                           (SELECT NULL))
          FROM
            ( VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) a (n)
            CROSS JOIN ( VALUES (1), (2), (3), (4), (5), (6), (7), (8), (9), (10)) b (n)
            CROSS JOIN ( VALUES (1), (2)) c (n) )
        , cteCurrYearDates (myDate)
    AS ( SELECT DATEADD (DAY, n, DATEADD (yy, DATEDIFF (yy, 0, GETDATE ()), 0))
          FROM myTally)
        , cteMay (Mondays)
    AS ( SELECT cteCurrYearDates.myDate
          FROM cteCurrYearDates
          WHERE
            DATEPART (WEEKDAY, cteCurrYearDates.myDate) = 2
            AND MONTH(cteCurrYearDates.myDate) = 5
    )
    SELECT --TOP 1
            Mondays
    FROM cteMay
    ORDER BY cteMay.Mondays DESC;

    Now I can get Memorial Day for the current year.

    SQL New Blogger

    This is a great example of breaking down an algorithm and explaining it to the reader. If you write T-SQL code for a living, you might write a series of posts on how you solve various problems in T-SQL and explain the process. Link to places where you learn, and show some results that give a feeling for how you built the code.

    This took about 10 minutes to write once I’d built all the code. I didn’t go into details with individual result sets, but you could easily do that to show how the code works.

  • Posting Data and Code–#SQLNewBlogger

    Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Here are some hints to get started.

    I wrote a SQL New Blogger post recently on running totals and used some images to show data. A reader sent me a note to say it would be nice to see the code that helps put the post together. I thought that was a good suggestion, so that’s this post.

    How can you add code and data to your post?

    Publishing Code

    There are a number of ways to add code to a blog post. Some of the platforms have specific plugins to handle different types of code. There are also some plugins in Open Live Writer, which I use, for code.

    I’ve stopped using those.

    It’s simpler to me to just use the PRE tag, for selecting preformatting for a set of code. I use WordPress here, and when I added the code used in this post, I highlighted the code and then selected the Preformatted style. This is shown below.

    2022-05-03 09_27_25-Edit Post “A Monthly Running Total–#SQLNewBlogger” ‹ Voice of the DBA — WordPres

    The PRE tag works well for me and it’s simple. If you want to spend the time configuring code for colors specific to your language, feel free, but I am less concerned about that.

    Generating CREATE Statements

    For most demo set ups, I create a new table and I copy the code over. For example, I was doing some DDM testing and I made this table.

    2022-05-03 10_49_09-SQLQuery1.sql - ARISTOTLE_SQL2017.sandbox (ARISTOTLE_Steve (55))_ - Microsoft SQ

    To add that to this post, I’d CTRL+A to select all the code and then copy it. I can paste it below and then format with the PRE tag.

    CREATE TABLE [dbo].[DDMEmailTest](
         [MyID] [int] IDENTITY(1,1) NOT NULL  CONSTRAINT [DDMEmailTestPK] PRIMARY KEY ,
         [MyName] [varchar](100) NULL,
         [Email] [varchar](100) MASKED WITH (FUNCTION = 'email()') NULL,
         [Salary] [int] NULL,
    GO

    In LiveWriter there isn’t a PRE tag, so I just quickly edit the source and add a “re” to the paragraph tags. You can see where I’ve added PRE below, highlighted by the arrow:

    2022-05-03 10_50_04-Posting Data and Code–#SQLNewBlogger - Open Live Writer

    The other option is to script the table to the Clipboard. I can do this in SSMS easily.

    2022-05-03 10_55_23-SQLQuery2.sql - ARISTOTLE_SQL2017.sandbox (ARISTOTLE_Steve (66))_ - Microsoft SQ

    I then paste the code into Live Writer.

    Note, I usually edit out the USE and SET statements.

    Adding INSERT Statements

    After the DDL for tables is around, the next important thing for readers is data. If you have the INSERT statements from your testing, then use those. As above, paste those in here.

    If not, there are a few options. I have SQL Prompt, so I can select from the table, highlight the results, and then “script as insert”.

    2022-05-03 10_57_38-SQLQuery2.sql - ARISTOTLE_SQL2017.sandbox (ARISTOTLE_Steve (66))_ - Microsoft SQ

    The other option is to “build” an insert statement like this:

    SELECT TOP 10
            '(''' + CAST(spt.ProductionDate AS VARCHAR(10)) + ''', ' +
           + CAST(spt.Actual AS VARCHAR(10)) + ', '
           + CAST(spt.Estimate AS VARCHAR(10)) + ')'
    FROM dbo.SolarPowerTracker AS spt;

    I can wrap the INSERT tablename VALUES in front of this and then paste this into the post.

    SQL New Blogger

    Not really a technical post, but this does show a little technical skill. Writing about how you solve problems or build on other posts is a good way to show you have some varied skills that an employer might like.

    This post took me about 15 minutes to write.