Author: way0utwest

  • Factivism

    Bono is embracing his inner nerd.
    Bono is embracing his inner nerd.

    I believe that we should all give back to the world outside of our families at some point in our lives. This might not be the time for you, and you might not have a cause right now. However I think at some point performing volunteer work to better the world will give each of a deep sense of purpose in our lives. I was reminded of this recently while watching a talk from Bono, on the state of hunger and poverty in the world. Bono has devoted considerable personal energy and resources in working with one.org to fight extreme poverty around the world. This may or may not be your cause, but it was an interesting talk, one worth watching.

    The presentation wasn’t what I expected, with Bono embracing his “inner nerd”, as he put it. He talks not only about the state of his cause, but he uses data to present his case and inspire people to continue to work to make things better around the world. The term he used was not activism, but factivism. This is the idea of using facts and knowledge to further a cause. Whether you agree with Bono’s vision, his case, or his goal, the idea of using data to make changes is something most of us embrace as data professionals.

    The idea of using more data, more information to make decisions is something most of us do each day. However we don’t have to limit ourselves to just the data and knowledge we have at work. I find many people looking to showcase their skills or improve their brand. This might be a way in which you choose to do this. With the availability of data from governments and organizations, you can present your own data analysis, or even a visualization that shows how you work with a set of data. It’s the chance to make your own mark on the world, perhaps making the world better, or perhaps disputing the claims of others. Having data freely available allows us to check and verify someone’s work or dispute their conclusions. Both of which are important for a healthy debate.

    However you might tackle some data interpretation, you don’t need to make a political or social statement. You could just provide a story on how you examine the data, perhaps learning along the way. By showcasing your knowledge of data in some way, perhaps even explaining how to load the data, you get experience and practice in both a technical area, as well as in communicating your knowledge to others. If you choose to do so in pursuit of a cause, perhaps the world may be a little better for your efforts.

    Steve Jones


    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Flight Data

    I'm hoping this isn't the flight board when I travel tomorrow.
    I’m hoping this isn’t the flight board when I travel tomorrow.

    I’m getting ready to fly tomorrow, heading to SQL Saturday #175 in Fargo, ND. I have a morning flight, to give me a little time in the afternoon to go see the woodchipper. That’s if I don’t have any delays. I am planning on working in the airport and on the plane, so I won’t be less productive if there are, but I might not get the chance to see the famous machine from the movie.

    I’ll have a quick turnaround after this trip as well, coming home for a little over 24 hours before I turn around and head to the UK for meetings at Red Gate next week and SQL Bits XI next weekend. I’m hoping that none of the airlines have data issues since those can severely impact my travel times. Last week American Airlines had computer issues, and while United hasn’t had any in awhile, my fingers are crossed that this won’t be the weekend their systems go down.

    We have gotten so computerized with many of our transportation systems that problems can interrupt service, usually just causing annoyances, but there is a potential for lives to be lost. Since we don’t often know why the systems fail, it’s hard to know to what extend we have poor coding issues, improper secure development, incorrect configuration, employees susceptible to social engineering or something else. We even had a computer failure in space, though I’m sure the astronauts weren’t annoyed by the alert.

    These days it becomes increasingly hard to roll back to manual processes, not because of the complexity of the systems, but also because the knowledge on how the processes should work is being lost as employees leave the company. This means more delays, but hopefully not important safety rules being ignored or forgotten. I know the airlines have invested heavily in IT, and flights record a lot of data, but I hope they’ve done so in a distributed fashion. Lots of caching, fault tolerance, and most importantly, communication to allow their operations to proceed when the real world interferes.

    Steve Jones


    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.

  • Adding Data to Filestream with T-SQL

    I’ve written on how to enable Filestream and how to add a filegroup, but I haven’t touched the Filestream impact on your tables. This post will look at the table side of Filestream and how you get your binary data into the database.

    Let’s assume you have a filestream enabled database. If you look at my File and Filegroups post, you’ll see how to do this. Given that, we now need to create a table and add data to it.

    Creating a Table with Filestream

    Filestream doesn’t have a separate entity like Filetables. Instead, the Filestream attribute is added to a column in a table, much like the identity property.

    I’ll create a simple table that holds Filestream data.

    CREATE TABLE FSDemo
    (
        id UNIQUEIDENTIFIER ROWGUIDCOL NOT NULL UNIQUE
      , name VARCHAR(20)  
      , jpg varbinary(MAX) FILESTREAM
    );
    go

    Here we are creating a basic table that has an ID and name, which are essentially meta data for our filestream data. You could have any amount of information in this table, but the Filestream data is stored in the jpg field, where we have a varbinary(max) datatype, and the FILESTREAM attribute on the table.

    I could have a more complex schema, such as the Production.Document table in Adventureworks2008 (shown below)

    CREATE TABLE [Production].[Document](
        [DocumentNode] [hierarchyid] NOT NULL,
        [DocumentLevel]  AS ([DocumentNode].[GetLevel]()),
        [Title] [nvarchar](50) NOT NULL,
        [Owner] [int] NOT NULL,
        [FolderFlag] [bit] NOT NULL,
        [FileName] [nvarchar](400) NOT NULL,
        [FileExtension] [nvarchar](8) NOT NULL,
        [Revision] [nchar](5) NOT NULL,
        [ChangeNumber] [int] NOT NULL,
        [Status] [tinyint] NOT NULL,
        [DocumentSummary] [nvarchar](max) NULL,
        [Document] [varbinary](max) FILESTREAM  NULL,
        [rowguid] [uniqueidentifier] ROWGUIDCOL  NOT NULL,
        [ModifiedDate] [datetime] NOT NULL
        );

     

    Adding Data

    To add data to this table, we have a variety of choices using T-SQL. It’s possible to use an application, and pass in parameters that are streamed to the table, but for the simplicity of this example, let’s show two ways in T-SQL.

    First, let’s look at a direct insert. I can CAST my data to varbinary, and then insert it into the table. I have a small image that looks like this:

    circle

    It’s a simple circle, very small. The actual data in this image is this:

    0xFFD8FFE000104A46494600010101006000600000FFE100684578696600004D4 D002A000000080004011A0005000000010000003E011B00050000000100000046 01280003000000010002000001310002000000120000004E00000000000000600 000000100000060000000015061696E742E4E45542076332E352E313000FFDB00 43000201010201010202020202020202030503030303030604040305070607070 706070708090B0908080A0807070A0D0A0A0B0C0C0C0C07090E0F0D0C0E0B0C0C 0CFFDB004301020202030303060303060C0807080C0C0C0C0C0C0C0C0C0C0C0C0 C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C 0C0C0C0C0CFFC0001108000A000D03012200021101031101FFC4001F000001050 1010101010100000000000000000102030405060708090A0BFFC400B510000201 0303020403050504040000017D010203000411051221314106135161072271143 28191A1082342B1C11552D1F02433627282090A161718191A25262728292A3435 363738393A434445464748494A535455565758595A636465666768696A7374757 67778797A838485868788898A92939495969798999AA2A3A4A5A6A7A8A9AAB2B3 B4B5B6B7B8B9BAC2C3C4C5C6C7C8C9CAD2D3D4D5D6D7D8D9DAE1E2E3E4E5E6E7E 8E9EAF1F2F3F4F5F6F7F8F9FAFFC4001F01000301010101010101010100000000 00000102030405060708090A0BFFC400B51100020102040403040705040400010 277000102031104052131061241510761711322328108144291A1B1C109233352 F0156272D10A162434E125F11718191A262728292A35363738393A43444546474 8494A535455565758595A636465666768696A737475767778797A828384858687 88898A92939495969798999AA2A3A4A5A6A7A8A9AAB2B3B4B5B6B7B8B9BAC2C3C 4C5C6C7C8C9CAD2D3D4D5D6D7D8D9DAE2E3E4E5E6E7E8E9EAF2F3F4F5F6F7F8F9 FAFFDA000C03010002110311003F00E97FE0E87FF82CA7ED35FB04FED85E09F03 FC2BD6A5F87BE1097408B5C4D523D2ADAF1BC41746795248CBDC4722F970848C1 8940399373E43478FD75FF00827CFC68F177ED17FB0FFC2AF1D78F3461A078C7C 59E19B2D4F57B110B42B14F244199846DF346AFC384392A1C024E335E95E2EF87 BE1FF880968BAF687A3EB6B612F9F6C2FECA3B916F27F7D3783B5BDC60D6C5007 FFFD9

    I can insert this data directly into my table:

    INSERT INTO FSDemo(ID, name, jpg) 
    Values (NEWID()
          , 'circle.jpg'
          , 0xFFD8FFE000104A46494600010101006000600000FFE100684578696600004D4D002A000000080004011A0005000000010000003E011B0005000000010000004601280003000000010002000001310002000000120000004E00000000000000600000000100000060000000015061696E742E4E45542076332E352E313000FFDB0043000201010201010202020202020202030503030303030604040305070607070706070708090B0908080A0807070A0D0A0A0B0C0C0C0C07090E0F0D0C0E0B0C0C0CFFDB004301020202030303060303060C0807080C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0C0CFFC0001108000A000D03012200021101031101FFC4001F0000010501010101010100000000000000000102030405060708090A0BFFC400B5100002010303020403050504040000017D01020300041105122131410613516107227114328191A1082342B1C11552D1F02433627282090A161718191A25262728292A3435363738393A434445464748494A535455565758595A636465666768696A737475767778797A838485868788898A92939495969798999AA2A3A4A5A6A7A8A9AAB2B3B4B5B6B7B8B9BAC2C3C4C5C6C7C8C9CAD2D3D4D5D6D7D8D9DAE1E2E3E4E5E6E7E8E9EAF1F2F3F4F5F6F7F8F9FAFFC4001F0100030101010101010101010000000000000102030405060708090A0BFFC400B51100020102040403040705040400010277000102031104052131061241510761711322328108144291A1B1C109233352F0156272D10A162434E125F11718191A262728292A35363738393A434445464748494A535455565758595A636465666768696A737475767778797A82838485868788898A92939495969798999AA2A3A4A5A6A7A8A9AAB2B3B4B5B6B7B8B9BAC2C3C4C5C6C7C8C9CAD2D3D4D5D6D7D8D9DAE2E3E4E5E6E7E8E9EAF2F3F4F5F6F7F8F9FAFFDA000C03010002110311003F00E97FE0E87FF82CA7ED35FB04FED85E09F03FC2BD6A5F87BE1097408B5C4D523D2ADAF1BC41746795248CBDC4722F970848C18940399373E43478FD75FF00827CFC68F177ED17FB0FFC2AF1D78F3461A078C7C59E19B2D4F57B110B42B14F244199846DF346AFC384392A1C024E335E95E2EF87BE1FF880968BAF687A3EB6B612F9F6C2FECA3B916F27F7D3783B5BDC60D6C5007FFFD9
          );
    go

    The other method I can show is a direct insert, streaming from the OPENROWSET command. Here I’ll pic an image on my system and insert it.

    INSERT INTO FSDemo(ID, name, jpg) 
    Values (NEWID()
          , 'uma.jpg'
          , (SELECT * FROM OPENROWSET(BULK N'C:\Users\Steve\Documents\SampleData\Uma.jpg', SINGLE_BLOB) AS CategoryImage)
          );
    go

    Now if I query the table

    SELECT
      id, name, jpg
     from FSDemo;
    go

    I see my data.

    Capture_027

    When I look at this in Management Studio, I see the actual hex data, which isn’t easy to understand. I need another way to extract this data and render it. However I can show one more quick demo here. Let me take an XML document and insert it:

    declare @list XML
    select @list = '
    <Lists>
      <Groceries Store=''Safeway''>
        <Item>Milk</Item>
        <Item>Eggs</Item>
        <Item>Bread</Item>
      </Groceries>
      <Drinks Store=''Tipsys''>
        <Drink>Fat Tire</Drink>
        <Drink>Klinker Brick Cabernet</Drink>
        <Drink>Patron</Drink>
      </Drinks>
      <Ranch>
        <Item>electric fence ribbon</Item>
      </Ranch>
    </Lists>'
    ;
    INSERT INTO FSDemo(ID, name, jpg) 
    Values (NEWID()
          , 'sample.xml'
          , CAST( @list AS VARBINARY(MAX))
          );
    go

    Once this is done, the data is in binary, as I see from the selection from the table.

    Capture_029

    However if I cast this result back, I get the data as expected.

    Capture_028

    Hopefully this helps you to understand how to insert data into a Filestream table. In another post, I’ll look at how you can extract this data for rendering.

  • Natural Data

    Can you patent a gene, which is essentially data?
    Can you patent a gene, which is essentially data?

    Can you own data? In one sense, that’s what the arguments before the US Supreme Court over a Myriad Genetics patent are about. The company has patents for a few genes, which in and of itself, are really just data about a particular organism. There are a lot of technical issues here, and I’m not completely sure how I feel about this topic or who I’d like to see prevail in the case. I do think, however, that this does have lots of implications for how we might view other data in the future if the patent is upheld.

    As data professionals, we usually think of the mechanics of dealing with the various pieces of data in our organizations. We must protect data, ensure it’s integrity, make it available and more. A large part of our careers is spent manipulating or managing the bits and bytes stored in databases. We must extract information out, and often need to understand what the data represents to do this, but most of us haven’t been exposed to the issues of ownership or rights with data. I think that will increasingly change as government rules, regulations, and laws scramble to catch up with the digital world.

    Digital ownership is a very thorny subject, complex in its breadth and depth. We’ve seen technology companies fight against some laws, like CISPA, while supporting others. Often our government officials don’t understand the complexity of the topics, and I’m not sure that we in the technology business explain the the situation well. Even very intelligent people often don’t convey the implications of these complex topics, many of which often don’t have good analogies in the analog world.

    Ultimately I think we will find our jobs as data professionals becoming more and more complex as we must not only solve problems and manage data, but we will also end up being constrained by the legalities governing the data.

    Steve Jones


    The Voice of the DBA Podcasts

    We publish three versions of the podcast each day for you to enjoy.