Voice of the DBA

Tag: T-SQL Tuesday

  • T-SQL Tuesday #138–Technology Changes

    tsqltuesdayThis month the topic for T-SQL Tuesday comes from Andy Leonard, whom I reached out to be a host. I was running low on hosts last year, and Andy agreed to help me out. I had expected something on ETL, but I was surprised with this one.

    Andy asks us about managing technology change. He does mention how SSIS changed as he was writing a book, meaning that things he’d learned while a project was underway no longer applied. That’s a scary place to be, and it’s one I’ve rarely experienced.

    Avoiding Shifts

    One reason I don’t like to test beta software, especially from Microsoft, is that the APIs, the functionality, the effects can change. I prefer to have something fairly well settled before I spend time on it.

    I took a chance early on with Windows 7, when they had the “skydrive”, which become OneDrive. I could sync files from my desktop to laptop, and for someone that travels a lot, this was fantastic. It worked through the beta and until a few years later. At some point it stopped and I moved to Dropbox. I still use DropBox, even though this was reborn as OneDrive and enhanced to allow me to have  personal and business “OneDrive” running at the same time.

    Moving on from Immature Software

    I don’t like to abandon software, and sometimes I can’t. For me, I’ve been involved with some of the products that Redgate Software has built. I often see things early and test them to give feedback to developers. This means I sometimes see things before they work well, or even before the development teams might know what they want to build.

    This has happened a few times, but a couple years ago we were evolving one of our products to expand capabilities for customers. I thought we were moving in a good direction and spend quite a bit of time evaluating how the software worked. I was even starting to incorporate this into some automation pipelines and build demos, when the product direction shifted.

    A fair amount of code I’d written had to be abandoned. I felt as though I’d wasted some time, but as I stopped to think, I realized that I’d been learning and growing in this area. I’d understood how things worked, and while I had to write new code, I was able to write it quicker because I understand the general problem space already. Even though I had to expend other effort, I’m not sure my early work was wasted.

    Fortunately, I didn’t have deadlines for a deliverable. If I did, I might have been more upset. If it happened in the future, while needing to work more to get things done might annoy me, I’d likely understand that the growth and knowledge mean I don’t have to double my investment, and likely I’d be better positioned to write better code the second time around.

  • T-SQL Tuesday #137–Wrap Up

    I hosted the blog party this month, with the invite to write about notebooks. These are a neat technology, and I’ve written about them at SQLServerCentral.

    This post is a wrap-up of the various responses to my invitation. First, quite a few people give credit to either Aaron Nelson or Rob Sewell for their writings and work with notebooks, so check out their blogs.

    With that, Rob’s post this month has a number of links to his notebook work and a list of how he has used them with clients.

    Aaron has kids, lots of them, and they keep him busy. However, he asked his entry this month to be a video where he discusses parameterizing notebooks.

    Rob Farley talks about executing a notebook from Powershell, which to me is one of the key components that makes notebooks usable for something besides teaching.

    Glenn Berry has a fantastic set of diagnostic DMV queries that are incredibly useful for anyone administering a SQL Server instance. He writes about his queries being bundled up in a notebook.

    Chris Johnson was disappointed his organization didn’t adopt notebooks, but found another use. He makes notes to help learn new T-SQL features in notebooks and has a Github repo with them in it.

    Benni De Jagere is rewriting some scripts into notebooks (with the help of PoSh and Bing), starting with extracting Power BI logs into csv files. Hopefully we’ll see him give a presentation on this process sometime in the future.

    Deepthi Goguri writes about how notebooks work, including giving us some magical commands that you might consider adding to your notebooks.

    Barney Lawrence uses notebooks often at work on a regular basis. One of his tasks is a notebook to run git commands for branches. He has also converted a series of blog posts on XML into notebooks that you can use to follow along with his posts.

    Andy Cutler writes about testing the flow of presentations in a notebook. He embeds images near the code to help walk through the process of setting up Synapse.

    Deborah Melkin writes about her challenges getting started, similar to mine. She is using these for some PoSh scripts, but the challenge of running some cells to set up variables and then other ones to do some work. The big win is keeping similar scripts together and documented with text around the code cells. She would like to see some things like execution plans, and I agree.

    Justin Bird has played with them, but not much use in practice. He did some research and learning after the invite and shares some thoughts.

    Julie Koesmarno tells us how she challenged herself to work on Python and microeconomics to build a chart that displays some data. Notebooks help her quickly get hands on and become a better learner. A great use of notebooks.

    Martha Clancy has a few ideas to make notebooks a part of your daily work. Some good tips in there in her humorous flow, which doesn’t sound great, but it shows there is some value in thrashing around a bit.

    Chris Voss tells us the best use he’s found. He puts a number of scripts together with documentation, even  when he can’t run the the code. A colleague can. He also would like to see execution plans in notebooks.

    Mikey Bronowski gives a few scenarios where he has found notebooks to be useful, notably learning and improving processes.

    Hugo Kornelis takes the approach that Microsoft is pushing notebooks a little too much, even in places they may not be appropriate, but he has a few use case: production support, data analysis, and presentations. Another vote for execution plans in notebooks.

    VR expert Todd Kleinhans uses notebooks to work with GPUs, SQL Server on Linux, containers, and RAPIDS. Quite a technical list.

    Linda Kovacheva discovered how powerful notebooks can be with Azure Machine Learning.

    Kevin Chant has been using notebooks for a number of things in the Azure Data Engineering Services at Microsoft.

    That’s the wrap. If I’ve missed anyone, let me know and I’ll update the post.

  • T-SQL Tuesday #137–Notebook Uses

    It’s actually my month to host T-SQL Tuesday, and I came up with the notebook idea last year. Aaron Nelson (b | t) sent me a link to a talk he did at a Meetup. It contained some interesting things, showing how you can use notebooks. Worth a watch.

    For me, I don’t use notebooks a lot in my work. I had thought about using them to show clients and customers how to use our products, but I hadn’t moved far in that direction. I think too many customers are still using SSMS primarily, and haven’t moved to ADS.

    The one place I’ve found them interesting is with Python. When I am sometimes trying to work through an issue, I think it’s easier to run a cell of code and get some results to work through an issue. If I use the REPL, I lose things. In VS Code, which I prefer, I don’t like the split code/terminal. I find that slightly annoying. Notebooks make things easier.

    I was doing this recently, as I tried to work on the Advent of Code. I tried different parts of an algorithm in different cells, just to see what the results were. This helped me to work out some logic.

    2021-04-13 10_02_06-● Day5.ipynb - Data Analysis - Azure Data Studio

    That’s not a great algorithm, and I got better over time, but this was really a good way for me to think about the sections of the problem, putting each on in a separate area.

    It’s a little of “teaching myself” within the construct of a notebook. After all, that’s what a lot of data scientists are doing when they start to go through different code items in a notebook.  They can share the notebook with others, which helps to teach.

    For me, it’s teaching myself.

  • #TSQL2sday #137 Invite Using Notebooks Every Day

    tsqltuesdayIt’s time for the T-SQL Tuesday blog party, and I’m hosting this month. I don’t host too often, but since I had some schedule mix ups, I’ll handle this month. Read the invitation below and then write your post for next Tuesday, Apr 13.

    If you want to host a T-SQL Tuesday, let me know. You can ping me on Twitter.

    Jupyter Notebooks

    I first heard about Jupyter Notebooks years ago. At the time I was just getting started in Python and I thought these were a great way to share code online with others. However, the setup and administration was a pain, and I quickly gave up.

    When I saw Microsoft add notebooks to Azure, I got slightly excited again, but once again, it was a bit of a pain to work with these. Too much administrative overhead, in my opinion.

    That changed a bit with Azure Data Studio, which has seen notebooks get added, and become easier and more stable. If you haven’t tried notebooks, get ADS and give it a try today. I even wrote a short piece on using these.

    Your Invitation

    For this month’s T-SQL Tuesday, I want you to write about how you have used, or would like to use, a Jupyter notebook. This seemed to be exciting for many people at first, but I haven’t seen a lot of uptake from users in general. So I’m curious if you are using them.

    A few resources for you:

    There is even a way to run these from PowerShell.

    So let me know how you are using notebooks, or your plans for the future.

    Publish your post sometimes on 13 Apr, UTC time.

    The Rules

    Publish your post sometime on 13 Apr 2021, UTC time.

    Leave a comment or traceback on this post.

    Use the T-SQL Tuesday logo at the top of this post in your post and link that back to this invitation.

    Spread the word.