Experiments with GitHub CoPilot

There has been a lot of attention given to ChatGPT and AI over the last month or two. I’ve tried a few things with the public interface at Open.ai. Some worked well, like this one:

2023-04-29 09_06_27-

Others not so well:

2023-04-24 10_26_10-Window

This post looks at a few things I tried with VS Code and GitHub Copilot.

Getting Access

I saw a note in our internal Redgate Slack that all developers were given access to Github Copilot. This is something you can subscribe to for about US$10/month, or if you are a student, you can get it for free. In my case, I filed a ticket:

2023-04-29 09_08_52-Inbox - steve.jones@red-gate.com - Outlook

It took a week or so as someone was out on holiday and this was a low priority ticket. In any case, I got a note from our ticket system, as well as GitHub, that I had access:

2023-04-29 09_10_04-Inbox - steve.jones@red-gate.com - Outlook

So I added the extension to VSCode.

2023-04-29 08_58_23-Extension_ GitHub Copilot - sqlsatwebsite - Visual Studio Code

Once installed, I got a note to sign in to GitHub, and when I had completed that, I saw the Copilot icon in the lower right corner of my IDE.

2023-04-29 09_11_04-Window

Getting Started

My first experiment was to open my ZeroDowntime client code and see what happened. This is a VS 2019 project, but I opened it in VSCode, specifically the form1.cs code. I highlighted some code and …

Nothing.

Then I tried something I’d seen. I added a comment above some code. Still nothing, but when I opened the Github Copilot completions panel, I saw this:

2023-04-29 09_02_27-● Form1.cs - sqlsatwebsite - Visual Studio Code

Not helpful, nor what I asked for. Both solutions were similar.

Starting from Scratch

I decided to then start from scratch. I created a new file, set this to C# and wrote this:

2023-04-29 09_14_18-● __ write the outline of a console applic • Untitled-1 - sqlsatwebsite - Visual

Initially I got nothing, but when I opened the Copilot panel I saw this:

2023-04-29 09_14_46-● __ write the outline of a console applic • Untitled-1 - sqlsatwebsite - Visual

That is more interesting. Clearly poor specifications on my part.

Let’s try something else. I added more detail, and as I did, Copilot even added a few things after to help me specify what I needed.

2023-04-29 09_16_06-● __ write the outline of a console applic • Untitled-1 - sqlsatwebsite - Visual

I finished these lines and hit enter and I got something, though not what I wanted.

2023-04-29 09_18_07-● __ write the outline of a console applic • Untitled-1 - sqlsatwebsite - Visual

Let’s try something else:

2023-04-29 09_19_22-● __ write the outline of a console applic • Untitled-1 - sqlsatwebsite - Visual

That’s better. Not great, but better. This doesn’t quite match what I asked for. I copied this to a template project and then added another comment. I want to check for the existence of the parameter, and I got this code, which is better than something I’d write.

2023-04-29 09_30_22-● Program.cs - sqlsatwebsite - Visual Studio Code

Not quite right, and I have some lines to delete, but this gives me something, and as a middling C# dev, this is helpful. Once cleaned, I compiled and ran the code, and it seemed to work, at least for a very basic console app.

2023-04-29 09_37_18-cmd

Let’s try SQL

I opened a new file, set the type to SQL and wrote a comment, using the appropriate comment style.

2023-04-29 09_38_33-● Untitled-1 - sqlsatwebsite - Visual Studio Code

I guess that’s OK. Not great, but I didn’t provide much detail. Interesting that it chose to use a created_at column with a timestamp. I never use timestamp, which is deprecated. Not sure how the AI learned about this type, perhaps because of a large corpus of training data using old code with this? Who knows.

Let’s try something else. I’ll ask for a query using a known schema.

2023-04-29 09_40_29-● Untitled-1 - sqlsatwebsite - Visual Studio Code

A start, but not quite right. I do have to keep hitting enter to get the query written. A couple more Enters and Tabs to accept code get me this:

2023-04-29 09_48_03-● Untitled-1 - sqlsatwebsite - Visual Studio Code

If I keep going, I get a long query, but it doens’t work. At least not on my version of AdventureWorks.

2023-04-29 09_50_45-SQLQuery2.sql - ARISTOTLE.AdventureWorks (ARISTOTLE_Steve (51))_ - Microsoft SQL

Not great.

What if I add a comment? I’ll ask for window functions.

2023-04-29 09_51_45-● Untitled-1 - sqlsatwebsite - Visual Studio Code

I get something else, but again, the final query doesn’t work. This time there are less errors, but the join listed seems to think the Customer table has a first name and last name, which it doesn’t.

Initial Thoughts

I’m not sure how useful this is. I think this is going to be one of those tools that I’ll have to practice with and understand how it works. My basic tests are mostly because I’m not sure what to do with it, or how it can be helpful.

I have lightly seen some demos, but I realize that I need to watch a few more and also experiment with the features. I was hoping it would clean up some of my C# code, which is fairly basic, but it didn’t, at least not with my prompts.

We’ll see how this goes, and I’ll see if I can use it in ADS and if it can actually recognize and use my database schemas to write queries or tests.

About way0utwest

Editor, SQLServerCentral
This entry was posted in Blog and tagged , , , . Bookmark the permalink.

5 Responses to Experiments with GitHub CoPilot

  1. Steve – The answer to Who has the most #1 Billboard hits in teh US wasn’t wrong. While the Beetles (a group) has the most #1 hits in teh US the individual with the most #1 hits is Mariah Carey. If you re-word your question asking “what artist or group” than ChatGpt will tell you that for a group it’s the beetles and for an individual it’s Mariah Carey

    This actually demonstrates a problem I’ve had with Google, knowing exactly how to word your query (for Google) or your question (for an AI) and it’s not always easy to determine as you’ve seen here with what you asked.

    Like

  2. way0utwest says:

    I think when it says 19 hits exceeded 20, it’s wrong.

    knowing how to query is important. It’s why there are jobs for $200-300k out there for AI trainers to prompt well

    Like

    • HAHAHAHAHHAHHA!

      I missed that part until I re-read it just now and that is quite humorous. To me this is the flaw in these bots, they are relying on pulling info from various online sources which they can do but then trying to properly compare the 2 just doesn’t work sometimes. I mean they’re very impressive but there still just a machine.

      That’s funny

      Like

  3. way0utwest says:

    To be fair, that’s what humans do. Also sometimes poorly. Hence all the misinformation and memes out there.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.