Voice of the DBA

Author: way0utwest

Python Jupyter Notebooks in Azure

There’s a new feature in Azure, and I stumbled on it when someone posted a link on Twitter. Apologies, I can’t remember who, but I did click on the Azure Notebooks link and was intrigued. I’ve gotten Jupyter notebooks running on my local laptop, but these are often just on one machine. Having a place to share a notebook in the cloud is cool.

Once I clicked on the link, I found these are both R and Python notebooks, as well as F#. These allow you to essentially build a page of code and share it. It’s kind of like a REPL, kind of like a story. It’s a neat way of working through a problem. I clicked the Get Started link to get going and was prompted for a User ID.

Once I had my moniker setup, the next step was to edit my profile. That’s more important than a library, right?

Of course, I needed to fill out the profile with my avatar and some information.

Next I need to create a library, which I’m guessing is a collection of notebooks. I clicked the link and had to enter a name. I decided on the classic HelloWorld name. I decided to keep this public, as I might want to share this with others.

I’ve got a library, now let’s add something. I clicked the Readme.md, but it didn’t load. There was nothing there, as this is a blank file.

I discovered I could right click the file in the list. This lets me edit it. Strange UX, but whatever. The file uses markdown as editing, which is fairly simple, but consists of a few characters to designate titles, lists, etc.

I entered some text, and then my readme appeared below my notebook list, much like it goes on Github. My screenshot got taken after I’d experimented a bit, so you see a couple python notebooks as well.

From there, I could add a notebook. I have choices. I started with Python, since that’s one of my learning goals.

I give the notebook a name and create it.

Once this is created, it appears as a Jupyter notebook. Essentially I have a repl-like command area, and once I enter code, I can click “Run” to execute it. You can see that my Hello, World program ran.

I can enter other code, and I’ve done a few things, just to practice some basics in Python. I’m working through some courses, and I’ll enter code in here to practice concepts.

Jupyter notebooks are a good way of working through a problem and showing flow. They’re especially useful for sharing information with others and letting them follow your thought process.

April 18, 2018
Good Security Needs Layers

How many of you have wanted to know who started or stopped a SQL instance? Probably a few of you, as disruption to the service can affect customers. Most of us are concerned with the changes made inside SQL Server to objects and data, and that’s what the auditing features inside SQL Server are watching. The problem is that the database platform is dependent on the host OS, and as such, some actions take place at that level. Auditing inside SQL Server isn’t setup to capture this information.

Should you care? Well, restarts, or the stopping or a service are one way that a malicious actor could alter files, change the error log without you realizing it, or even copy files to other systems. All these actions might be outside of any auditing or event tracing you’ve set up. Good security needs multiple layers because the system you need to protect is often dependent on some other part of our infrastructure.

Databases depend on the host OS and perhaps directory services. Your OS may depend on a hypervisor, and certainly needs patching, so it depends on human administrators. Many of our systems depend on networking and firewall configurations. There are other layers, but the more that we can ensure each layer is secure, the better off we are. Certainly our systems always depend on humans not giving away credentials or installing malware, but that is often something many of us can’t control.

I ran across an article that explains how to use auditing at the Windows level to track this down, and ensure that there aren’t more unexplained restarts. You can implement this, but if you don’t have Windows administrative privileges, you’ll need to get help from someone that does. Likely a couple of you have been glad that there isn’t a great way to audit this from the OS, as you were the one performing a restart without permission. If that’s your MO, I expect you might not want to pass this piece along to your security staff or auditors. If that’s the way you work, though, I would advise you to change your habits.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 3.2MB) podcast or subscribe to the feed at iTunes and Libsyn.

April 18, 2018
An Open Letter to PASS and John Q Martin
Dear Mr. Martin and the PASS Organization,

In the spirit of advocating for change in a public way, I’m writing this letter to ask you to revisit your policies and processes and embrace the transparency that we, members of the PASS organization, deserve with regards to the SQL Saturday events.

I am directing this to you, Mr. Martin, after seeing your tweet, with the hope that you will follow through and not only provide feedback, but conduct a root cause analysis that updates future decisions. I also am hoping these are not merely words, but also the beginning of some action that you take and about which you publicly disclose the results.

A change was recently made to the SQL Saturday site that requires speakers to register for an event before their submissions can be approved. I realized this when a fellow speaker posted a note on Twitter. While there are potential implications here that are disruptive for event organizers and speakers, those are worthy of a separate debate. In fact, they are worthy of debate.

Today, I examined the PASS blog to look for an announcement. This what I saw.

The last update is from Mr. Fritchey, noting how much PASS wishes to express their affection for SQL Saturday. I appreciate the Board of Directors, most of whom are fellow speakers, may feel that way, but I’m not sure the organization believes this. Again, a separate discussion.

Early on in the life of SQL Saturday, a few people had to make decisions, but those decisions were publicized, with conversations with organizers and speakers. When fundamental changes were made, we announced changes publicly to ensure that others were informed. We setup a board of advisors to help ensure that change was made in a more open way, and to solicit feedback. Andy continued to provide updates later, even as PASS has declined to do so.

That doesn’t seem to have been done any longer. As I scan through the blog, I see no announcements of changes to SQL Saturday. The last posts were from 2016, when the 600 mile radius was announced and then a subsequent post explaining the reasoning.

Once again, almost two years later, PASS lacks transparency and vision. There are no discussions on slack that I find, NDA or otherwise, no emails I’ve received, no list of changes or updates.

What I would request is that you use this as an opportunity to improve governance at PASS. Examine the timeline of when this decision was made, the timeline, and look for opportunities to engage, or at least inform, the membership of PASS of changes. There should be milestones that require
- an announcement of potential changes, NDA or public
- a roadmap – is there a public one? Even one without dates.
- a roadmap update
- an announcement of an upcoming change
- an announcement of the change, publicly
- an update to the FAQ
I appreciate that there may not be funds for future changes, though there are obviously some still in progress. However, this is separate from funding. This is a process and governance issue that should be addressed.

It’s also simple common courtesy.

Sincerely,

Steve Jones

PASS member since 1999
April 18, 2018
Machine Learning Challenges

If you know someone well, you may think that you can predict what they’ll do in a situation. Those that know me might think that if I go to New York, I’ll make time to run in Central Park. That’s a good bet, and the last 7 or 8 times I’ve gone there to speak, that’s what I’ve done. However, that’s a very small data set, and you might not want to attach a high level of confidence to that prediction. There are certainly times that I’ve gone to New York and not run in Central Park.

Machine Learning (ML) is a field that tries to make predictions about some data, based on previous known actions or results. By inferring relationships between lots of data and known actions, the ML model can guess what might happen with future data inputs. Lots of companies and fields are experimenting with ML techniques, with some of them working quite well. Others aren’t faring as well, and there is a lot of research underway to better understand the entire field. One possible explanation is emergence, which is discussed in this piece.

If it’s raining, I likely won’t run. If it’s 20F, but dry, I will. Those that know me would inherently consider the type of weather as a new factor that has a high level of predictability for me. Time in the city, location, whether my family is with me, these are all other factors that would come into play. With enough data, and enough factors, you could likely come up with a model that predicted the likelihood of me running in Central Park when I visit New York City. It might not be perfect, but it would be good if you had enough data.

However, that’s a very simple item, and few decision points. Would that work for a larger data set? I bet it would. If we were able to collect data from lots of 5k events, we could perhaps predict how many people would register show up. There are certainly more factors at play, and since we can’t control things like weather, the prediction might not be that valuable. Though if we could change the amount of resources the day of the event, maybe that would help reduce costs or make the event more successful.

I think ML is a valuable tool that can help with quite a few problems, but in many cases, we just don’t know enough about the complex factors, such as emergence, that might impact us. I know our SQL Saturday organizers would love a model that helped them decide how much food or drinks to order for an event. My view is that there are likely too many unusual factors, but I wonder if we could start a data gathering project to come up with a model. Perhaps we should gather marketing metrics, registration metrics, and more, with the intention to predict the number of no-shows. Even if it wasn’t useful, it might be a fun project for people in this industry.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

April 17, 2018