Voice of the DBA

Tag: syndicated

  • Do You Have a SQL Server Estate?

    At Redgate, we do a lot of research. Maybe not as much as Microsoft Research, but we are always trying to investigate more about customer problems and challenges before we build or change software.

    Take our survey on SQL Server Configuration Management and you could win prizes. Or a prize.

    I’ve never thought about managing an estate, but not that I live on a horse ranch, that has a little more meaning. In the past, I’ve managed hundreds of instances and thousands of databases. That’s certainly an estate, but even when I had 5 servers, that is something of a small estate of things to keep me busy.

    At Redgate, we know there are lots of challenges in managing SQL Servers. We constantly have a team of people working on SQL Monitor, and they are releasing changes every week or two. In fact, we just released v9.0 this year, full of enhancements customers have asked for.

    In addition to the tactical team that’s building and fixing things day to day, we are doing future research into where to take our monitoring and tuning software. That’s some of what this survey is about. Let us know about which things might help you smooth the process of managing systems.

    Give us a few minutes and a little information and take our survey. You might find that a future version of SQL Monitor contains something you thought would be useful.

    Take the Survey today.

  • Don’t Leave the Database Behind…Or Win If You Do

    If you’ve had struggles with database development and had a real world disaster, you could win in a new Redgate contest. We’re looking for the places where old style, waterfall, traditional database development has been a problem. Especially when application software developers have been adopting DevOps, small, rapid release strategies.

    Enter before Mar 20 and give us your story.  Don’t worry, we’ll anonymize your name and company details. You can enter as many times as you like, but can only win once.

    My Struggles

    I started working with a small startup company that had 8-10 application developers. We tried to work in a rapid format, releasing a new version of our website every 2-3 weeks. This was going OK, but not great. Often we’d release and developers would go to test their new feature, only to realize that they’d forgotten a database component.

    They would roll forward, finding code and deploying a database change during the release process, usually getting the feature working in 10 or 15 minutes.

    Until they broke something else because they hadn’t really tested the database component complete. Or they realized they needed yet another database change. Or they had forgotten to code for some edge case that another developer tried during the release smoke test.

    When I started, we all gathered in the office on a Sunday night and spent an hour or two releasing code. Not one of my favorite times.

    The app developers were using source control, and trying to test, but they often sent code to QA and then fixed things in QA, forgetting that they needed to ensure those changes were deployed again to production.

    Across a few months, we implemented what people would not call a DevOps process, reducing our release time to a few minutes, with myself and another developer on the phone, releasing automated changes from a CLI.

    Database DevOps makes everyone happier.

    Share your story before Mar 20 and good luck.

  • Practical Web Scraping–Getting Started

    As part of my learning goals for 2018, I wanted to work through various books. This is part of  my work with Python.

    After going through a few first chapters, I decided to start my February learning with Practice Web Scraping for Data Science, which looks at data acquisition using Python to pull data from the web. I found the book interesting and also this would be a nice setup for two of the other books (Power BI and natural language processing).

    Like many people, I find lots of data on the web, but I’m constantly struggling to get it into a database. I find myself going through gyrations at times to get data. Even with the cool features of Power BI, it hasn’t been as smooth as I’d like to get data, so I thought this would be a good book.

    Part 1

    The first few chapters of the book are about the basics of web scraping. We learn what this means, and get a little bit of a tutorial on who uses this technique, with some specific examples. We also get a basic python tutorial, which I skimmed. I know a bit about Python and this was a very basic, getting started.

    The next part of the early book deals with the basics of what http transport looks like and how some networking works. This is interesting to me, though not sure it matters for scraping. We’ll see. There is some discussion on GET and the http standard, so perhaps that’s helpful. It is good to at least know what codes might come back or what headers or parameters you need to use.

    The third chapter starts to get code working.  It opens with a discussion of HTML and how you can examine the structure of pages in your browser. This is a good reminder and basic tutorial of some of the web page developer tools that exist in your browser and that you might want to use when trying to build applications, especially those that scrape pages. There is also a basic CSS tutorial, which was good as I needed a little refresher. I rarely deal with CSS stuff, leaving that to others.

    The last part of the chapter starts with the BeautifulSoup library, which is built to parse out text, and specifically, makes working with HTML easier. The examples are with a Wikipedia Game of Thrones page, but I added some examples, trying to translate this to a sports page. It worked OK, and I learned a few things.

    The last part looks at Regular Expressions with BeautifulSoup and how you can search out elements and then start to copy data. It’s more complex and tedious, but then again, lots of programming is tedious. Once it’s working, it’s amazing.

    Experimenting

    I started to work with this in Azure Notebooks as a different way of tracking some work in Python. I’ll want to store things in a file at some point, but for now, this lets me start and stop learning and keep track of where I am without worrying about files and names.

    Not sure if anyone can access it (it’s marked public), but my project and notebooks are here: https://notebooks.azure.com/way0utwest/projects/web-scraping-with-python

    I ran some of the early scripts, which are just getting you used to working with Python and accessing web pages. I then copied some examples from my Calibre view of the book and executed them. I even tried to experiment a bit.

    One note: copying the code seems to leave some invalid character in there for Azure Notebooks, so I ended up editing the beginning of every line to remove the offensive character.

    This got me the basics of working with web scraping. Now to try and grab some data from another page and see what I’ve learned.

  • Learn about the State of Database DevOps Next Week

    It’s a week to my webinar with Donovan Brown (b t, y g), leader of the League of Extraordinary Cloud DevOps Advocates, Principal DevOps Manager at Microsoft, and the guy that wants to Rub DevOps on Everything. He’s a passionate, intelligent, exciting guy whose enthusiasm is infectious. I’ve been honored to present with him at Build and I’m looking forward to our chat next week.

    Join us next Thursday for our webinar on the 2019 State of Database DevOps report. You can register now and we’ll be live at 11am EST.

    The 2019 State of Database DevOps report is available now and you can download it today. We’ll be analyzing some of the findings and discussing what this means for many organizations.

    Register today and I’ll see you next week.