Voice of the DBA

Tag: Azure

  • Azure DevOps

    I’ve been a fan of Visualstudio.com and VSTS for some time. I moved most of my demos to this platform a couple years ago and I’ve been pretty happy with it since then. There is tremendous flexibility in how you can use automation and dashboards to build software and coordinate the activities of your team.

    The VSTS system underwent a rebranding and reorgnaization renently. The various pieces of the system were renamed a part of Azure DevOps, with the different parts being given new monikers such as Azure Pipelines, Azure Boards, and more. This was combined with an initiative from Microsoft to better support open source projects by giving them unlimited build minutes for public Github repositories on a variety of platforms such as Windows, OSX, and Linux.

    I’m not a bit fan of name changes, as I think that if the software performs well and provides value, it will succeed. However, marketing people need work, too, and management inside a company often rearranges things to put their own mark on a project. I’m actually glad the marketing effort was ramped up as I think the Microsoft platform based on TFS for tracking work, version control, builds, and releases has become a fantastic platform for anyone building software. That’s not to take away from some other products like Bamboo and Octopus Deploy, which might work better for you. If they do, they plug into the Azure DevOps platform easily.

    If you haven’t tried Azure DevOps, I’d urge you to give it a try. There’s an all day recording of various parts of the system being used to produce software that will show you how to get started and use the system. I’m sure there will be more information and talks this week from Ignite. There aren’t a ton of database tools, but there are some add-ons from various companies to help you build and deploy databases alongside your application software.

    While there are challenges with databases, I’d argue that incorporating a known process will increase reliability and lower risk for making changes. This won’t help you build better code. That’s something you still need to ensure your developers are doing. This system just helps ensure simple, silly mistakes aren’t made and everyone knows exactly how your changes will be deployed to your production environment.

    Give Azure DevOps a try today and see how you can build a smooth, repeatable, reliable process for your software.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • The Hybrid Cloud

    When various vendors started to push the idea of using the cloud for computing infrastructure, the focus was on moving all your systems to a hosted solution. It didn’t matter if you were interested in IaaS, SaaS, or PaaS solutions, there press from salespeople and even some technology professionals was for a complete migration. Put all your infrastructure in AWS, Azure, or some other place and your systems will run great.

    Over time we’ve started to realize that moving to the cloud isn’t as simple as just renting a new box or service and there may be plenty of work to get settled. Even then, there are some limitations or restrictions, either technical or otherwise that might make the cloud a poor choice. Security and compliance are often reasons to not move, but certainly there are also challenges with technology mismatches between on-premises and cloud platforms as well.

    One of the very interesting things that I saw demonstrated a few years ago was the idea of the Azure Stack. The idea behind this technology is that a hybrid cloud can be built, with a single management view. Both the Azure cloud and local resources are combined together and managed as a single entity, with the choice of whether to run workloads on the vendor’s infrastructure or your own on-premises machines. There are other versions of this, for example, with OpenStack working in conjunction with AWS.

    There are many advantages to this, including the ability to limit certain workloads from bursting or moving from one environment to the other. One reason for this might be compliance. An organization might be required to keep some resources off of public networks, for example, the need to ensure that certain data is domiciled in a particular country. The reverse could also be true, with the desire to use a secure public portion of the Azure stack as their system might be certified in ways that your local infrastructure may not be.

    I think the idea of cloud resources, managed, moved, and scaled as needed is a great one. It doesn’t matter if this is all in your data center, all in the cloud, or both. Working this way allows a much more flexible use of any resource, wherever its located. Cutting edge organizations already do a small portion of this with ESX or Hyper-V hypervisors, software configured storage networks, or other software configured resources that can be reallocated on demand. Azure Stack, OpenStack, and other frameworks are a great way to start to leverage the cloud, but in a controlled manner that continues to use any capital investments you’ve made.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.1MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Storage that Expires

    Whether you like the idea of the GDPR (and the new California law), I’m sure you agree that these laws will likely change our data handling in business. Even if they are weakened through legal challenges, many companies have already started to comply and change some of their practices.

    I’ve written about the GDPR plenty of times this year, and I like the law. I hope the law stands strong and resists most challenges. While I’m sure there will be plenty of spurious or silly requests and complaints, I do think these laws are asking for the good data handling practices that most data professionals have advocated for years. These include not only security but also integrity. How often have many of us advocated for corrections to problematic data and been told no? How many times have we complained about security practices?

    One area that I think has been neglected too long in most industries is the area of retention. Most companies I’ve worked for have retained data indefinitely, without any thought or policy. In my mind, we ought to explicitly think about how long we hold data, and remove older data that isn’t needed for our organization’s operation. I feel more strongly about this over time as we find that data beaches become more and more prevalent.

    Azure has started a preview of immutable storage, essentially WORM (Write Once, Read Many) drives as an Azure container. I’ve used WORM storage, but it’s often been viewed as a way of keeping information forever. that can change with this new Azure storage, as you can set a lifecycle management period. The blobs will be removed after this time, which removes one management headache from administrators.

    I could see quite a few uses for this type of storage. If it’s inexpensive enough, what about storing backups here? We could have policies set to remove files after some limited period. I’m sure there are plenty of other uses for storage the is immutable, but also contains lifecycle management options. What creative use would you have for this type of expiring WORM storage?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.5MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • It’s Azure T-SQL Tuesday #103

    tsqltuesdayI’m slightly surprised that there haven’t been more invitations about Azure, with the main development branch of SQL Server being in the Azure cloud, but I’m glad this one came along.

    The invitation is from Björn Peters, and he asks us to write about Azure SQL Database or Managed Instances. We can really write anything that’s a part of the platform, which gives us lots of choices.

    I’ve got two quick items.

    T-SQL Tuesday

    Did you know tsqltuesday.com is hosted on Azure? It’s actually at http://tsqltuesday.azurewebsites.net. When Brent gave me to the domain, and Adam wanted a quick site, I provisioned a WordPress site on Azure in a few minutes. It actually took longer to square away the initial DNS and domain name movement than getting the site up and configured. Templates and known configuration settings help to ensure that you can build a new version of xxx, very quickly.

    The cost was low as well, about $3.60/month and initially $5/month for the hosted MySQL database.

    This changed last year, when the MySQL hosting option was discontinued. I could have upgraded to $15/month, but decided to experiment. Azure has a MySQL as a Service PaaS offering for the same $15/mo, so I moved to that. I was somewhat amazed how easy it was to provision this database on my own and get things running. While the site costs me a little money, it’s a donation to the community, and I’m happy to see so many people participating in the monthly event.

    Quick Labs

    One of the things that I needed to do last year was setup a new cluster and Availability Group for a proof of concept. I have typically done this by getting some VMs ready on my desktop, making connections, etc. However, in this case, I decided to use Azure instead. I was curious about whether this was faster or easier.

    It was.

    Across a few days, as I learned a bit about provisioning new machines and connecting them with networking, I was able to get my set up with 3 machines, a load balancer, storage, etc. I had a working set of Availability Groups, that I could access from anywhere.

    Then I tore it down and started again.

    One of the great things about Azure is that you can use the Azure Resource Manager to provision new services fairly quickly. It’s not as instantaneous as you would like, though it’s not bad. I think the fact that you request actions and then wait until you are notified of something completing in the portal makes the process seem slow.

    In my case, the few days became < 1 hour the second time around.

    Plus, I didn’t use resources on my local machine, where I might have to balance the load with other tasks. I could also shut down my system and reduce costs. With ARM, I could even get rid of the lab and rebuild it in an hour in the future.

    To me, this is the place where the cloud makes lots of sense. If you have a load that is sporadic and can be up or down on demand, then the cloud is helpful. You can remove resources you don’t need, only paying as you go.

    If I were using this every day, I’d probably stick with the local system, but for a periodic use item, it makes sense to fire up the lab when I need it, rather than keeping it alive all the time.

    I look forward to doing more with Azure in the future and learning even more about how to build up the resources I need, when I need them.