Tag: sql server

  • Python in SQL Server

    The Microsoft Data Amp event was this week, with a number of SQL Server announcements. We now know this is going to be called SQL Server 2017, which is no surprise. CTP 2.0 was also released, which includes a few improvements, but one in particular caught my eye. Python code can now run inside SQL Server, meaning we are not limited to the R language for data analysis. That’s exciting to me and I think that’s a great change. It’s one that’s been rumored for some time. I like Python, and think it’s a nice, elegant language. My son has been learning it in college and I’ve spent some time playing with it after the SDTIG got me interested with their beginning Python and Data Science modules. Since then I’ve spent a little time learning R, but find it to be a cumbersome language to work within. Having code embedded in a string in a SQL Server stored procedure made things harder, and I kept hoping that Python would be added.

    It’s there now and you can play with it. While you might debate whether this is a good use of SQL Server resources, I think for quick, short, specific analysis calls, it’s a good choice. Certainly complex calculations and data manipulations are not always efficient in T-SQL and having another option is great. You’ll want to be careful when you run Python or R code, but these are good options to have. There is also GPU processing support is available to take some load off your main CPU.

    CTP 2 is supposedly feature complete and there are more changes than just Python. Tempdb has a better setup experience, we get the first version of graph data and query support, resumable online index rebuilds, adaptive query processing, and more. You can read the announcement, which isn’t heavy on technical details, but gives you an idea of what’s changing. I’m not much of an SSAS person, but there are some enhancements in this platform as well.

    There are other improvements and announcements this week. Azure Analysis Services is generally available (no longer beta), as as Azure Data Lake Analytics and Data Store. DocumentDB is improved and some Cognitive Services are GA, and in general, Microsoft is all in on the analytics and deep learning.

    The world of data in the Microsoft world is growing and changing rapidly. We’ll have a new version this year, just a little over a year after SQL Server 2016. I expect that we will continue to see more, and new changes, occurring at a rapid pace in the future. None of us will completely keep up and become experts in all areas, but we have lots of options for new learning and experimenting with advanced features. I’d encourage you to take a little time here and there and read about and play with the new features. If you have Docker, you can get a version of SQL Server 2017 up and running without installing anything on your machine to see if there are any changes that might help your organization take better advantage of  the data you work with every day.

    Steve Jones

     

  • SQL Server on Linux (Ubuntu)–Updating from Early CTP

    I’ve been testing the SQL Server on Linux for a long time. After the recent Microsoft Data Amp event, I saw the announcement for CTP-2. So I did what I’ve been doing for some time on my Ubuntu machine:

    sudo apt-get update
    
    sudo apt-get install mssql-server

    However, I got a message I didn’t expect.

    2017-04-19 10_59_27-Ubuntu 64-bit SQL Server .210 - VMware Workstation

    A few people on Twitter noted they had updated to CTP 2, but I was still on CTP1.4.

    Hmmm.

    I looked back through the apt-get log and I saw this:

    2017-04-19 11_00_22-Ubuntu 64-bit SQL Server .210 - VMware Workstation

    I’m cutting off part of the path, since I think it’s probably NDA. No worries, apparently the old location for me hasn’t been updated with new packages, which makes sense.

    I decided to check the MS docs and see how a new user would get SSoL running? At the new docs.microsoft site, I found the Install SQL Server on Ubuntu doc.

    Following the instructions, I updated the GPG keys and registered the repository with curl:

    curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
    
    curl https://packages.microsoft.com/config/ubuntu/16.04/mssql-server.list | sudo tee /etc/apt/sources.list.d/mssql-server.list

    Once those were done, I re-ran my apt-get update/install and it worked.

    2017-04-19 10_58_46-SQLQuery9.sql - 192.168.1.210.master (sa (51))_ - Microsoft SQL Server Managemen

    If you haven’t worked with Linux, it’s not really that hard. Some of you may get asked to install or manage a system, and while you should seek help for anything production related, in dev areas, play around. Read and learn a bit about Linux (I’d recommend watching the SDTIG book club meetings I, II, III, IV on Linux), download Ubuntu (or anything) and play around. Get used to a command line and try things.

    You’ll love SQL Server on Linux. It’s mostly just SQL Server, but updates and installs as simple in a test world.

  • SQL Server Telemetry

    One of the things that I’ve seen blogged about and noted in the news over the last year is the amount of data being collected by various software systems. In particular, I’ve seen numerous complaints and concerns over what data Microsoft collects with its platforms. Perhaps this is because Microsoft is the largest vendor, and plenty of other software collects usage information, primarily to determine if features are being used or working correctly. I think much of this is overblown, but I can understand having sensitivity about our computer usage, especially for home operating systems.

    Microsoft conducted an extensive review and published data about what is being collected and why. Windows 10 and other products have undergone review and must comply with the published policies. There’s even a Microsoft Privacy site to address concerns and explain things. It’s an easy to read policy that explains what  Microsoft is collectin, if you’re connected to the Internet (if you’re not, don’t worry, no bloating files). That’s a huge step forward in an area that is evolving and something I wouldn’t have expected to see in the past. I am glad Microsoft is making strides here, even if I may not agree with specific items in their policies. I do think that most of the companies collecting this data are doing so to improve the products, not spy on customers. I’m sure some do, but likely smaller organizations with some sort of criminal intent.

    As data becomes more important, telemetry for software is potentially a data leakage vector where private, personal, or customer information might be leaked. Certainly as more speech and other customized services are used in businesses, I worry about what data could be accidentally disclosed. After all, it’s not that super powerful smart phone that is actually converting audio to text in many cases; it’s a computer somewhere in the vendor’s cloud.

    With databases, this has also been a concern from some people. I’ve seen the Customer Experience Improvement Program for years and usually opted in. I’m rarely doing something sensitive and I hope that with more data, Microsoft improves the platform. That’s the stated goal, and I’d seen them talk about this a few times. The SQL Server has moved forward and published an explicit policy that spells out what and when data is collected. It was actually just updated recently and all new versions of the platform must provide this information (if anything is different) and adhere to what they disclose. There is a chance that user data could leak into a crash dump, though users have the opportunity to review data before it is sent to Microsoft. I’m not sure how many will, but they have the chance.

    I would like to be sure that anything sent is secured, and perhaps have an easy way to audit the data sent in a session, but I know this entire process is evolving. One important item to note is that customers can opt-out of data collection for any paid for versions of SQL Server. That’s entirely fair, but if you have regulatory concerns, you should be sure that you don’t restore production data to development machines. You shouldn’t anyway, but just an FYI.

    Usage data is going to be a part of the future of software, especially as more “services” integrate into what we think of as software. Those services aren’t always going to be under our control and certainly part of the reason many of these are inexpensive is that additional data is captured about the people using the software. I hope all companies publish and adhere to some sort of privacy statement, and maybe that’s a good start. Rather than any regulation on specific privacy that must exist, start forcing companies to publish and stick to whatever policy they choose.

    Steve Jones

     

  • T-SQL Tuesday #89–Changing Times

    tsqltuesdayThis month the invitation is from Koen Verbeeck and it has to do with the cloud changes coming to the data world, especially SQL Server. That’s an interesting topic, not necessarily technical, but it is an interesting one to think about.

    I’m going to be a little short because life is busy, but I think this is an interesting item to keep an eye on for many people. Not necessarily at your current position, but what if you need to move on?

    If you’re interested, I host all the T-SQL Tuesday topics at tsqltuesday.com.

    The Cloud is Changing Things

    I sat in a talk at SQL Bits from Conor Cunningham of Microsoft. He’s one of the principal architects of SQL Server and Azure SQL. In the talk he talked about some interesting ideas in how SQL Server engineering has changed in the last decade. Victoria Holt wrote a short piece on some of the things Conor talked about.

    There are a couple interesting things that the cloud is enabling. First, Microsoft runs their cloud without any Ops team, really having developers be responsible for things in production. This is 1.7mm databases, without any DBAs. Why? They gather lots of data, so they learn when things are broken, unstable, or problematic. They do this with the 600TB of telemetry they gather every day.

    Of course, you and I won’t have that much information, but the cloud does enable Microsoft to think about how to make SQL Server more stable, and also how to add automation capabilities into the product. We haven’t seen much of this change in current versions, but the Query Store is the start of one thing, and Adaptive Query Processing (coming in v.Next) is another. I wouldn’t be surprised to see more, and that means our jobs as DBAs will change.

    I think there will be less, check, configure, verify, maybe even some less tuning work for DBAs. There will always be developer needs, especially with more complex reporting, visualizations, and just understanding large data sets. There will also constantly be the need to write better SQL as the optimizer can only do so much with bad queries.

    The cloud interests me and excites me. There are issues, concerns, and challenges. However, I also find working with Azure through Powershell, being able to access different services from various places, keeping some data there (non PII) and avoiding the need to manage infrastructure to be key.

    I don’t know if it will happen, but I would hope at some point that the Azure cloud, the AWS cloud, the Google cloud would license their service, or even allow others to resell and manage portions to encourage competition and give us some choice in who we might choose to deal with. If so, they I could see more and more companies just considering moves to the cloud for more data, especially when there could be different levels of service and protection for different needs.

    The cloud is changing things, even if you aren’t in the cloud. That can be opportunity if you take advantage of it.