Tag: databases

Hybrid Databases

SQL Server already handles hybrid data.

We are dealing with more and more data all the time, and in more and more formats. As noted in this piece, companies are receiving data in new formats, from a variety of sources, and the traditional relational database isn’t always the best fit for the processing, storage, and management of the information.

However SQL Server isn’t just a relational database. It handles unstructured data well, with the Filestream and Filetable enhancements to varbinary columns. We can even read into this unstructured data with iFilters that we install on our instances. In fact, I’ll be talking about these filters at DevConnections on Wednesday.

SQL Server has StreamInsight for complex event processing, allowing you to handle very quick streams of data that could overwhelm your OLTP architecture. If that’s not enough, there are connectors that allow you to integrate with other data processing architectures, like Hadoop.

With all the various subsystems like SSIS (ETL), Reporting Services (exports to other formats), Service Broker (messaging), spatial data, and more, I think SQL Server is an amazing hybrid database platform in and of itself, without any extra charges for the various features (like some other platforms). That’s not to say you shouldn’t use any other platfoms. If they work, use them and integrate with SQL Server, but if you don’t have other databases in place, I’m not sure you need to look past SQL Server for many of your needs.

September 30, 2013
Best Practices for Database Deployment
The easiest part of any deployment is running the deployment script on production. But what have you done to prepare that script for production? Was the script tested? Is the script tightly coupled with the application code?

The difference between a successful deployment and a failed or problematic deployment is all the preparation work you’ve done to ensure that you’re deploying the right script, the right way to your production environment. This session will explore best practices that you can implement to ensure that your deployments are as successful as they can be.

Slides:
- SQL in the City – Best Practices for Database Deployments_US.pptx
September 27, 2013
Automated Build and Test
Continuous Integration (CI) is a well known process in the software development world, but it’s not often implemented with databases. This is despite the fact that database code could benefit from the practice of checking the changes on a regular basis for potential issues. This talk will show how to implement continuous integration for databases and include more realistic testing of the code for potential issues.

We will set up CI for a SQL Server database project, using the following tools:
- Team City (CI Server)
- Subversion (VCS)
- SQL Automation Pack (database -> CI integration)
- tSQLt testing framework
- SQL Data Generator
Slides
- DevConnections 2014 – CI for Databases.pptx
- SQL Saturday #300 – Kansas City – CI for Databases.pptx
- SQL in the City – SITC13_Sacramento_CI.pptx
- SQL Intersection Spring 2014 – Automated Build and Test Intersection.pptx
September 27, 2013

Citizen Scientists

The tremendous growth of computing power, especially in the form of mobile devices, means that more and more people can analyze data themselves, or have applications do it locally, reducing the need for large central systems to perform all the calculations. Indeed some of the value from Hadoop and other cluster computing systems comes from using lots of small, relatively powerful systems to do more work than one single computer could.

However we can not only analyze, but also gather data. There is a rise in the citizen scientist, in which individuals can help scientists by gathering data and sharing it with others. This can reduce the costs of gathering data, but more importantly, gather much more, and varied, data than might otherwise be possible. There are a number of projects available if you’re an amateur scientist that wants to join in.

Most of us are data professionals and might be interested in working with more data. Perhaps we ant a set for demo purposes, but perhaps we want to work on a project that interests us and could use our technical talents. Many of the projects out there make their data available, and you should consider using such a set if it strikes your fancy.

More importantly, I would hope that more people would analyze the raw data and look to verify, or dispute, the conclusions of the project owners. As data professionals, we can help people learn how they can analyze and manipulate data better with the tools we have available. With more transparency, we can have healthier debates on the causes, the effects, and even the potential future implications of how to build a better world in the future.

Steve Jones

Video and Audio versions

Today’s podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.