Voice of the DBA

Category: Editorial

  • Should We Move to Azure?

    This editorial was originally published on May 23, 2014. It is being re-published as Steve is out of the office, with a few minor editorial changes.

    Update: We made a number of changes last year, upgraded the forums, and have continued to implement bug fixes. We are still considering this idea.

    At SQLServerCentral we have been debating some architectural changes to the site. We’d like to improve the way a few of our features work, add new ones, and replace a few items that are buggy and hard to maintain. The site has worked well over the years, but we’ve made relatively few improvements, mainly due to the fact that like many companies, this website isn’t our business, and work has a lower priority than some other areas in the company.

    Recently I had a suggestion presented to me and I wasn’t sure how I felt. There were certainly pros and cons to both taking advantage of the opportunity as well as passing it up. While we are still talking about the process, I wanted to ask the community what they might think this week.

    Should we move SQLServerCentral to Azure?

    I really like the idea of PaaS. There are issues, and certainly I think Microsoft could do a better job of creating competition as well as securing and even deploying their platform. However I think the idea of just consuming services as opposed to installing and managing pieces of software, is a great direction in which to move. SQLServerCentral would seem to be a perfect type of application that fits in the cloud.

    We run a generic ASP.NET site, using fairly common and simple database features in SQL Server. We don’t store or manage much in the way of PII, financial, or medical data outside of email addresses, and we don’t have the need to scale to high levels, either in data storage or computational resources. The one problematic area is our email requirements. I’m not sure what the cost is to send emails from Azure, but we send millions per week, and CAN-SPAM requirements can be tricky to meet in the cloud.

    However I’m just wondering what the community thinks. If we successfully moved to the cloud, and disclosed details about the process, would that give you more confidence in the cloud? Would it be interesting? Or is SQLServerCentral just a service and you don’t care how it works, as long as it does. Let us know this week.

    Steve Jones

  • Physical or Virtual Storage

    When I started working with SQL Server, every server had what we’d call das-dee, or DASD (Direct Attached Storage Devices). These were hard drives inside the same physical case as the rest of the Windows computer. I’ve added lots of drives to various server systems over the years. As databases grew, we even had separate boxes in our racks that were attached to the main server, but only filled with drives.

    Technology has changed, and today most of us work with SAN or NAS devices, where the storage is addressed across some type of network. Either a private one (copper or fiber), or the same Ethernet that connects the various computers together. A few of us might even have cloud storage that is located at Microsoft, Amazon, or elsewhere. The Stretch Database feature takes advantage of this last configuration. In all these cases, the storage that our databases see is often cobbled together from other disks that hide the underlying organization from the system.

    Recently I read a piece from Randolph West that talked about recovering data from a RAID array. That reminded me of my early career, where I had to make decisions about how to structure storage. I’ve run RAID 1, 5, 10, 0+1, 6, and maybe more in my career to store data files. However, at some point I stopped worrying about the underlying configuration. I just expected, and trusted, the storage people to ensure that space was available. I even stopped thinking of the z: or y: drives on my database server as disks. Those drives were just storage that existed somewhere in the ether, just available for the database to use.

    In thinking about Randolph’s experiences, I wondered how many of you out there might still deal with physical drives. Do you still make decisions about RAID levels? Do you even know what RAID levels are being used by your databases? If you’re a storage admin, you might, but for those of you that aren’t, do you know anything about your storage configuration?

    Really, I’m speaking of production systems, not development ones. Certainly many of us might know there’s a development server with RAID 5 that holds a bunch of dev/test VMs, but I would expect that might even be rare. Outside of our own workstation, we likely don’t know the storage setup. Plenty of development systems these days probably even use a SAN, maybe even the same one as production, for storage.

    For me, I have no idea of our systems. I used to build the SQLServerCentral servers, and when Redgate took over that part of the business, I helped spec the initial machines we rented as physical hosts. At some point we moved to virtual machines, and while I was asked about the specifications, I didn’t care about any of the hardware. I just said that I wanted enough CPU, RAM, space, and IOPS to handle the load. Deciding what that was, and ensuring it was available, was someone else’s job.

    If you spec hardware, or pay attention, let me know. There certainly are plenty of hardware geeks, like Glenn Berry, that pay attention and prefer particular configurations. Those are the people I’m glad I can ask for advice if need it. I certainly ask for help with my personal systems, but for servers, I just need capacity. Do you feel the same way?

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.7MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Unstructured Data

    Is unstructured data a bad term? I saw some data professionals complaining about this, saying all data is structured. That’s usually true. A CSV, even a ragged one has structure. XML and JSON have structure, even if it might vary node to node. Certainly our relational tables are structured and some formats can be rigidly mandated between organizations (like EDI). Even data in PDF, Word, MP3, MP4 or other audio/video mediums is structured in that we know the format.

    Given that, is it a misnomer to use the term, unstructured, when describing flexible formats, such as XML? Is it OK for a PDF? I have had a presentation called Unstructured Data in SQL Server. This is primarily about FileStream, FileTable, and searching those objects. In the talk, I classify data in known formats as structured. These would be SQL Server tables and similar objects. At any point in time, we know what all data in the table looks like, even though we can have NULLs or missing data in rows.

    I call XML and JSON semi-structured formats. We can certainly determine the format for any node or section, but we wouldn’t know without querying or examining the data. It’s semi-structured in that there is a hierarchy, but the structured from section to section (essentially row to row) can vary. There can even be depths to hierarchies that vary. In many ways, that makes these great formats for flexibility in data exchange.

    I tend to view data in Word, PDF, MP4s, as unstructured. We don’t necessarily know where the data is, or how to separate it. We can get pages in Word or PDF, but those can vary and don’t necessarily help us extract information. They are XML, but the XML tags don’t relate to the content, unlike many other XML documents. Scenes or tracks in audio/video files might be separators, but those aren’t necessarily helpful in gathering information. Instead, we need other tools that can help deal with that data, finding words, concepts, or more inside of the binary stream.

    I like the term unstructured data because it helps me understand where the information is. While the tables in a database might be full of nonsensical information in some rows, or be poorly designed with data combined into text fields, at least I know where the fields are. Actually, in that case, I’d argue the data in varchar(max) text fields is really unstructured. You might disagree, but give me a better term to describe there the information is stored in a data format.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.7MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Gigging for a Career

    There are some people that like working for an organization, sacrificing some compensation and flexibility for stability and security. Others enjoy the chance to experience constantly changing environments and a rich variety of projects at the expense of regularly searching for new work. Neither is necessarily better or worse than the other, and these aren’t polar opposites. In the real world, each includes some of the advantages and disadvantages of the other.

    For many of us that work with data, we realize there are some advantages to understanding the business meanings and implications of the information derived from data. Indeed, those that tend to work with the same types of data become fluent and comfortable manipulating and discussing how analysis, transformation, and more relate to that type of data. This leads me to think that someone that works for a company or stays within a particular industry might be preferred as a data professional for organizations in that space. Maybe this is someone that an organization wants to hire and retain over time.

    I ran across a piece that discusses the Gig Economy from the perspective of data professionals. Their view is that many data professionals would rather work on interesting projects, and analyze data that is interesting to them, rather than being stuck with a single organization. Certainly from the organization’s view, having very skilled professionals available for project work means less costs in training, benefits, hiring, etc. While the cost per day might be high, there is no need for an ongoing, or at least not a constant, commitment.

    I do think that many of the changes in technology have made it possible for talented workers to find plenty to keep them busy and earn a very good living. However, there’s a cost in spending time looking for projects on a regular basis. It takes not only time, but mental strength and a desire to be a bit of a salesperson and marketing professional. Some people may find long term clients that call them over and over, which isn’t that different from working for an organization unless you can interleave a number of clients together on a regular basis. However, most people that work at “gigs” are regularly spending time looking for work.

    I would postulate that most of us prefer some amount of security and stability, preferring to work for an organization for a period of time. While we may change jobs at times, it’s often at a lower pace than those that might prefer to work in the Gig Economy

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 3.6MB) podcast or subscribe to the feed at iTunes and Libsyn.