Voice of the DBA

Category: Editorial

  • Relational By Default

    I’ve read a few pieces from Vladimir Khorikov and I tend to like the thoughtful way in which he approaches building software. When I saw a link to his SQL v NoSQL databases, I was intrigued to see what his thoughts would be. It’s a good read, though with relatively few comments or debates posted at the time I read the article. I was hoping for more and I encourage you to add your thoughts.

    The main contentions in the piece are that any of the simpler development techniques that we can use with NoSQL databases don’t remove the need for implementing data quality or management features. They just require those features to be implemented by the developer. Specifically the need to deal with multiple schema versions over time, or the requirement we enforce parent child relationships.

    Perhaps that’s not too bad. After all, if we evolve schemas and the code can easily deal with the changes, that’s good. However I think the point about having relational storage when you can, and adding in other stores makes sense. This is what I really believe as well, since we often need to query and combine data, which is something relational stores do very well and very efficiently.

    Ultimately I’m sure we will continue to see arguments about relational and NoSQL models being better or worse for particular problems. I actually welcome the arguments  because there are certainly domains of problems where one system works better than the other, and it’s good to develop some clarity about those particular problems. However there are also likely many situations where multiple platforms can work, and in those cases, we should use what works well for our team.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 1.9MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Private Data

    I think that as we evolve into a more digital world, we really need to modify and enhance the various legal systems around the world to cope with the challenges of digital information. The world changes when vast troves of information can be gathered, indexed, easily maintained, queried, and copied without anyone being aware, or few people understanding how any particular data is used. Data fundamentally is different in the digital world precisely because the costs and barriers to its movement are so low.

    I’ve been concerned with this for a few reasons. One is that I like my personal privacy and would like to ensure that data collected about me and my family is something I have some say in. Or at least some understanding. However as a data professional, I also have concerns about the responsibilities and potentials liabilities of managing data in the future. There’s also the not-so-little concern about employers pressuring employees to deal with data in a way that might conflict with their personal ethics, or even the local laws.

    I’m glad someone at Microsoft is taking a stance, asking the US and EU to recognize the privacy of digital data as a right we have as individuals and corporations. The request asks that governments treat digital data like they treat analog data, serving subpoenas and warrants to the owners of the data, not the custodians. While this isn’t what always happens in the real world, we certainly should have more protections for digital data than we have now.

    I’d like to see governments amend their laws to also exclude IT workers from liability in working with data that they are custodians of. We often don’t make the decisions about what data to gather or how it should be managed and moved. We just implement the decisions, often under coercion from our employers. Liability should rest with those employers, not those the employers have hired.

    I don’t have hope that things will change anytime soon, but I hold out hope they will at some point.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Rogue Software Changes

    Could a group of software developers make changes that fundamentally alter the way a software system should work without management being aware?

    That’s the question being asked of VW right now. Most people are skeptical, but I ran across a piece that wants to lend credence to the idea that a few software engineers acted with few people being aware. They did this, not because they wanted to defraud everyone, but they wanted a solve a problem that they couldn’t do in other ways. They also didn’t see the alteration of test results as much of an issue because they thought the tests were too stringent.

    I’m not sure I believe that’s what happened. Certainly there is some disagreement from various parties, but with my experience in software projects, management always wants to know how things are proceeding, with more and more questions whenever the applications don’t work as expected. When problems are solved, natural human curiosity leads more managers to ask for details, even when they don’t understand. In this case, I can’t imagine lots of VW management weren’t aware that software was being used to pass tests. Many people report to many others, and everyone would have wanted to know how VW solved this engineering problem.

    The stakes for organizations will continue to rise in a global economy, and software will play increasing roles in many companies. Will we see more and more pressure to manipulate our world with software, even in criminal ways? I suspect so, and I sympathize with those that might face the loss of employment for not complying with the requirements they’re given.

    Ultimately I think transparency of software is the best way to bring about better software that complies with regulations and rules. Transparency also ensures that copyrights aren’t violated (since violators code is available), and we can determine if security is being built into systems. Perhaps best of all, developers can all learn from each other, seeing exactly what works and doesn’t in each system.

    I doubt we’ll get there, but transparency would be a nice place to be.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.7MB) podcast or subscribe to the feed at iTunes and LibSyn.

  • Custom Schemas

    One of the things I seem to see more and more is an expanded use of schemas in their development efforts. This isn’t something I had done much in the past as almost half of my career was with SQL Server 2000 or earlier, where we didn’t really have schemas and we were loathe to tie too many objects to owners. However the last decade has had schemas as a first class citizen in all SQL Server versions since 2005.

    I’ve seen a number of software packages use schemas, but it seemed more as a way of separating out development efforts than any well thought out architecture. However there have been a few interesting uses of schemas as a way of duplicating tables for various uses. ETL and auditing are two of the more common uses, though there have been others.

    I ran into an interesting approach recently that I found interesting. A group of developers had added a new schema to separate out their custom objects from all other objects in the database. They did this as the database was created by, and maintained by a third party application. They wanted to create new objects, but wanted them separate from the vendor’s objects, and used a new schema.

    I thought this was a good idea and wondered who else be doing something similar. This Friday, I wanted to ask the question of the rest of you.

    Do you have a custom or reporting schema as a way to separate objects from those that aren’t under your development control?

    Let us know if you’ve created some interesting use of schemas. Or are there reasons you don’t use schemas? Have you never considered the advantages of using schemas to group  objects in their own section of the database?

    I think schemas can be a powerful way of grouping objects, applying separate security, and  organizing your database in a way that makes it easier for developers, and report writers, to manage the complex objects that may clutter up your database.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn.