Voice of the DBA

Author: way0utwest

  • The Downside of Tool Autonomy

    I am a big believer in letting teams of people work in a way that allows them some autonomy to solve a problem. I think this is especially important in the software world, where developers need to experiment and learn from the new ideas that they investigate. They need to be able to try new techniques, technologies, and platforms. Yes, that includes NoSQL data stores.

    At the same time, this can lead to developers becoming captivated by new tools and techniques, all of which are good for advancing skills and code maturity, but can also lead to a very disparate environment where there are no best of breed solutions, or no standards for writing features. As the number of developers grow, this also raises the bar for each individual to know and understand more languages or technologies.

    While I do think it’s easy to pick up logic and syntax, when writing production code, we want to ensure we’re optimizing the way we work. The way in which we might write some code when we start with a new API, SDK, language, etc. is often not the way we’d write that code after a year or experience. At least, I hope it’s not the same way. There are nuances, tips, and tricks in how to better structure code that we learn as our experience grows.

    When I was at the Redgate Software office last year, I noticed that there was a short article posted on our internal blog. This noted that there is a set list of tools and technologies to use when building our products. This isn’t intended to be a long term immutable list, and the post noted that other choices are possible, but we want to avoid an unending growing list of tools when we have identified there is a good tool in a space. There is an approval process to propose and new tool and explain why it really is needed.

    At Redgate, we’re pretty flexible and open, so I’m sure this isn’t a tough process. However it does prevent every developer from implementing their special little code into a project. We’ve had people use various different data stores, different types of web technologies, and it becomes a maintenance issue over time. Even the VB.NET component we used for years at SQLServerCentral caused headaches as there wasn’t any experience in the company in this language.

    I think it’s worth having some light overview that ensures an organization isn’t adding technology just for the sake of making one or two developers happy. There is a finite amount of time to spend learning and growing, and while it’s important that technology workers do this, we also want to developer more experience and expertise in a few areas. Limiting tools and technologies helps preserve a balance that allows for growth, but ensures the expansion isn’t unchecked.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 4.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • Webinar next week on Database DevOps

    Next week is my webinar on why the database is the heart of DevOps. You can still register today to discuss and debate why we need to ensure that database development is integrated into any DevOps work we do for other software.

    Webinar social_Heart of DevOps

    Register: https://attendee.gotowebinar.com/register/3005019550714731265?source=KB

  • Reserved Space

    Many of us that work with SQL Server are familiar with the idea of reserving space that our system might need later. This is normal operation in SQL Server, where we pre-allocate space to data and log files, knowing that SQL Server will manage and use this space over time. This can often be a strange concept for newcomers to the platform, who expect our database behaves like a text file, only using the space it needs for data. Choosing the amount of space to reserve is an art that many of us both struggle to articulate and explain to newcomers. It’s also a problem that we often solve by guessing high and implementing monitoring.

    Lots of people worry about storage, even in this era of TB sized drives. I ran across an article noting that Windows is reserving 7GB of space for updates, which makes sense to me. I’ve had a few updates for smaller devices, like mobiles, fail because of space. I think the idea of having some space reserved for temp storage and then using that for updates is a good one. I also recognize that I’m in a different place than others. Most of my devices have plenty of free space. Well, not my phone, but that’s a separate issue.

    From the comments of the article, it seems that many people feel as I do. Reserving the space is a good idea. There are plenty of complaints about older, small storage devices, and I feel bad for those people. I’ve own a 32GB Windows device and over time it’s a pain. Those devices likely need to have some sort of network/cloud storage to be usable, so There are also no shortage of the “Windows is already too large” complaints, but to me, those are separate discussions.

    Ensuring enough space is around for normal operation is one thing. Keeping extra around for emergencies and critical functions is another. I’ve been a big fan of placeholders on drives, especially database servers. I keep a few on all my drives, both on desktops and laptops. If I run low on space and really need some, I can make the decision to remove these without thought. Something I can’t always do with other files.

    Like the DBCC SHRINKFILE, which lets us get out of situations where we’ve made a mistake in allocation, I’d like all OSes to include some options that remove older files, such as updates, temp files, etc. that we might not need, but could make a difference in our space usage. I know modern Android helps me when I’m low on space, and would like to see that in Windos. My other hope is that  we get quantum, PB sized storage devices that can fit in our phone. I can’t ever remember anyone on Star Trek running out of storage space on their drives. I look forward to the day in my life when that won’t happen to me.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 6.3MB) podcast or subscribe to the feed at iTunes and Libsyn.

  • A Large GDPR Victim

    During the last two years, Redgate has been preparing for the GDPR to take effect in the European Union. As a company based in the UK, we recognized that there were both challenges and opportunities for our business. We needed to ensure we were compliant with the regulations, which would likely require us to change processes and educate our employees. At the same time, our customers would face similar challenges and there was an opportunity to help them achieve their own compliance with software tools.

    The GDPR enforcement began last May, though fairly slowly with few fines and decisions being handed down. Across the EU, it seems to have been a quiet period with few companies told they were non compliant. Most organizations likely think all their preparation has been worth the effort and likely believe that they are prepared for any complaints from customers or investigations from regulatory authorities.That confidence may have been shaken in the last week as Google was assessed a fine over $50 million for violations. In particular, the EU regulators in France found that Google had not obtained the consent needed for using certain data in personalizing ads. They also decreed that Google had not clearly presented information about how users data would be handled and stored, as well as creating a difficult process to opt out.

    This fine isn’t much for the tech giant, but it’s just the start and will likely force Google to change the way they handle data. It may also have implications for other tech companies of all sizes. Google is appealing the decision, and this will be an interesting case to follow for data professionals since we may need to ensure that we can comply with the final ruling. Many of us view the data in our organizations as belonging to our employer, with free reign in how we handle, process, and store it. That may change quickly if the ruling is upheld.

    Much of the decisions about how companies will deal with data is made by others, but data professionals often need to ensure that we do comply with whatever rules our organizations decide to use. This means a number of practices that we must consider. At a high level, we need to know what data is affected by the GDPR, or any other privacy regulation. This requires that organizations have a data catalog that allows them to track which data is sensitive and must be handled carefully. Few organizations have a comprehensive data catalog already, so this will be an area in which to focus resources during 2019.

    Once we are aware of where our sensitive data is stored, we must take precautions to protect this data throughout our organization. Most companies have implemented security in their production environments, but their data handling practices in test and development areas are often not the same. The GDPR calls for anonymization, randomized data, encryption, and other protections, which data professionals will need to implement in a consistent manner throughout their IT infrastructure.

    Finally, accidents and malicious attacks will take place. This means that every organization really needs a process to detect data loss and a plan for disclosing the issues to customers. Auditing of activity, forensic analysis, and communication plans need to be developed, practiced, and distributed to the employees that may be involved in security incidents.

    There may be other preparations needed, and the larger the company, the more work that will be required. Tools are critical to ensuring this process can be completed in a timely manner, both to save time in implementing processes and also to show regulators that actions are underway to better protect data. Fine levels aren’t mandated, and the more effort put into achieving compliance, the less likely that regulators will assess a fine equivalent to 4% of your annual revenue.

    There will be plenty of other GDPR fines in the future, and it is worth following this case with Google to see how stringently the regulations will be enforced. The world of data handling practices is changing and all organizations need to get used to better disclosure of practices, tooling for customers, and protection of the data assets they hold.

    Steve Jones

    The Voice of the DBA Podcast

    Listen to the MP3 Audio ( 5.5MB) podcast or subscribe to the feed at iTunes and Libsyn.