Category: Editorial
-
Building Bonds with Entertainment
Over the years I’ve had the chance to work in a lot of companies, and I’ve seen a lot of different team-building attempts take place. In restaurants, these were often nights out with too many adult beverages. At a power station, we had some large outdoor BBQs, where again, sometimes there were too many adult drinks consumed.
When I came to Denver, I worked in a company that had a small fund they used for team outings once a quarter. A movie, bowling, maybe a meal, usually during an extended lunch hour or an early end to the day. At Redgate, we have a Feel-Good-Fund in each office and teams can make use of this for an activity periodically where they engage in something together. I rarely get to participate because I’m so remote and it makes me a little sad, since I see pictures on Slack of escape rooms, punting (boating), visiting museums, archery competitions, and more. However, we have taken the time to do a few things when Grant, Ryan, and I get together. Most recently, we went to a science museum in Cambridge one afternoon.
The Data Community Summit has been a part of my life for over 20 years, and I’ve often gone with the goal of spending more time with people than in sessions. The bonds I’ve built there have been amazing, whether in the convention center, over a meal, or out at some event. This year is no different, as I look forward to meeting new friends and greeting old ones.
Building a bond with others is something that takes time, but it can grow more quickly when everyone involved is enjoying themselves. Years ago Andy Warren and I were discussing how to get more people to network. There were lots of parties thrown by vendors, but not everyone was invited, and a lot of people don’t want to go out drinking in a loud atmosphere. We came up with Game Night and convinced PASS to give us a room. It was a wild success, and it’s continued many of the years since.
This year Game Night is back. It’s a little more formally organized, but it’s on the community page as an event Wednesday night, after the exhibition reception. The staff is looking to bring some games, and they’ve asked what others think or what they want to play. Knowing many of you will want to bring your own games, I’m not worried, but I do hope that everyone going spreads the word and the night is a rousing success. I stopped by the Game Night at That Conference this summer and it was a good time. Lots of people having fun, in big and small groups, playing all sorts of games. Some short, some long, some easy, some very complex.
Engaging with someone over a game of some sort is a good way to get to know them, and enjoy their company during an activity that is more engaging than watching a movie or going to a loud concert. Those things are fun as well, but part of professional events is getting to know others and building your network. Hopefully to take advantage of those opportunities at user groups, SQL Saturdays, or large conferences.
And if you’re coming to the Summit, consider bringing a game to share with others or coming to Game Night and learning a new one.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
-
The Need for DevSecOps
One of the things that happens with many companies that start adopting DevOps is that they release new features constantly. They publish their lists of changes, and they try to attract customers and grow their businesses. They may make some mistakes, but they fix those quickly and keep pushing forward. That’s the idea, and it works well.
However, many of the developers (and most managers), don’t think about the security side of their changes. This piece looks at the way hackers and criminals view DevOps, often using release notes and feature changes as a target to focus their efforts. In this way, they exploit holes and vulnerabilities in software to attack data storage. The examples include S3 buckets of storage and Elasticsearch, which is notoriously poorly secured by many people.
I’m sure there are hacks that also expose relational data stores and NoSQL stores, but those are often more secure and harder to directly attack. Certainly, hackers do get credentials and can query data, it’s more often I hear about data breaches from other sources than direct relational database access. SQL Injection is definitely still an issue, and I hope that more and more developers are learning patterns that avoid these vulnerabilities.
DevOps works. I think it’s great. However, it’s not enough to trust developers to build features without including static code analysis, pen testing, and other security evaluations before you release code. Often developers can build features just as fast with good patterns as bad. Use automation to catch bad patterns and force developers to learn new ones.
Also, avoid letting developers implement data stores out of convenience and speed. Ensure that strong security practices, long passwords, service accounts, secrets, and more are implemented from the start. It’s easy to shortcut these, but it’s also harder to explain to customers and investors why we didn’t do better.
If you’re a developer, the main thing to keep in mind is that you will often be the scapegoat for these issues. Upper management might not support you and pressure you to move faster, but when there are issues, they’ll also be quick to blame you and let you go first. Push back and ensure that you have the tools and the process to evaluate if you are building problematic code. Always use long passwords, and document what you do for others to follow.
And if your boss insists on cutting corners, get that in writing. It might not save your job, but that documentation has served me well in the past when issues come to light.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
-
A Staffing Disaster
There was a failure recently at an Azure data center in Australia when a utility power sag caused equipment to trip offline at one of the Azure data centers in Australia. You can read about it here, but essentially the headline is that there were only three people on site when the incident occurred, and that caused them to be unable to restart the equipment in time before an outage occurred.
In a little more detail, there weren’t enough people to quickly restart the equipment chillers after the incident. The staff had to access the equipment on a roof when 13 of the units didn’t restart. They were able to get to 8, but when they got to the last 5, the temperature of the water had risen to a level that wouldn’t allow a restart. So they had to power down some computer equipment and go through a more lengthy process to get everything running.
This sounds bad, but in reality, this is exactly the type of thing I’ve seen in private data centers, who almost never have all the staff they need, or the knowledge necessary, to deal with large-scale failures. While I haven’t seen the chillers, I have seen people trip electrical systems and be unable to restart or reset UPS’s or generators for hours until qualified staff could come in. If you read the incident history, there is a good retroactive of what happened, and then some actions taken to try and prevent this in the future. They increased staff levels but also identified some places where the previous staffing level would have been fine with some equipment and protocol updates.
I wish more organizations would review incidents and examine them with an eye towards not only what happened and where there were failures, but how to prevent issues in the future. Too often I see people going through this exercise in order to blame someone and “prevent this from ever happening again”, which usually means we fire someone and don’t change anything else. We need psychological safety in reviews of actions to get better.
As we build more complex systems, or even more complex organizations with lots of teams, people, equipment, procedures, etc., it’s easy to build in lots of points of failure without realizing there will be problems in the future. My goal often these days is to assume I’ll have some inexperienced or less capable staff and design processes and systems to survive issues. To keep things simple, and not get too cute with engineering. I like robust, resilient systems that anyone can operate, not those that require me to ensure my senior superstars are always on call.
Of course, it often takes the senior superstars to design and test these systems and protocols, which is a good use of their time.
Many businesses struggle with staffing, in many areas. Technology groups are no different, and we have to learn to work smarter, not assume we will just get more staff and solve our problems.
Steve Jones
Listen to the podcast at Libsyn, Spotify, or iTunes.
