I’ve seen this list of why projects fail going around the Internet for awhile (recently from Brent Ozar Unlimited). There are a number of items in here, and if you substitute some other field for the data scientist part, you might have seen some of these issues as well. One of the items that really struck me was number 5, which is titled: You shouldn’t have hired scientists. The other part of the slide is that ETL needs data engineers (whatever those are) and reporting needs BI analysts.
I with that we had good ideas and definitions of what different jobs are supposed to do in a company, and what skills are helpful. The more I look at job descriptions and responsibilities, the more I find that DBA, analyst, data xxx-anything are just amorphous concepts that vary dramatically from organization to organization. In fact, since the title that one has often determines pay scales, I think that we start to covet titles and work towards getting a title, regardless of the work we do. This isn’t helpful for anyone, least of all HR departments that try to match salaries to value and pay for particular jobs.
There isn’t a great solution here. Our industry is young and changing more rapidly than any other in history. New jobs are created out of thin air, to meet the changing face of software development and system administration. Build master, data scientist, blockchain engineer, and a few others didn’t exist when I started working in technology. Whether those jobs are needed in an organization might not relate to whether or not anyone has those titles, or if they perform the tasks we might expect of those positions.
I think that many of us are expected to be able to perform any task tangentially related to our position. Or we should be able to learn it quickly. DBAs should understand replication as easily as they might Availability Groups and configuring Extended Event sessions. A developer should be able to write C# as well as T-SQL. Both should be able to use SSIS to import and export data. While I’m sure many of us could do those tasks at a rudimentary level, are we really competent and capable at each of those tasks? Maybe enough for our organizations, maybe not.
When we embark on projects, there may be needs to accomplish tasks outside of the core competency of our staff. That’s fine, and since many people in technology are willing (and excited) to learn new skills, this situation may be perfectly acceptable for our project. When there are core skills critical to the project, such as a deep understanding of large scale ETL processes, we are probably better off hiring people that have those skills or investing heavily in just-in-time training for existing staff. Hoping that the average DBA or C# developer can just “pick up some tips” is a recipe for project failure.