T-SQL Tuesday #165–Job Titles

tsqltuesdayThis  T-SQL Tuesday is from a new host, Josephine Bush, leader of the Boulder group just North of me. It’s an interesting invitation, asking what our job titles really mean. I like this as the titles do affect how our industry advances and what opportunities are available to us.

I manage the tsqltuesday party, so if you want to host, ping me.

The DBA

When I was a software developer, I worked with a DBA that made a lot of $$$. That was very attractive to me. So, I pursued that title. There were few of them in any company, and I needed to learn SQL, modeling, performance, etc. but once I got to that point, I found it was a good job.

These days I see some companies with DBAs, but lots of the customers I work with don’t have DBAs as a title and have some sysadmin or developer doing those jobs.

New Titles

The invitation shows other titles, but often what I see is titles are for HR and pay levels. The job often is very similar. I see DBAs/Database Engineers/Database Reliability Engineer, often doing very similar work. They might watch over servers with admin stuff (backup/security, etc). They might do DevOps, IaaC, deployments, they might also do modeling and consulting with devs.

Database Developer is something I see often, sometimes with BI developer, and often they do the same work. The former might do more T-SQL on OLTP systems and the latter might only want to work with BI/OLAP/warehouse systems. However, I find people who want to limit themselves are limiting opportunities and might be those that don’t hold onto a job for a long time. That being said, I do find some BI developers might not have access to all development databases, but database developers often do.

I’ve seen a few data scientists. They’ve been around for years, there has been a big push to get them to work on ML/AI systems, and I see less of that in the last few years as there AI tech grows and gets more commoditized. It doesn’t seem that as many companies want to pay highly for this role. I have no idea what they do other than lots of ETL and experiments to try and get a computer to make better decisions from data.

Architects – I know a few people that have this role, but I don’t see a lot of them. These are usually people that evaluate tech, set standards, and try to oversee a common direction for database designs or tech, however, I also find these people need to be doing other work to justify their existence. They often fill in where work is needed.

In general, I think titles sometimes reflect a specialization of a portion of technology, but aren’t necessarily needed. I’ve been a DBA and done all the things the invitation lists for various roles. However, I find that if I wanted to be a database engineer rather than a database administator, I could likely justify a higher salary.

My advice is pick the title that has a good salary and aim for it. Learn all the skills, since the title might not matter when someone needs work done.

Posted in Blog | Tagged , , | Comments Off on T-SQL Tuesday #165–Job Titles

A Multitude of Summary Views in SQL Compare 15

A client asked for a summary of changes, so I wrote a post to show where to find this in SQL Compare 15. As I was checking out the Summary, I realized there are different ways to present this info, which can make your job as a data professional easier. This post shows the different views.

This is part of a series I have on SQL Compare from Redgate Software. It’s an amazing piece of software that you should try if you haven’t. Download an eval today.

In the previous post, I showed this image, which is the default view. This lists the changes by object, meaning that I see the objects listed on the left with schema and name, and then the actions on the right. In the image below, the first item on the left is dbo.Street, with two items after it. On the right, we see this is a need to drop a PK and then create a new PK.

2023-07-20 14_21_37-Deployment

This is a good list, but there are other choices. In the upper right, there is a drop down. If I click it, I see this:

2023-07-20 14_27_13-Deployment

The Object is the default one, but if I click Modification, I see this view, which lists the types of changes. You can see that I have 1 drop PK, 4 table alters, 1 view alter, etc.

2023-07-20 14_26_56-Deployment

A good view, though I don’t often case about this, other than I usually would look for drops because those are problematic.

The third view is the modifications, but in the order they’ll take place. I like this, since as someone who has been a production DBA, I want to know what changes are happening in what order.

2023-07-20 14_27_07-Deployment

In this view above, the alters are split out. So I get one alter of a table, BaseTable, then a view, then a refresh, then alters for two other tables.

Using these summary views are good ways to initially conduct a review of what is changing for a deployment. This is valuable because for a lot of changes, this makes sense. For some, I might want to see the code, so having the changes in order here let me quickly find this change in the script and view the exact way this is going to be deployed.

SQL Compare is an amazing tool and it makes life much easier. If you haven’t tried it, download an eval today and give it a try. If you’d like to see this in a pipeline or more automated system, give the Flyway suite of tools a try.

Posted in Blog | Tagged , , | Comments Off on A Multitude of Summary Views in SQL Compare 15

Archive to the Lake

Microsoft Fabric was announced at Build in May 2023. This is the next evolution of data warehousing from Microsoft, folding in Synapse and a number of other technologies to create a simpler location for storing and analyzing data. We’ve published some articles on the platform and there’s a great presentation from Mr. Paul Andrew on Linked In. It’s worth listening to, even in the background. Paul has a nice style and a great voice.

Part of this platform is OneLake. This is a data lake for your org, just one of them, and while it’s able to store data in many formats, it’s mainly optimized to read tabular data in the delta parquet format. This is essentially a compressed text file that allows for some transactional changes to the original data in parquet format.

I don’t do a lot of work with text files, and I’ve been suspicious of using lots of CSV or other text files in a warehouse environment, which is what a lot of people were advocating a few years ago. Exporting tables into lots of files split on some field, like date, while easy, didn’t seem like the best way to move data for reporting.

Fabric, however, is optimized for reading delta files. A few presentations I’ve seen from people have advocated for exporting your data from SQL Server (or other platforms) into parquet. While I don’t know there’s a native way to do this (yet), I suspect one is coming. I’ve seen lots of articles (one, two, three, more) about how to do this now. We also have SQL Server able to read these files with external file formats already, so I’m sure we’ll have an easy way to write them soon.

Many of us struggle with large systems, especially with query performance. We’d love to archive off data, though that’s often impractical. However, in an amazing, wonderful world, maybe we’ll get lots of people doing this, writing about it in the media, and our bosses will start to let us establish an archive in the lake. We could move some data there, especially old, unchanging data. We could delete that from source systems. We could have all our users happy.

I don’t know if I see lots of data moving to the lake, but I certainly expect lots of it to be copied. If you haven’t thought about archives, data lakes, and text formats, it’s an area that seems to have a lot of growth. Perhaps it’s of interest to you and you might find a new career.

Or maybe you just hope it gets widely adopted to relieve some pressure on your OLTP server.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Posted in Editorial | Tagged , , , , | Comments Off on Archive to the Lake

100 Days to the Data Community Summit Keynote

The start of learning at the 2023 PASS Data Community Summit is 100 days away. I checked.

2023-08-04 09_24_01-SQLQuery2.sql - ARISTOTLE_SQL2017.master (ARISTOTLE_Steve (52))_ - Microsoft SQL

The Summit starts for many of us on Monday with precons or an early arrival and spending time with friends. However, the first keynote is Wednesday, and that’s the main conference for most people.

100 days away.

It’s amazing to think we’re that close. Speakers are working on their sessions, polishing decks and practicing demos. Hopefully attendees have gotten all their travel arranged. The sessions are out, and you can build a schedule. You can even still add a pre-con if you want.

Hopefully I’ll see you there. If you’re valuable at work, if you get things done, make a case to send you to the Summit to learn things and network with others. Those contacts might be invaluable in the future when you have a question or problem.

And if you’re going, consider coming to SQL Saturday Oregon the Saturday before and taking the train up with us. It’s a fun time.

Posted in Blog | Tagged , , , | Comments Off on 100 Days to the Data Community Summit Keynote