Republish: Writing Before Reading

I’m still in Europe, now in Rome. I’m sure it’s amazing, but while I’m gone, you get Writing Before Reading. I don’t see this in the archive, so here it is in total:

Writing Before Reading

This editorial was originally published on May 21, 2020. It is being republished as Steve is gone this week. Have your thoughts changed on reading and writing code?

Coding is one of the few (or maybe only) places where we learn to write a language before we learn to read it. Many of us get started in some way by working through exercises, looking at very small snippets of code and then starting to write our own. Perhaps that’s because we experiment by changing the behavior of the computer, which naturally takes some writing work on our part.

Perhaps we can do better, and should to better. Perhaps we would become less susceptible to the Not Invented Here (NIH) stance that so many developers seem to take. Not with major projects, but often with small sections of code or libraries, preferring to structure something new than re-used (and take the time to understand) what’s already there.

This isn’t to imply that most developers need to write their own code or don’t work to understand others, but often they prefer to rebuild something in a way that is more familiar. It’s a form of the Mere-exposure effect, and it sometimes leads developers to prefer certain patterns, tools, platforms, and more. Perhaps it’s the opposite of the shiny ball syndrome, which leads people to chase the newest thing available in their area.

I know there are plenty of developers that copy and paste code from the Internet as well, which might seem to be a way of getting around NIH syndrome, but I think many people do this when they find code that solves a problem. Even if they don’t understand how it works, they’ll seek to reapply that same code to a new problem.

I do think reading code is hard. In some sense, it sucks. I’d often prefer to understand the basic inputs and outputs, perhaps rewriting the code to work in a way that’s more comfortable for me to understand. Maybe I better understand my own code, maybe I’m falling victim to the IKEA effect (I’d rather depend on my own software than someone else’s), or maybe I’m just being lazy.

There are plenty of articles on how to read other people’s code, and I am trying to get better at this. I do try to debug and reason out how queries are written that solve problems in the forums. I may break apart queries and try to understand how someone has used a window function or a CROSS APPLY in a certain way. It helps build my skills, but it also helps me learn to better read other people’s code and perhaps carry that skill into the future.

Steve Jones

Posted in Editorial | Tagged | Leave a comment

T-SQL Tuesday #198–Change Detection

This month we have a new host, Meagan Longoria, who graciously agreed to help me this month. I’ve known Meagan for a number of years and she’s been a person whom I’ve asked questions about data visualization and analysis in the past. I was slightly surprised by the topic this month, but only slightly. I’ll write my answer below, but if you want to try blogging and host a future month, let me know.

Change Detection

The topic this month is change detection, which is important for efficient ETL work, but also for other areas, such as auditing. I haven’t tended to work in high volume systems where we did a lot of ETL and needed to very efficiently detect changes. Most of the time I’ve had ETL pipelines they were busy, but not excessively so.

As a result, in the past, I’ve often used a roll-your-own approach in the SQL 6.5->2008 era. As I’ve worked on those systems, we’ve usually used a simple update or modified date in the table that tracked when something was altered. By knowing the last time a pipeline ran, we could gather all data from that point forward and extract it.

We used a similar approach to send a lot of emails from SQL Server Central years ago. That can work well, and as long as you track the last execution of your pipeline process, whatever that is, you minimize the data being transferred.

A Modern Approach

I ran into dbt a few years ago and did a one day class on how it works. It was interesting to me, and I could see the appeal. Recently I had a conversation with John Miner, who’s used it in his Fabric Modern Data Platform series. I would be very tempted to use dbt, in conjunction with a modified date as described above, to ETL data around today.

However.

I have been seeing that Change Data Capture (CDC) is being used by lots of products these days. It’s behind the Fabric mirroring, Oracle has used it for years, when you look to move data into Databricks, CDC is common, it seems like it’s everywhere. 

If I were going to be regularly moving data in 2026, CDC seems like something I’d experiment with and test, since it’s a known technology that works across many platforms and there is a lot of knowledge out there on how it works. AI can certainly help with experiments, and with understanding the overhead on your system, because there is some overhead.

I haven’t looked at the new Change Event Streaming, though I’m always wary of anything that limits me to one cloud. The more generic CDC, with AI assisted configuration and maintenance, seems like a better approach.

Posted in Blog | Tagged , , | Leave a comment

Redgate and Rome

I’m at the UK Redgate office today, meeting with senior leaders in all areas of the business. I was honored to be awarded a spot in our President’s Club. This is where the top performing sales people get to meet with leaders in the company and then head off for a few days of relaxation and excitement.

I’m here today and tomorrow and then off to Rome. We’re heading to tour the city, see the Vatican, and visit some great restaurants. I’m excited for the trip and I’ll have a few photos and thoughts next week, but I’m off this week.

Hopefully it’s a quiet one for you.

Posted in Blog | Tagged , , | Leave a comment

Republish: Other IDEs

I was very lucky, and honored, this year to be picked as a part of the Redgate President’s Club. This is our sales award club, which primarily goes to reps, but sales engineers and others are picked. I was one of the wild cards, as I helped on a lot of deals last year, so today I’m in Cambridge meeting with product people and execs.

While I’m over there, you get to re-read Other IDEs. Leave a comment and let me know if things have changed since I first wrote that. Are you using another IDE?

Posted in Editorial | Tagged | 4 Comments