Category: Editorial

  • The Loss of Knowledge

    This article has a great opening quote. It says: “We are drowning in information but starved for knowledge”. It’s from John Naisbitt, who wrote the book, Megatrends in 1988. I think this quote can be very apropo in organizations as we have data, we have plenty of reports deriving information, but sometimes we don’t have a lot of knowledge, especially when there has been turnover in our staff.

    We can train new people on many things, but not everything. The knowledge of the culture, of what others know, of the little strange bugs or behaviors that can’t get fixed, the tribal knowledge accumulated by living in an environment. These are the learnings that can’t easily be replaced, and until they are, often new employees are less productive.

    Of course, sometimes new employees will view the world in a different light and find solutions others haven’t considered. That happens, but it’s more likely that they will make mistakes, break something, or just be less effective than their predecessors.

    There are no shortage of articles, like the one above or this one, that discuss the concerns IT leadership has about employee turnover. Perhaps the leadership does see that as a problem, but often first level managers don’t. Often leadership doesn’t realize how poorly trained or effective first line managers are in working with their technical staffs. I sometimes think that the world in Dilbert is far too prevalent precisely because of very poor management skills. It certainly seems that efforts made to retain employees are relatively rare in many companies.

    In my career, we’ve had some boom and bust times in the market for developers. There have been times where anyone with a certification or a hint of experience could get hired (or get a raise). There were other times when people were careful to hold onto their jobs because finding a new one could be hard.

    I think employers should not only invest in their staffs, but work to train and upskill them, demand more from them over time, but treat them fairly with more than just a paycheck. The dividends from a hardware allowance, a training budget, and more can easily pay for themselves with better productivity and the lack of fees to recruiters. You should certainly hold staff accountable and responsible for getting work done, but make sure you also treat them fairly and support them.

    That’s if you also spend time managing your managers and ensuring they balance the demands they make of employees with the support needed to ensure their staff performs well. If you ignore the managers, you might as well ignore the staff, set aside more recruiter fees in your budget, and hope for the best.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Big Data Downsides

    Companies often want more data to help them make decisions on how they run their business. There has been this quest to gather and analyze as much data as possible to increase the efficiency of their operations to help reduce costs or increase profits. This has led to the importance of data as an asset, and the need for more data professionals in many organizations.

    That’s good for many of us that work with data.

    However, using data to try and improve your efficiency has a downside. It can lead you to a very narrow focus in your approach. That can be good in narrow, well-defined areas, such as minimizing the distance driven or packing containers. For less focused tasks, such as telling a story or writing code, this can mean you get stuck in a rut and limit your opportunities to improve.

    There’s an interesting article about big data and Hollywood, specifically looking at the types of products produced. Big data analysis leads companies to aim for the most effective types of movies that make money. Good for a company, not so good for society. Arguably, not even good for a company over time as people will tire of the same story, or type of story over time. Eventually, making simple decisions based on past data will start to fail.

    I can see the same thing in other industries as well. Using Big Data to drive decisions can help, but many of the areas where we use these techniques will evolve and change over time. The way we solve problems with code change over time as we develop new tools, techniques, platforms, languages, etc. There isn’t a perfect way to design a database or write a CRUD app precisely because new capabilities or new possibilities emerge. You could say the same things about marketing, manufacturing, medicine, and many other endeavors.

    This isn’t to imply big data and complex analysis isn’t helpful or useful. It’s just not everything. We need to balance human input, with some creativity, some instinct, some diverse thought, and some guessing. Most importantly, we ought to experiment and learn, not only from what machines might extrapolate, but from how humans change their thinking over time.

    Find a balance, accepting some imperfection in your process and in the world at large. Hopefully that will lead you to some success.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • A Prompt Engineer

    No, Redgate Software isn’t hiring for SQL Prompt. I’m sure quite a few of you depend on SQL Prompt and would like more engineers working on it. Maybe a few of you would find that an interesting piece of software to work on.

    Rather, a Prompt engineer is someone that works with AI, trying to get a system to produce better results. I can’t decide if this sounds like an interesting job that stretches your brain or the equivalent of a mediocre developer that just keeps copying something from Stack Overflow, hitting compile, getting an error, and repeating that cycle.

    I can picture a scientist using their voice to continually correct some AI system by saying “no, that’s not quite right. I want to see more blue” or some other type of guidance. Is this job the equivalent of raising a digital toddler?

    I actually found one of these jobs on Indeed.com. For a company in San Francisco, a hybrid position paying between US$175k-335k to help create steerable AI systems. I’m not entirely sure what this is, but at that pay rate, it is tempting to learn.

    I think AI systems are fascinating, especially for someone that has done some development and can see how difficult it would be to specify an algorithm that can handle some tasks. Like composing some prose result. Or suggesting code based on what you’ve typed. These are some handy features that have been added to some IDEs and services.

    GitHub CoPilot is interesting, and if you haven’t tried it, I think it helps Java/C#/app programmers. Go watch some videos or sign up for a trial. I am not sure about these helping us in database work. There is probably a little more work needed before an AI delivers help for your particular database issue. However, I’ll be testing and working with some tools to see what I think.

    If you are interested in AI and have a lot of patience to train models, and a lot of patience to work on data cleansing and loading, then maybe you can find a new career in AI. It might be a fad, but I don’t think it will ever go away, and I do think there will be a lot of opportunities in this space in the next decade.

    Note: If you find this interesting, I had a discussion recently with a few others in a webinar.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Information Schema Strangeness

    There was a Slack thread at Redgate recently where a developer was showing some code where they decided to use the “extra” column from the information_schema.columns view. They were making decisions on how to detect certain metadata about a column based on the data in this column. Apparently, the data in here is overloaded for different options that might be set on a table.

    This caught my eye because I had no idea there was a column named “extra” in this view. I flipped over to SSMS and decided to check what was being stored in here. To my surprise, there was no “extra” column. As I dug in a little deeper in the thread, I realized the developer was talking about Information_schema.columns in a MySQL database.

    That was a surprise to me. While I know different platforms will add features and functionality to their databases, I thought the information_schema views were consistent across platforms. They should give you a set of information you can count on. Apparently, that’s not true. You can count on some things, but not all, which means that these aren’t consistent structures.

    Perhaps it doesn’t matter. It seems every product out there will extend the SQL “standard” where they see fit, adding features or functions that suit particular use cases. Commercial vendors do this for profit, and OSS projects likely do this because an individual wants a change. That has resulted in a wide variety of database platforms that meet different needs and solve different problems.

    It would be nice if we could write SQL code and be sure it would run on SQL Server, Oracle, Snowflake, PostgreSQL, or any platform. And in many cases, we can. Lots of basic queries are the same. However, what would be the point? I certainly don’t want more people in management wanting to switch from one platform to the other, just because they feel like it. I’d imagine that we’d thrash between platforms every time a senior developer or VP decided a system should run on their favorite platform.

    A base standard is good, like a base class in programming. However, they aren’t always as useful as they seem, and extending them to meet needs is better for us all. I don’t need a standard implementation of the SQL language or the information_schema views, it was just a surprise to realize that this actually how the platforms are coded.

    Note: If you find this interesting, I had a discussion recently with a few others in a webinar.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.