Big Data Downsides

Posted on April 12, 2023 by way0utwest

Companies often want more data to help them make decisions on how they run their business. There has been this quest to gather and analyze as much data as possible to increase the efficiency of their operations to help reduce costs or increase profits. This has led to the importance of data as an asset, and the need for more data professionals in many organizations.

That’s good for many of us that work with data.

However, using data to try and improve your efficiency has a downside. It can lead you to a very narrow focus in your approach. That can be good in narrow, well-defined areas, such as minimizing the distance driven or packing containers. For less focused tasks, such as telling a story or writing code, this can mean you get stuck in a rut and limit your opportunities to improve.

There’s an interesting article about big data and Hollywood, specifically looking at the types of products produced. Big data analysis leads companies to aim for the most effective types of movies that make money. Good for a company, not so good for society. Arguably, not even good for a company over time as people will tire of the same story, or type of story over time. Eventually, making simple decisions based on past data will start to fail.

I can see the same thing in other industries as well. Using Big Data to drive decisions can help, but many of the areas where we use these techniques will evolve and change over time. The way we solve problems with code change over time as we develop new tools, techniques, platforms, languages, etc. There isn’t a perfect way to design a database or write a CRUD app precisely because new capabilities or new possibilities emerge. You could say the same things about marketing, manufacturing, medicine, and many other endeavors.

This isn’t to imply big data and complex analysis isn’t helpful or useful. It’s just not everything. We need to balance human input, with some creativity, some instinct, some diverse thought, and some guessing. Most importantly, we ought to experiment and learn, not only from what machines might extrapolate, but from how humans change their thinking over time.

Find a balance, accepting some imperfection in your process and in the world at large. Hopefully that will lead you to some success.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Posted in Editorial | Tagged , | 2 Comments

Creating a Simple Graph in RedisGraph

Posted on April 10, 2023 by way0utwest

I delivered a session on graph databases, and in it I used RedisGraph to show how you can work with graph data on that platform. This blog shows a basic creation of a graph in RedisGraph. This isn’t intended to be a comprehensive post, but more a basic look at the code to work with graphs.

Adding Data

I connect to Redis with the command line. If you don’t know how to do that, I covered this in another post. Once I’m connected, I can run this code to create a graph and a node:

GRAPH.QUERY Northwind "CREATE (:employee {employeeID: 10, firstName:'Steve', title:'President'})-[:REPORTS_TO]->(:employee {employeeID: 11, firstName:'Tia', title:'CEO'})"

This will create a graph, two nodes, and one edge between them. This starts with GRAPH.QUERY, which let’s Redis know this is a graph query. I then give the name of the graph, which is Northwind since I based this on the Northwind dataset.

Next I use a Cyper language CREATE and enter the data in what is very similar to JSON. I label a node (employee), add the properties and values (employeeID and firstName with 10 and Steve as values).

I use the square brackets with the edge name in between this and another node. This will get me the results shown below. You can see that one label, two nodes, six properties, and 1 relationship created.

2023-03-24 15_34_05-cmd - redis-cli

I can get the employee nodes with this code:

GRAPH.QUERY Northwind "MATCH (e:employee ) return e"

This returns the two nodes.

2023-03-24 15_37_23-cmd - redis-cli

If I want to see the relationship, I need to ask for that with this code:

GRAPH.QUERY Northwind "MATCH (e:employee)<-[:REPORTS_TO]-(sub) RETURN sub.firstName as employee, e.firstName as manager"

This returns my graph as a series of data elements.

2023-03-24 15_37_23-cmd - redis-cli

Tada!

There is more you can do and certainly you can create more complex (or larger) graphs in Redis. The drivers from various languages should return JSON to you that you can deserialize and visualize or otherwise process results.

A lot of this is basic, but if you want to experiment with Redisgraph,  a container and the CLI is a good way to get started.

Posted in Blog | Tagged , | Comments Off on Creating a Simple Graph in RedisGraph

A Prompt Engineer

Posted on April 10, 2023 by way0utwest

No, Redgate Software isn’t hiring for SQL Prompt. I’m sure quite a few of you depend on SQL Prompt and would like more engineers working on it. Maybe a few of you would find that an interesting piece of software to work on.

Rather, a Prompt engineer is someone that works with AI, trying to get a system to produce better results. I can’t decide if this sounds like an interesting job that stretches your brain or the equivalent of a mediocre developer that just keeps copying something from Stack Overflow, hitting compile, getting an error, and repeating that cycle.

I can picture a scientist using their voice to continually correct some AI system by saying “no, that’s not quite right. I want to see more blue” or some other type of guidance. Is this job the equivalent of raising a digital toddler?

I actually found one of these jobs on Indeed.com. For a company in San Francisco, a hybrid position paying between US$175k-335k to help create steerable AI systems. I’m not entirely sure what this is, but at that pay rate, it is tempting to learn.

I think AI systems are fascinating, especially for someone that has done some development and can see how difficult it would be to specify an algorithm that can handle some tasks. Like composing some prose result. Or suggesting code based on what you’ve typed. These are some handy features that have been added to some IDEs and services.

GitHub CoPilot is interesting, and if you haven’t tried it, I think it helps Java/C#/app programmers. Go watch some videos or sign up for a trial. I am not sure about these helping us in database work. There is probably a little more work needed before an AI delivers help for your particular database issue. However, I’ll be testing and working with some tools to see what I think.

If you are interested in AI and have a lot of patience to train models, and a lot of patience to work on data cleansing and loading, then maybe you can find a new career in AI. It might be a fad, but I don’t think it will ever go away, and I do think there will be a lot of opportunities in this space in the next decade.

Note: If you find this interesting, I had a discussion recently with a few others in a webinar.

Steve Jones

Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

Posted in Editorial | Tagged , | Comments Off on A Prompt Engineer

Dropping Career Goals for Q2

Posted on April 7, 2023 by way0utwest

I set goals at the beginning of 2023 for Q1. I didn’t do well, as my evaluation earlier this week has a D for my efforts.

I’m re-evaluating things, and for now, I’ve decided not to set more goals for the time being.

Why?

One of the things I end my career presentations with is this: we work to live, not live to work. Find balance in your life, between your career and life.

I truly believe that, and as I re-evaluated myself last week, I realized that I am out of balance. There have been some stressful things in life that have been hard on me, but more than the actual thing, it’s had an impact on my mental health. I’ve been a bit drained, so I haven’t wanted to take time to focus, preferring to relax a bit more with my wife, guitar, or a good book.

I started to recognize this when I did my Feb update, but thought I could push through. I didn’t, or couldn’t, and while I appreciate the optimism, I realize that I don’t want to ignore the additional stress in my life.

A Deliberate Pause

I don’t want to let goals go, but I am going to ignore them for Q2 and focus on being kind to myself and recognizing I need balance here. I’ll revisit this in late June and see how I am doing then.

Posted in Blog | Tagged , , | Comments Off on Dropping Career Goals for Q2