Category: Editorial

  • An Unlimited Token Budget

    I don’t like unbounded situations. I’m always wary of “unlimited” or “lifetime” offers, as I think they’re entirely too optimistic. A few people on an endless cruise recently learned this. An unlimited offer is only as good as the company that provides it, and many companies have either gone bankrupt, failed, or had to alter the terms of their amazing offer (T-Mobile, American, Red Lobster, Movie Pass).

    Similarly, I’ve always been worried about giving developers unlimited resources, especially in places like Cloud CI systems. I’ve seen how often (I and other) developers will make commits, submit PRs, amend those, make new commits, etc. I don’t like the idea of potential uncapped builds or the costs they entail. Similarly, the bursting that some monitoring products offer in the cloud to capture all data can sometimes result in extremely high bills. Especially when it’s easy for users to add systems or change configurations that create additional compute or storage requirements. Vendors love this, as they get increased billing without the hassle of a sales effort.

    I saw Cisco recently offer a developer unlimited tokens. This isn’t quite as open-ended as it appears, and I do like the approach of a short timebox in which to experiment. A young developer has 4 weeks to create a PoC (proof-of-concept) showing he can use AI to rewrite legacy Cisco code. That’s a place AI might shine, and it’s well scoped. We know how their code works; we can write tests around it, and there’s likely a lot of training data available on how the systems work. The goal is 98% of their code rewritten in 2 years. I don’t know what the estimate would be for a rewrite with humans, but I’d guess this would be in man-years, maybe man-decades.

    It’s a challenge from the President, and if it works, well, I don’t know. If this is to prove AI can work well and inspire other engineers, maybe that’s good. Maybe it’s worth millions of dollars. And at unlimited token budgets, with creative engineers able to spin up hundreds (or thousands) of agents, it might be millions in cost.

    Per year.

    Or per month.

    I am rooting for this to continue past 4 weeks as it’s a great case study. Can a human rewrite lots of code effectively with agents? Can a human truly manage a swarm of agents with a well known and scoped target? There have been plenty of people experimenting here, but they are often doing so for research or fun, without a real goal other than to see what works. Cisco is making a business project here, and the results will be fascinating. Even a failure will teach us something.

    Even if it’s only about AI finance impacts from an engineer without any quotas.

    This will be an interesting story, and I hope Cisco provides updates and results. I suspect those reports might be light on details, but I will be following along.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Data Lost in the Cloud

    There’s been a war in the Middle East for many months now. As different countries battle, there have been attacks on infrastructure, including businesses that have facilities in the region. Among the casualties were a few Amazon data centers, and Amazon has acknowledged that some data is permanently lost. This isn’t to ignore the loss of life or horrific events, but for data professionals, where is a portion of this that is applicable.

    The cloud providers will group multiple data centers into Availability Zones (AZ), with some redundancy across zones to provide resilience. However, in this case multiple facilities were struck, and the damage exceeded what the AWS services were designed to withstand. Amazon did suspend billing in the affected regions, but that doesn’t necessarily help customers who were counting on data in the cloud. Note, the article discusses AWS systems, but Azure and GCP could have similar issues.

    While it’s unlikely that most of the AZs around the world would be engulfed in war, it is worth noting that none of the major cloud providers are doing anything to protect your data from catastrophe. For all the talk and marketing of “zones” and “redundancy”, there could be a wide-scale disaster that affects multiple places, and potentially renders data inaccessible in that region.

    Not necessarily all data, but it could cause problems with your data, which is what you should be concerned about.

    All the major cloud providers are susceptible to issues, and while they have implemented systems designed to tolerate failures, ultimately it’s your responsibility to protect data. Amazon warned customers when the first strikes occurred to copy or migrate data. I hope most customers did this, but I’m sure more than a few technical and managerial people thought AWS was fault-tolerant.

    AWS is, but there are limits.

    I wouldn’t abandon the cloud, nor would I stop trusting their backup services. I would, however, ensure I made some periodic copies of my data elsewhere. Losing a few weeks of data might be a major headache for many organizations, but losing all or most of your data might be something else entirely. Even monthly backups might mean the difference between a hassle and closed doors.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Pay for Performance

    There’s a new TPC-H benchmark from Microsoft; this one running on cloud infrastructure in Azure. It’s the top-performing SQL Server benchmark at the 1TB level, getting to 1,667,428 QphH (roughly queries per hour). This value exceeds the next highest on-premises setup from HPE, which has 1,531,016 QphH@1000GB. This is the 2nd-ranked TPC-H benchmark, though the top slot is from an Exasol database.

    Notably, the benchmark comes in at $558.17 in the Price/QphH price performance column. The other SQL Server systems are in the $200-ish range (from $192.37 to $263.13). That’s quite a difference in price, and it might dissuade some of you from moving to the cloud instead of running a large on-premises setup. That’s a point of concern, especially as the cost of the cloud has become an issue in many organizations.

    There’s a blog post from Pam Lahoud that outlines the effort to get this benchmark audited and approved. Part of the post notes that the rules for the benchmark mean that public pricing needed to be used. This is the spot, pay-as-you-go pricing, which is the most expensive rate. The storage was also configured with the maximum capabilities, which weren’t needed. She notes that the hybrid benefit for moving licenses isn’t available, which I’m not sure makes much of a difference, but it might. Pam does note that the licensing was priced at 3 years of the Pay-As-You-Go level, which is 2/3 of the cost.

    There are likely ways to optimize costs, and there is the promise of a future blog that covers how you might size and price what you need for a workload. I’m sure there are savings to be had, though it remains to be seen if you could get a price lower than that of the on-premises systems. Don’t forget that the on-premises systems might have other savings from a different approach there as well.

    The cloud, in many cases, is expensive. If you want to get great performance out, then the people whose computer you decide to rent will likely charge you a premium. Perhaps you can gain some savings with flexibility, shutdowns, resizing, and more, but that really depends on your being able to take advantage of the flexibility of the cloud. If you have a steady workload, or one that has to run most of the time, you’ll likely be paying quite a bit.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Helping an English Student with R

    I got an interesting message from my daughter recently. She’s doing graduate studies in Belgium at KU Leuven. Her field is literature and linguistics, and as part of that, she has to analyze various writings. While visiting with her last month, she mentioned that her professor was using R (R Language) and giving them scripts to run to look at language usage.

    The other day she texted that her R script wasn’t running and could she get some help. She was a little intimidated to ask the professor why the script didn’t run on her laptop, when she saw it running in class on the professor’s machine. I had some time, so we started a screen-sharing session on WhatsApp, which is amazing. Side note, I can’t believe I can talk with my daughter and watch her work on her computer in real time, with a few clicks of a button.

    In any case, she told me she was getting a file size error. She couldn’t load a 32MB file because the RStudio limit was 5MB. That seemed a little funny to me, as I’ve had some large files loaded into RStudio. My suspicion was that this error was masking another problem, but I watched my daughter work to see.

    What she had done:

    1. download a zip file
    2. extract this to a new folder in Documents
    3. open the R script in RStudio
    4. drag the text file for analysis into RStudio

    That seemed strange. In looking at her script, I realized there was likely a pathing issue. We ran a getwd() and then a setwd() to experiment. We got a different error, which looked to me like a collation/formatting issue. We added a fileEncoding parameter to read.delim() and the script started working. She was amazed and pleased.

    Explaining the technical aspects of this to an English(ish) student, and helping her to lightly understand what to do was interesting. She made notes in a file on using setwd(), changing backslashes from Explorer to forward slashes, and how to add the parameter to the various read commands. Ultimately, she was back to being productive, without learning much about R or technology, but having a path forward of things to try when something broke, with notes saved in a document.

    It reminded me of myself when I’m trying to fix something on the ranch I’ve never done before. I have to quickly learn what’s wrong, how to troubleshoot the extent of the issue, and decide how to move forward. Those troubleshooting skills are invaluable, and that’s likely the biggest thing my daughter learned. That’s what I think is a struggle for many people, in all fields, but certainly in technology.

    The next time you get a chance to help someone troubleshoot, work with them, explain your process, and help them improve for the future.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.