An Unlimited Token Budget

·

I don’t like unbounded situations. I’m always wary of “unlimited” or “lifetime” offers, as I think they’re entirely too optimistic. A few people on an endless cruise recently learned this. An unlimited offer is only as good as the company that provides it, and many companies have either gone bankrupt, failed, or had to alter the terms of their amazing offer (T-Mobile, American, Red Lobster, Movie Pass).

Similarly, I’ve always been worried about giving developers unlimited resources, especially in places like Cloud CI systems. I’ve seen how often (I and other) developers will make commits, submit PRs, amend those, make new commits, etc. I don’t like the idea of potential uncapped builds or the costs they entail. Similarly, the bursting that some monitoring products offer in the cloud to capture all data can sometimes result in extremely high bills. Especially when it’s easy for users to add systems or change configurations that create additional compute or storage requirements. Vendors love this, as they get increased billing without the hassle of a sales effort.

I saw Cisco recently offer a developer unlimited tokens. This isn’t quite as open-ended as it appears, and I do like the approach of a short timebox in which to experiment. A young developer has 4 weeks to create a PoC (proof-of-concept) showing he can use AI to rewrite legacy Cisco code. That’s a place AI might shine, and it’s well scoped. We know how their code works; we can write tests around it, and there’s likely a lot of training data available on how the systems work. The goal is 98% of their code rewritten in 2 years. I don’t know what the estimate would be for a rewrite with humans, but I’d guess this would be in man-years, maybe man-decades.

It’s a challenge from the President, and if it works, well, I don’t know. If this is to prove AI can work well and inspire other engineers, maybe that’s good. Maybe it’s worth millions of dollars. And at unlimited token budgets, with creative engineers able to spin up hundreds (or thousands) of agents, it might be millions in cost.

Per year.

Or per month.

I am rooting for this to continue past 4 weeks as it’s a great case study. Can a human rewrite lots of code effectively with agents? Can a human truly manage a swarm of agents with a well known and scoped target? There have been plenty of people experimenting here, but they are often doing so for research or fun, without a real goal other than to see what works. Cisco is making a business project here, and the results will be fascinating. Even a failure will teach us something.

Even if it’s only about AI finance impacts from an engineer without any quotas.

This will be an interesting story, and I hope Cisco provides updates and results. I suspect those reports might be light on details, but I will be following along.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.