Voice of the DBA

Author: way0utwest

  • Finely Tuned Models

    I have no idea if this is true, but this post on X says that Thomson Reuters used information they’ve collected for decades to fine tune and train a model. They started with one of the qwen models and then spent $40 million to add their knowledge to the model. The post says this model is comparable to the Gpt5.5 and Sonnet 5 models. They have used 10% of their data, which spans over 100 years.

    I have many questions. But first, $40mm? How many tokens is USD$40mm? A quick calculation is trillions of tokens, and while I’m sure there is some payback if lots of their customers use the model for work, there’s also a compute cost every time they use the model. Perhaps they can fine-tune and train the model more efficiently over time, but I can’t see many individual companies spending this effort on training their own model.

    I also wonder what the time it took to train this. The story notes 2 years, but if I want to update this, then how much more effort. What’s the cost? This certainly reduces the dependence on the frontier models from Antropic/OpenAI/etc., but is this something that makes sense for other companies? If I choose to use the TR model, would it be cheaper than Sonnet 5? Maybe it uses less power, which is always good, but will T-R charge me less than Anthropic?

    I do think that fine-tuned models might make sense, especially if you can use a small language model and train it for a reasonable cost. Like $100,000, not millions. In that case, would it make sense to you? Would your company fund this themselves? Is this something that a trade group could do with support from multiple organizations? Or is the competition between companies so high that they can’t work together?

    I do think that AI has a lot of possibilities to assist humans, and focused models can be useful to solve specific problems, with a lot less cost than the best frontier models. I’m just not sure if the training effort is something that many are willing to put forth. It will be interesting to see how GenAI models evolve as the costs and resources needed by the frontier models continue to rise and smaller models become more capable.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Local Models Using GPUs with Ollama

    Can you use your GPUs when running a local model under Ollama? You can, and it really depends on how you run Ollama and what your hardware is.

    Ollama supports NVidia and AMD GPUs with some exceptions. You can read about their hardware support here. There are drivers required from the vendors, and configuration, but it can help with performance.

    Of course, GPUs aren’t cheap.

    I have an NVidia GeorForce RTX 2060, which is listed as being supported with a compute capability of 7.5. I need a driver version of 550+. I’m supported with driver version 560.94.

    2026-09_0096

    However, I need NVidia CUDA drivers installed for this to work. Those don’t really install on Windows, so I have to install them in the WSL subsystem, with the Linux install guide for this to work on my system. I don’t use Docker, I have Rancher for reasons …, so I haven’t done this so far.

    If you use Docker Desktop, there is native support for GPUs.

    On Macs, there is native support.

    If you are looking to run local models seriously, you’ll likely want either a dedicated machine, or you will go the Docker Desktop route as an individual. In an org, you might pick a dedicated server and allow multiple users to connect and get answers in a secure, controlled way.

  • Creating a Local Model Chatbot on Windows

    After my session at VSLive last week, I had a few questions from the audience. I’m adding some of these into blogs, and this was one:

    How do I run the docker compose to get a local chatbot?

    I didn’t include the dockerfile in a a repo, mostly because it’s simple, but just to make it easy, it’s in this zip: ollama-docker-compose.zip.

    To start this, I’ll assume you have Docker (or equivalent), but no images. Create a folder structure like this:

    2026-09_0091

    Put the docker-compose.yml in here. Then open this in a CLI and run: docker compose up

    You should see something like this. The images start pulling. This is slow, but once the download is there, it should be smoother.

    2026-09_0090

    Once the images are pulled, the containers should start and you should get some output in your CLI window. Something like this. Note the first part of the output is the container.

    2026-09_0093

    Once it’s up, it will first ask you for an admin account. Enter your credentials, as I’ve done. I didn’t use this email, but it doesn’t matter which one you does. It’s local!

    2026-09_0094

    You should then get the chatbot interface. Note, like me, the first time you run this, there’s no model, so use the post below to add your model and get started.

    2026-09_0095

    I have a few posts on this that might help. Note that these are from the past the the GUI has changed slightly, but the items are still in the GUI, just moved.

  • Make It Routine

    The first one is hard. The rest are boring.

    I hear this statement from someone recently, and it sounds like something a technical person would do with a task. It’s often been a goal of mine, or maybe a direction to aim, that I want to make things easier with some script or automation. Sometimes I think it’s more aspirational than actual. I might want to make most things routine and automated, but that doesn’t always work out.

    The first time I tackle something, it should be hard. It’s new work. It’s a new process/code/thought/action/etc. It’s unfamiliar and I spend more time on it than I want. After that, I’d hope I could repeat the thing again in much less time. That’s the goal, and that’s what we aim for in a lot of DevOps work. Make the things we think are hard, less hard. Make them boring by codifying things, using automation, have the computer replicate the things.

    I think the second time I tackle a task is often hard as well. How often have you tried to reproduce something you did and can’t quite get it? Heck, I now depend (and use) SQL History constantly because I will write some code, change it a bunch, and then realize that I can’t reproduce the version that did the thing (or broke the thing) I was working on. I need to rewind things and figure them out again. Usually by the 4th time I’ve done something, it’s starting to become easier. By the 10th its boring.

    Unless I’m playing guitar, in which case, some things take a few more reps than 10.

    I’ve had plenty of developers say never repeat yourself. If you can automate it, you should. In practice, that’s hard. Sometimes I’m unsure of whether I’ll do something again, or often enough to spend the time automating it. There are also times I’m not sure it’s worth the effort. I spent a day once trying to automate a bunch of Outlook appointments, only to realize the whole Office API and deluge of information out there made this much harder than the 15 minutes a year I spend putting in all my Database Weekly reminders.

    I don’t want to discourage you from automating things and making them routine or boring. My database deployments ought to be routine. The daily checks should be so boring and automated that I don’t bother with them because I know the machine is doing the work and will let me know if there is something I should examine. The efforts to refresh dev dbs, or respond to audit requests, or even reset a password ought to be boring and easy. Some of us build ways to smooth these tasks and ease our jobs, and some of us treat every one as an ad hoc thing that we do over and over.

    The over and over stuff is going to be taken over by AI. Maybe soon, maybe in a few years, but a lot of simple stuff that you keep doing, that mindless, tedious stuff that doesn’t require a lot of thought, is going to be handled by AI agents. Either you’ll direct them, or your boss will ask someone else to do it after you leave. AI can handle things like figuring out disks are full and cleaning out old log files, shrinking disks, archiving things, and then writing scripts (and scheduling them) to prevent issues. If that’s your job, your days are numbered.

    Make things routine by thinking about the pattern, how we could reduce or eliminate a lot of labor, and how we can use a computer to handle them. Even better, learn how to guide an AI to do that work and prove your worth to your employer.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.