Tag: AI

  • Adding a Local Model to Ollama through the GUI

    I saw some good reviews of the small gemma3 model in a few places and wanted to try it locally. This

    If you want to get started, read my post on setting up a Local LLM. This post gives an alternative to connecting to the container from the CLI and running a command.

    This is part of a series of experiments with AI systems.

    Adding a Model

    Using the Ollama-WebUI server, it’s easy to add models. I wrote about setting up the UI and I’m running that as my interface to the local model. I created a new container for this post, so when I see the interface, there are no models to choose from. You can see this below.

    2025-09_0090

    I need a model because if I try to ask a question, I get an error that the model is not selected. You can see the error at the top in this image and the query at the bottom.

    2025-09_0091

    If I click the gear icon by the model drop down, I get a list of settings. One of these is “Models”, which is third on the left.

    2025-09_0092

    I click Models and get this list. As you can see I have no models. Notice I need a model name to enter. However, where the cursor is below, there is a “click here” for a list of models.

    2025-09_0094

    This opens the Ollama site at https://ollama.com/library

    2025-09_0095

    If I scroll down, I see the gemma3 model. If I click this I see a few different ones. I am not a big fan of latest, but this is a test. I decided to just grab it since I’m not programming things.

    2025-09_0097

    If I enter this in my WebUI and click download, it starts downloading.

    2025-09_0098

    Progress is shown, and it’s not quick. I went on to other work and this is the view about 15 minutes later.

    2025-09_0100

    Once this is complete, I close settings and see my model in the list. Since this is the only model for this container, I’ll click “set as default”.

    2025-09_0102

    Now I can get my fun fact:

    2025-09_0103

    Now I have a safe, secure, local model to use. If you want to see this run in real-time, check out this video:

  • AI Model Size, Parameters and Download

    At SQL Saturday Boston 2025, I gave a presentation on local LLMs and there was a great question that I wasn’t sure about. Someone asked about the download size vs the model size as listed on Hugging Face (or elsewhere). It was a good questions, and I assumed the parameter size (258M, 7B, etc.) relates to the download size or size on disk.

    I did a few searches and this was a great article on LinkedIn, called The Intergalactic Guide to LLM Parameter Sizes. In it, there’s a quick guide:

    • Tiny (1-3B parameters): 1-2GB on disk
    • Small (4-8B parameters): 3-5GB
    • Medium (10-15B): 8-15GB
    • Large (30-70B): 20-40GB
    • Enormous (100B-200B): 60-150GB
    • Apocalypse-Inducing (500B+): 300GB+

    That’s a good rule of thumb, and a good thing to know. Now, the storage isn’t that important, but the power consumption is something.

    I have a laptop which has these specs:

    • 12 core Ultra 7 Intel CPU, 2.1GHz (base)
    • 32GM RAM

    I asked a simple question of my local model (Ollama, mistral 7b), so a small model, and I see this in task manager

    2025-09_line0013

    The power consumption in the guide says a small desk fan and any modern laptop can run it. I was playing with this on a plane and could see my battery going down. I don’t think that the storage is a problem on most modern laptops/desktops, though certainly you want a T/S/M/L model running. The power, however, can be a challenge, especially if you don’t have a good GPU. My laptop has an Intel NPU (AI Boost) designed to help AI stuff though I don’t know if the container uses it. I also have a GPU from Intel, but still, this GenAI stuff uses power.

    Be careful on a laptop that isn’t plugged in if you need to get a lot of work done.

  • Choosing an AI in Sept 2025

    Some of you out there are positive about AI and looking to use them. Some of you aren’t too thrilled with the tools and might avoid using them. I think that not learning to harness the power of an AI is a mistake. This technology is going to change the world in many ways and you need to learn how it can help you.

    You also should learn where the sharp edges are as there are some very, very sharp edges.

    Which AI should you use? That’s an interesting question. If you listened to my podcast interview with Tom Hodgson (YouTube, Spotify, Apple), he prefers Gemini as a developer at Redgate. I think that’s one of the more popular models to choose from as a developer. I see lots of people writing about their experiences using Gemini to code.

    It’s recommended (with others) in this article on choosing models, which is a topic that I think is important to keep an eye on. We often like to build some skill and trust with a tool, developing muscle memory that helps us work efficiently with the tool. In the AI world, things are changing constantly, though not so much that I would recommend jumping on the latest release of a new model.

    Instead, I’d recommend you stick with a tool and, perhaps every 3-4 months, you experiment with another model and see if you like it. Perhaps send some duplicate queries to your current tool and a second one. As much as we’d like to say this one LLM is best, there is some personal preference we will have for different tools. I find LLMs to be non-deterministic, so you want to work with a tool in which you have some trust.

    The current top models that most people look at are the OpenAI gpt models, the Claude models, and the Gemini ones. It is hard to keep track of these, so I’ll try to simply them.

    For many of us in the Microsoft space, we are used to OpenAI and the ChatGPT models. There is the gpt-4o and gpt-5o models in use by many people. I haven’t found these as useful to me for SQL coding. The way they respond slightly grates on me, so I have learned to check in various tools to switch the mode away from gpt-40 (often the default). They might work well for other code, but they seem to lean towards MySQL, which annoys me.

    Claude (from Anthropic) is my preferred model. I usually choose a Sonnet model (3.7 or 4) and have them work well. I like the structure of Sonnet output and find it helps me get database work done pretty well. It’s not perfect, and it’s certainly not always correct, but it’s good. I’ve had it solve a number of problems, and it’s the one I often use in my AI Experiments posts.

    Gemini (from Google) is the one I’ve used the least, but it’s the one several developer friends like. It’s on my list to experiment with this one a bit and see how it works. There are various types of v2.5 models here to choose from.

    Depending on what types of work you do, you might like other models. I primarily deal with text, so I haven’t spent much time with models that work with audio/video/pictures, though my wife has been playing with AI video editing. She doesn’t know which model she used, but she appreciated the LLM help in producing a sale video for a horse recently. It was way better than anything she had done on her own.

    I’ve tried Perplexity (sonar), and very lightly, grok, but they haven’t attracted me for some reason. I tried a local DeepSeek R1 model, but it wasn’t smart enough (or quick enough) to get much done. If you’re in doubt, just ask the AI which model is running. Most of them let you switch models easily, so give it a try.

    And let me know what works for you and why. I have found different models working differently, so pay attention and share what you learn.

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.

  • Getting More Time from AI

    As I get older, I find that time is the most valuable commodity I have. It’s the one thing that I need more of, but I can’t get it. Even if I find more efficient ways of doing things, it seems there’s an endless list of things at work and on the ranch that need to be done.

    It seems to be the same for most of my friends in other jobs, whether in medicine, law, or any other position; they’re often overloaded with more work than they can get done in a week. Arguably, it’s not all important work, as sometimes we might tackle a task, only to have our boss throw the work away or delay the project. That’s annoying, but I also understand priorities change.

    Mostly, I understand people ask for a lot of things they don’t really need, but because it’s easy to ask. Over time, they may forget about their request or realize they don’t need it anymore. So I tend to do the work I’m asked and then not worry about if it’s really needed.

    However, that’s not what this piece is about. Let’s say that you have a useful AI Assistant that can truly produce better code, faster than you can. You trust the coding agent (under your guidance), and it saves you time. Let’s say that this agent reduces your coding load by 30%. Forget the issues with the agent stealing your joy.

    A number of studies show that the average developer only codes around 2-3 hours a day. Some might say 4, but that’s just 50% of your workday (I hope). If you save 30% of that time, then you’re saving 1:20 a day, 6:40 a week. That’s a nice chunk of change, but what will you do with the time? Or maybe more importantly, what will your boss expect you to do?

    A lot of management might see 30% and expect you to get 30% more coding done (of 40 hours), asking for 12 hours more of work. I see that a lot as management somehow things that developers work 8 hours a day and should get 8 hours worth of code written. When that doesn’t happen, and it never does, they often schedule more meetings to check on your progress. Hence to 2-3 hours worth of actual work.

    The smarter group of managers might think that you could get 5 hours more work done (30% of 3/day), and the really smart group of managers might use that time to improve the coding skills of the staff. That’s more than a 10% allocation of time and if focused, could help developers write more efficient code (especially SQL), improve their secure coding knowledge, and perhaps even tackle some technical debt with refactoring of existing problem areas. Maybe you could learn to better judge AI-produced code.

    I know that many people are skeptical of AI assistance in coding. I think it’s a tricky thing, but even if the AI doesn’t produce production code, but allows me to conduct a lot of quick experiments that can prove out a hypothesis, I think it can help developers become more productive.

    If they learn to use it as a tool and management doesn’t expect it will turn every developer into a 10x (or even 2x) engineer.

    If you had more time, what would you do in your work day?

    Steve Jones

    Listen to the podcast at Libsyn, Spotify, or iTunes.

    Note, podcasts are only available for a limited time online.