AI Model Size, Parameters and Download

At SQL Saturday Boston 2025, I gave a presentation on local LLMs and there was a great question that I wasn’t sure about. Someone asked about the download size vs the model size as listed on Hugging Face (or elsewhere). It was a good questions, and I assumed the parameter size (258M, 7B, etc.) relates to the download size or size on disk.

I did a few searches and this was a great article on LinkedIn, called The Intergalactic Guide to LLM Parameter Sizes. In it, there’s a quick guide:

  • Tiny (1-3B parameters): 1-2GB on disk
  • Small (4-8B parameters): 3-5GB
  • Medium (10-15B): 8-15GB
  • Large (30-70B): 20-40GB
  • Enormous (100B-200B): 60-150GB
  • Apocalypse-Inducing (500B+): 300GB+

That’s a good rule of thumb, and a good thing to know. Now, the storage isn’t that important, but the power consumption is something.

I have a laptop which has these specs:

  • 12 core Ultra 7 Intel CPU, 2.1GHz (base)
  • 32GM RAM

I asked a simple question of my local model (Ollama, mistral 7b), so a small model, and I see this in task manager

2025-09_line0013

The power consumption in the guide says a small desk fan and any modern laptop can run it. I was playing with this on a plane and could see my battery going down. I don’t think that the storage is a problem on most modern laptops/desktops, though certainly you want a T/S/M/L model running. The power, however, can be a challenge, especially if you don’t have a good GPU. My laptop has an Intel NPU (AI Boost) designed to help AI stuff though I don’t know if the container uses it. I also have a GPU from Intel, but still, this GenAI stuff uses power.

Be careful on a laptop that isn’t plugged in if you need to get a lot of work done.

Posted in Blog | Tagged , | Comments Off on AI Model Size, Parameters and Download

Back in the EU

I flew to Amsterdam last night and hopefully by the time this publishes I’ll be at my hotel, catching up on messages and prepping for another Simple Talk Podcast recording.

This is my fourth trip across the water this year, with one more coming at the end of next month. It’s a lot of travel, but it’s also a lot of fun. I meet so many interesting people, learn new things about other cultures, and I really, really enjoy the food in Europe. It’s so much fresher and better than the US (on average).

This week is a day in the Redgate Amsterdam office, two days at the PASS On Tour event, and Friday at the Redgate Summit. If you’re anywhere near me, register and come join me. The events in New York and Dallas were informative, invigorating, and exciting. It will be the same here.

Please feel free to stop me and say hi if you’re attending. I always enjoy meeting people and talking about data.

Posted in Blog | Tagged , , , | Comments Off on Back in the EU

Choosing an AI in Sept 2025

Some of you out there are positive about AI and looking to use them. Some of you aren’t too thrilled with the tools and might avoid using them. I think that not learning to harness the power of an AI is a mistake. This technology is going to change the world in many ways and you need to learn how it can help you.

You also should learn where the sharp edges are as there are some very, very sharp edges.

Which AI should you use? That’s an interesting question. If you listened to my podcast interview with Tom Hodgson (YouTube, Spotify, Apple), he prefers Gemini as a developer at Redgate. I think that’s one of the more popular models to choose from as a developer. I see lots of people writing about their experiences using Gemini to code.

It’s recommended (with others) in this article on choosing models, which is a topic that I think is important to keep an eye on. We often like to build some skill and trust with a tool, developing muscle memory that helps us work efficiently with the tool. In the AI world, things are changing constantly, though not so much that I would recommend jumping on the latest release of a new model.

Instead, I’d recommend you stick with a tool and, perhaps every 3-4 months, you experiment with another model and see if you like it. Perhaps send some duplicate queries to your current tool and a second one. As much as we’d like to say this one LLM is best, there is some personal preference we will have for different tools. I find LLMs to be non-deterministic, so you want to work with a tool in which you have some trust.

The current top models that most people look at are the OpenAI gpt models, the Claude models, and the Gemini ones. It is hard to keep track of these, so I’ll try to simply them.

For many of us in the Microsoft space, we are used to OpenAI and the ChatGPT models. There is the gpt-4o and gpt-5o models in use by many people. I haven’t found these as useful to me for SQL coding. The way they respond slightly grates on me, so I have learned to check in various tools to switch the mode away from gpt-40 (often the default). They might work well for other code, but they seem to lean towards MySQL, which annoys me.

Claude (from Anthropic) is my preferred model. I usually choose a Sonnet model (3.7 or 4) and have them work well. I like the structure of Sonnet output and find it helps me get database work done pretty well. It’s not perfect, and it’s certainly not always correct, but it’s good. I’ve had it solve a number of problems, and it’s the one I often use in my AI Experiments posts.

Gemini (from Google) is the one I’ve used the least, but it’s the one several developer friends like. It’s on my list to experiment with this one a bit and see how it works. There are various types of v2.5 models here to choose from.

Depending on what types of work you do, you might like other models. I primarily deal with text, so I haven’t spent much time with models that work with audio/video/pictures, though my wife has been playing with AI video editing. She doesn’t know which model she used, but she appreciated the LLM help in producing a sale video for a horse recently. It was way better than anything she had done on her own.

I’ve tried Perplexity (sonar), and very lightly, grok, but they haven’t attracted me for some reason. I tried a local DeepSeek R1 model, but it wasn’t smart enough (or quick enough) to get much done. If you’re in doubt, just ask the AI which model is running. Most of them let you switch models easily, so give it a try.

And let me know what works for you and why. I have found different models working differently, so pay attention and share what you learn.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.

Posted in Editorial | Tagged | Comments Off on Choosing an AI in Sept 2025

SQL Saturday Boston Slides and Code

Thanks to everyone who attended my sessions today at SQL Saturday Boston 2025. I’ve got resources listed below

Slides

Here are the slides from my two sessions

Code

I’ve got code in repos, so feel free to fork, extend, send me PRs to fix my mistakes.

Posted in Blog | Tagged , , | Comments Off on SQL Saturday Boston Slides and Code