Downloading Ollama Models

00:00 Now that you have Ollama installed and running, you can now pull or download the required models. Ollama has a model library at ollama.com/library where you can browse and search for Ollama models.

00:14 But for now, I’ll just tell you that the models you’ll need for this course are llama3.2, which requires 2.0 gigs of disk space, and codellama, which needs 3.8 gigs.

00:26 So go to your terminal and run ollama pull llama3.2 and wait for the model to download. This might take a while. Then you can do ollama pull codellama and get that one too.

00:48 You can now try these out from the command line using the ollama run command. So ollama run llama3.2, that’s going to start a chat with this model.

01:02 I can send a message, hello, and that works. So press Ctrl+D to exit. And if that was successful, you’ve got enough started to move on to the next lesson.

01:15 But if you’re curious, I’m going to spend a bit of time now talking more about models and the Ollama model library.

01:22 The Ollama model library is located at ollama.com/library. And this is like the Python Package Index, but for open models that can be used with Ollama. So you can get to it by clicking on Models up here.

01:37 And you can view the popular models or sort by new or sort by the tool or the feature, the capability that the model can do. And to learn more about these capabilities, you can look at the documentation under Capabilities.

01:58 So you can search for models specific to a topic like coding or health, or we’re going to look for Llama, the Llama family of models, which is created by Meta.

02:12 So llama3.2 is their newest but smaller ones.

02:17 And there’s going to be a description here and some tags and a command to show you how to run this model. And specifically, we’re looking at this unique name identifier to target this model, which is llama3.2.

02:34 And that’s what we used earlier to pull the model. Now going down a little bit, we can see that there are different versions of this model. The main difference being the size.

02:47 So it shows the disk size here that you need, but also there are specific versions like 1B and 3B.

02:56 latest is going to pull the latest version of the model, which is just this 3B one. And that’s why I said we needed 2.0 gigs.

03:05 Let’s talk a little bit about model sizes now. So the size of models is measured in billions of parameters. These parameters are numeric weights that a model contains.

03:17 And the more parameters it has, the more capable a model is, the more advanced, but it’s also going to be slower and needs more RAM to run. As a general rule, you need more RAM in gigs than the size in billions.

03:33 For example, if you have eight gigs of RAM, you can run a 7B model, but it might be a little slow. So if you had more RAM or if you had a GPU to make use of, this can speed it up.

03:47 Then at the bottom, there’s a README,

03:50 which describes more about the model potentially, maybe compares it to other models. And then going back up to the models, you can do View all. And this actually shows you all of the different versions for a model.

04:03 Now these various versions have names with potentially more words and numbers and letters in it. And understanding these is a bit outside of the scope of this intro class, but look up fine-tuning to understand what instruct versus text versus code means.

04:21 And also look up quantization to see what this kind of code at the very end means. In the next lesson, you’ll install the ollama package for Python, and then you’ll be ready to start coding.