Running Ollama in your terminal allows you to start chatting with a local large language model (LLM) quickly. You won’t need API keys, cloud services, or ongoing costs. Ollama is a free, open-source tool that lets you download and run models directly on your machine. By following this guide, you’ll install Ollama, chat with local models from your terminal, and use them to power agentic coding tools:
Large language models traditionally require expensive API subscriptions and a constant internet connection. Ollama eliminates both requirements by running models directly on your hardware. Because everything runs locally, your prompts stay on your machine, and no per-token fees apply.
Get Your Cheat Sheet: Click here to download your free Ollama cheat sheet and keep the essential steps and commands for running LLMs locally at your fingertips.
Take the Quiz: Test your knowledge with our interactive “How to Use Ollama to Run Large Language Models Locally” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
How to Use Ollama to Run Large Language Models LocallyTest your knowledge of running LLMs locally with Ollama. Install it, pull models, chat, and connect coding tools from your terminal.
Prerequisites
To follow this guide, you’ll need the following software and hardware:
- macOS 14 Sonoma or newer, Windows 10 or newer, or a recent Linux distribution
- At least 8 GB of RAM, or 16 GB or more for larger models
- 5–16 GB of free disk space to store models
- Basic skills with the command line or terminal, including opening a terminal and running commands
No Python installation is required for this guide, and no prior experience with LLMs or AI is needed. If you want to integrate Ollama with Python after finishing here, check out How to Integrate Local LLMs With Ollama and Python.
Step 1: Install Ollama and Pull Your First Model
To quickly install Ollama on your operating system, run the following command based on your platform:
Once this command finishes, Ollama will be installed on your system.
Note: In some Linux distributions, you may need to install curl to download the installer and the zstd library for extraction. On Debian/Ubuntu, you can install them with the following command:
$ sudo apt update && sudo apt install curl zstd
Alternatively, you can download a dedicated installer for Windows and macOS. Visit Ollama’s download page to get the installer for those operating systems.
Note: Ollama has a GUI application for macOS and Windows users. This quick guide focuses solely on the command-line (CLI) tool. See Ollama’s app announcement if you want to explore that option.
After installation, you can verify that the CLI is available with the following command:
$ ollama -v
ollama version is 0.17.7
The Ollama service should be running in the background. Normally, you don’t need to start it manually. It runs on port 11434 by default. If you get a warning after running the command above, then you may need to run the background server manually:
$ ollama serve
