Running Ollama Locally on Linux with VS Code: A Practical Setup Guide

Laszlo Simon
Development Tools, AI, Backend
Page content

Running large language models locally has become a practical option for developers who want:

  • privacy-friendly AI assistance
  • offline coding support
  • full control over models and data
  • reduced dependency on cloud-based AI services

Ollama makes it easy to run local LLMs on Linux, and VS Code can integrate these models directly into the development workflow.

This article explains how to set up Ollama with VS Code and use a local coding model.

1. Installing Ollama on Linux

First, install Ollama on your Linux machine.

The official installer can be executed with:

curl -fsSL https://ollama.com/install.sh | sh

After installation, verify that Ollama is available:

ollama --version

Start the Ollama service if it is not running:

ollama serve

By default, Ollama exposes a local API endpoint:

http://localhost:11434

2. Download and Run a Coding Model

For programming tasks, a code-focused model is recommended.

Example:

ollama run qwen3-coder:30b

The first execution downloads the model locally.

Depending on the model size, this can require significant:

  • disk space
  • RAM
  • GPU memory (if available)

After the download finishes, the model can be used directly from the terminal.

Example:

>>> Explain this Java method

The response is generated completely locally.

3. Installing Ollama Support in VS Code

VS Code does not automatically know how to communicate with Ollama.

Install the Ollama for VS Code extension.

After installation:

  1. Open VS Code
  2. Go to Extensions
  3. Search for:
Ollama
  1. Install:
Ollama for VS Code

Why Not Use the Default Ollama Language Model Extension?

Some older VS Code extensions provided Ollama integration through a built-in language model provider.

However, the default Ollama language model integration is deprecated.

Using the dedicated Ollama for VS Code extension is recommended because it provides:

  • active development
  • better model management
  • direct Ollama integration
  • improved compatibility with current VS Code versions

4. Configure the Model in VS Code

After installing the extension:

  1. Open VS Code settings
  2. Configure the Ollama endpoint if required:
http://localhost:11434
  1. Select your local model:
qwen3-coder:30b

Now VS Code can send requests to your local Ollama instance.

5. Using Ollama as a Coding Assistant

Typical use cases:

Code Explanation

Example prompt:

Explain this Spring Boot service class

Code Review

Example:

Review this Java code and suggest improvements

Refactoring

Example:

Refactor this method following clean code principles

Architecture Discussions

Example:

Design a hexagonal architecture for this application

6. Hardware Considerations

Large coding models require significant resources.

For example:

qwen3-coder:30b

requires substantially more memory than smaller models.

Performance depends on:

  • available RAM
  • GPU VRAM
  • CPU performance
  • model quantization

If the model is too slow, consider smaller variants.

Example:

ollama run qwen3-coder:7b

or:

ollama run qwen3-coder:14b

7. Troubleshooting

Ollama is not responding

Check if the service is running:

ollama serve

Model not found

List installed models:

ollama list

Download missing models:

ollama pull qwen3-coder:30b

VS Code cannot connect

Verify that the Ollama API is available:

curl http://localhost:11434/api/tags

If this returns available models, the Ollama service is running correctly.

Final Thoughts

Running Ollama locally with VS Code provides a powerful development setup:

  • your code stays on your machine
  • no external API dependency
  • fast access to AI assistance
  • full control over models

For developers working with sensitive enterprise code, local LLMs can be an attractive alternative to cloud-based coding assistants.

With a capable machine, models like qwen3-coder:30b can provide useful support for:

  • code reviews
  • refactoring
  • architecture discussions
  • debugging

while keeping your development environment private.

Key Takeaways

  • Install Ollama locally on Linux
  • Use ollama run qwen3-coder:30b to start a coding model
  • Install the dedicated Ollama for VS Code extension
  • Avoid deprecated default Ollama language model integrations
  • Ensure VS Code connects to the local Ollama API
  • Choose model size based on available hardware