Add ollama instructions

Bootstrap: docker

From: ollama/ollama

%runscript

    # The default command to run when the container is executed

    exec /usr/bin/ollama "$@"

# Ollama Singularity Image at PIC

This guide provides the instructions to build and run an **Ollama Singularity (Apptainer) image** suitable for using Ollama inside PIC's HTC infrastructure, in particular inside the jupyter.pic.es service.

## Prerequisites

1.  **Singularity/Apptainer:** You must have the Singularity or Apptainer container runtime installed on your machine.

2.  **NVIDIA GPU (Optional):** If you intend to use GPU acceleration, your host system must have the correct NVIDIA drivers and the Apptainer/Singularity must be configured to use them (via the `--nv` flag).

## 1. Building the Singularity Image

We convert the official Ollama Docker image directly into the Singularity Image Format (SIF).

```bash

# Build the SIF file by pulling and converting the official Docker image

singularity build ollama.sif Ollama.def

```

Upon successful completion, an image file named ollama.sif will be created in your current directory.

## 2. Running Ollama

### 2.1. The `run_ollama.sh` script

The run_ollama.sh script provides a user-friendly interface to manage the Ollama service within the container, handling crucial steps like environment variables, volume binding, and port forwarding.

The script accepts the following keyword arguments:

* --model-folder: Host path where Ollama will download and store models. Must be persistent.

* --port: The host port on which the Ollama API service will listen, defaults to 11434

* --command: The Ollama subcommand and arguments to execute (e.g., serve, run llama3)

### 2.2. Execution Examples

A. Start the Ollama API Server (Default Command)

This starts the Ollama API service, which is necessary for connecting external clients (like Python programs or web UIs). Since the --command is omitted, it defaults to serve.Bash# Example using default port 11434

```

./run_ollama.sh --model-folder /scratch/user/ollama_storage

```

B. Start the Server on a Custom Port

Specify a custom port (e.g., 8080) to avoid port conflicts.Bash

```

./run_ollama.sh --model-folder /scratch/user/ollama_storage --port 8080 --command serve

```

C. Run a Model Interactively (CLI Mode)

Once you have an ollama server running, in a separate terminal, you can use the --command argument to execute an interactive session to download and chat with a model. Note the quotes around commands with arguments (like run llama3)

```

./run_ollama.sh --model-folder /scratch/user/ollama_storage --command "run llama3"

```

## Using ollama from a python notebook

Once you have an ollama server running you can interact with it from a jupyter notebook.

You need to have the `ollama` package installed.

```

pip install ollama

```

And then you can interact with the ollama server:

Initialize the ollama client

```python

from ollama import Client

client = Client(

  host='http://localhost:11434',

  headers={'x-some-header': 'some-value'}

)

```

Pull a model:

```python

client.pull('llama3')

```

Documentation [here](https://github.com/ollama/ollama-python)

 No newline at end of file

%% Cell type:code id:d261b789-be7e-49ba-8a61-ca5a758bac07 tags:

``` python

from ollama import Client

client = Client(

  host='http://localhost:11434',

  headers={'x-some-header': 'some-value'}

)

```

%% Cell type:code id:df14ec88-ac24-4b57-badc-8d7e22498c09 tags:

``` python

client.pull('llama3')

```

%% Output

    ProgressResponse(status='success', completed=None, total=None, digest=None)

%% Cell type:code id:a6433ae1-b4c8-4480-b9eb-3d9f7e3d1718 tags:

``` python

response = client.chat(model='llama3', messages=[

  {

    'role': 'user',

    'content': 'Why is the sky blue?',

  },

])

```

%% Cell type:code id:758bea87-ff74-4a50-b8e4-f6cd58718dd8 tags:

``` python

response

```

%% Output

    ChatResponse(model='llama3', created_at='2025-10-20T13:14:32.210551711Z', done=True, done_reason='stop', total_duration=10509994904, load_duration=4629343151, prompt_eval_count=16, prompt_eval_duration=62667190, eval_count=363, eval_duration=5515491379, message=Message(role='assistant', content="A classic question!\n\nThe short answer: The sky appears blue because of a phenomenon called Rayleigh scattering, which occurs when sunlight interacts with tiny molecules of gases in the Earth's atmosphere.\n\nHere's a more detailed explanation:\n\n1. **Sunlight**: When the sun emits light, it produces a broad spectrum of colors, including all the colors of the rainbow.\n2. **Atmosphere**: As this sunlight travels through space to reach us, it encounters the thin gases that make up our atmosphere, such as nitrogen (N2) and oxygen (O2).\n3. **Rayleigh scattering**: When these gas molecules collide with the light, they scatter shorter wavelengths of light more than longer wavelengths. This is known as Rayleigh scattering, named after the British physicist Lord Rayleigh, who discovered it in 1871.\n4. **Blue light**: The shorter wavelengths of light that are scattered most effectively are in the blue and violet parts of the spectrum (around 450-495 nanometers). This is why the sky appears blue during the daytime, as the blue light is scattered in all directions and reaches our eyes from all angles.\n\nOther factors can also influence the color of the sky:\n\n* **Dust and water vapor**: Tiny particles in the atmosphere, such as dust, smoke, or water vapor, can absorb or scatter certain wavelengths of light, making the sky appear more hazy or orange.\n* **Clouds**: Clouds can reflect or scatter sunlight, changing its apparent color. Thicker clouds can make the sky appear white or gray, while thinner clouds may produce a range of pastel colors.\n\nIn summary, the blue color we see in the sky is primarily due to the scattering of shorter wavelengths of light by the tiny molecules of gases in our atmosphere, known as Rayleigh scattering.", images=None, tool_calls=None))

%% Cell type:code id:709d61da-38b0-423f-9aa7-689d0795d4f4 tags:

``` python

```

#!/bin/bash

# --- Script Configuration ---

# Default values

DEFAULT_PORT="11434"

DEFAULT_COMMAND="serve"

SIF_FILENAME="ollama.sif"

# Initialize variables

PORT="$DEFAULT_PORT"

MODEL_FOLDER=""

COMMAND="$DEFAULT_COMMAND"

SIF_PATH=""

# --- Argument Parsing (Keyword/Long-form) ---

# Process keyword arguments

while [[ "$#" -gt 0 ]]; do

    case "$1" in

        --port)

            # Port is optional, if provided, use it

            PORT="$2"

            shift # Skip argument name

            shift # Skip argument value

            ;;

        --model-folder)

            # Model folder is mandatory

            MODEL_FOLDER="$2"

            shift # Skip argument name

            shift # Skip argument value

            ;;

        --command)

            # Command to pass to Ollama, defaults to 'serve'

            # Note: The command can contain spaces, so we grab everything until the next keyword argument

            # For simplicity, this script expects the full command string in $2

            COMMAND="$2"

            shift # Skip argument name

            shift # Skip argument value

            ;;

        *)

            echo "Unknown parameter passed: $1"

            echo "Usage: $0 --model-folder <path> [--port <number>] [--command <ollama_subcommand_and_args>]"

            echo "Example (Server): $0 --model-folder /data/models --command serve"

            echo "Example (Run Model): $0 --model-folder /data/models --command 'run llama3'"

            exit 1

            ;;

    esac

done

# --- Validation and Environment Setup ---

# 1. Check for mandatory 'model-folder'

if [ -z "$MODEL_FOLDER" ]; then

    echo "ERROR: Missing mandatory argument: --model-folder"

    echo "Usage: $0 --model-folder <path> [--port <number>] [--command <ollama_subcommand_and_args>]"

    exit 1

fi

# Convert model folder path to absolute path for reliable binding

MODEL_FOLDER_ABS="$(realpath "$MODEL_FOLDER")"

# 2. Determine the script's directory and the SIF image path

SCRIPT_DIR="$(dirname "$(realpath "$0")")"

SIF_PATH="${SCRIPT_DIR}/${SIF_FILENAME}"

# Check if the SIF file exists

if [ ! -f "$SIF_PATH" ]; then

    echo "ERROR: Singularity image not found at expected path: $SIF_PATH"

    echo "Make sure '$SIF_FILENAME' is in the same directory as this script."

    exit 1

fi

# 3. Set environment variables for Apptainer (Singularity)

# APPTAINER_OLLAMA_HOST configures the host/port *inside* the container

# Use 0.0.0.0 to listen on all interfaces inside the container

export APPTAINERENV_OLLAMA_HOST="0.0.0.0:${PORT}"

# APPTAINER_OLLAMA_MODELS configures the folder where Ollama stores models *inside* the container

# We set this to a standard, consistent location that we will bind-mount.

export APPTAINERENV_OLLAMA_MODELS=${MODEL_FOLDER_ABS}

# --- Execution ---

echo "Starting Ollama service..."

echo "--------------------------------------------------------"

echo "Ollama SIF Image: $SIF_PATH"

echo "Host Port:        $PORT"

echo "Model Folder:     $MODEL_FOLDER_ABS"

echo "Ollama Command:   ollama $COMMAND"

echo "--------------------------------------------------------"

# Run the Apptainer image

apptainer exec \

    --nv \

    --writable-tmpfs \

    --bind "${MODEL_FOLDER_ABS}:${MODEL_FOLDER_ABS}:rw" \

    "$SIF_PATH" \

    ollama $COMMAND