Can I use the ollama locally without internet?

[quzhixue-Kimi]

🚀 The feature

Hi there,

May I ask how could I use my Ollama model without internet?

Steps:

1. I have installed all the dependencies in my local ubuntu server with internet. (python packages, neo4j docker image and my python code)

2. Copied these code and docker image to a new ubuntu server without internet and tried to run my test code:

from mem0 import Memory

config = {
"graph_store": {
"provider": "neo4j",
"config": {
"url": "neo4j://10.100.xx.xx:7687",
"username": "neo4j",
"password": "neo4j"
},
"llm" : {
"provider": "ollama",
"config": {
"model": "llama3.2:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.100.xx.xx:11434",
},
}
},
"llm": {
"provider": "ollama",
"config": {
"model": "llama3.2:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.100.xx.xx:11434",
},
},
"embedder": {
"provider": "ollama",
"config": {
"model": "nomic-embed-text:latest",
"ollama_base_url": "http://10.100.xx.xx:11434",
},
},
"version": "v1.1"
}

m = Memory.from_config(config)
m.add("I'm visiting Paris", user_id="john")
memories = m.get_all(user_id="john")
print(memories)

m.delete_all(user_id="alice123")

m.add("I like going to hikes", user_id="alice123")
m.add("I love to play badminton", user_id="alice123")
m.add("I hate playing badminton", user_id="alice123")
m.add("My friend name is john and john has a dog named tommy", user_id="alice123")
m.add("My name is Alice", user_id="alice123")
m.add("John loves to hike and Harry loves to hike as well", user_id="alice123")
m.add("My friend name is kimi and kimi is a F1 fans.", user_id="alice123")
m.add("So, kimi and john are friends.", user_id="alice123")
friends_result = m.search("Who are my friends?", user_id="alice123")
print(friends_result)

The above code followed these two links as below:
https://docs.mem0.ai/open-source/graph_memory/overview of tabs Advanced (Custom LLM) and https://docs.mem0.ai/examples/mem0-with-ollama

The llama3.2 and nomic-embed-text models have been imported Ollama:
(mem0) byxf@gpu:~/workspaces/images/mem0$ ollama list
NAME ID SIZE MODIFIED
llama3.2:latest a80c4f17acd5 2.0 GB 2 months ago
nomic-embed-text:latest 0a109f422b47 274 MB 2 months ago

3. When I ran my test code, there is one error occurred as below:

Traceback (most recent call last):
File "/home/byxf/workspaces/images/mem0/mem0_with_neo4j.py", line 41, in
m = Memory.from_config(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/mem0/memory/main.py", line 63, in from_config
return cls(config)
^^^^^^^^^^^
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/mem0/memory/main.py", line 37, in init
self.embedding_model = EmbedderFactory.create(self.config.embedder.provider, self.config.embedder.config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/mem0/utils/factory.py", line 56, in create
return embedder_instance(base_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/mem0/embeddings/ollama.py", line 32, in init
self._ensure_model_exists()
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/mem0/embeddings/ollama.py", line 40, in _ensure_model_exists
self.client.pull(self.config.model)
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/ollama/_client.py", line 421, in pull
return self._request(
^^^^^^^^^^^^^^
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/ollama/_client.py", line 177, in _request
return cls(**self._request_raw(*args, **kwargs).json())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/byxf/miniforge3/envs/mem0/lib/python3.12/site-packages/ollama/_client.py", line 122, in _request_raw
raise ResponseError(e.response.text, e.response.status_code) from None
ollama._types.ResponseError: pull model manifest: Get "https://registry.ollama.ai/v2/library/bge-m3/manifests/latest";: dial tcp: lookup registry.ollama.ai on 127.0.0.53:53: server misbehaving

It is appreciated that you may guide me how to disable the checking model feature, or could you please add this feature?

BR
Kimi

Motivation, pitch

Since, some companies will not allow to download or install open-source models or software in these Linux servers directly.
These open-source models or software must be downloaded and installed in a virtual env in test server with internet (I just use the miniforge to make the python virtual env and install these dependencies), and then coping these virtual env directly to the prod Linux server (prod Linux server has no internet, could not use apt, pip command to install any dependencies)
So, could you please just check the models existing in local disk or not, and provide a warning message if not.

BR
Kimi

[quzhixue-Kimi]

Hi there,

I tried to edit the code to disable the checking local models. I really have interest in the "graph_store" feature with neo4j, but
One issue is below:

my config is
config = {
"graph_store": {
"provider": "neo4j",
"config": {
"url": "neo4j://10.162,xxx,xxx:7687",
"username": "neo4j",
"password": "neo4j"
},
"llm" : {
"provider": "ollama",
"config": {
"model": "llama3.1:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.162.xxx.xxx:11434/",
},
}
},
"vector_store": {
"provier": "qdrant",
"config": {
"host": "10.162.xxx.xxx",
"port": "6333",
"embedding_model_dims": 768,
"collection_name": "mem0"
}
},
"llm": {
"provider": "ollama",
"config": {
"model": "llama3.1:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.162.xxx.xxx:11434/",
},
},
"embedder": {
"provider": "ollama",
"config": {
"model": "bge-m3:latest",
"ollama_base_url": "http://10.162.xxx.xxx:11434/",
"embedding_dims": 768
},
},
"version": "v1.1"
}

I found that these data could be inserted into qdrant, but not in neo4j. Is that the "vector_store" and "graph_store" conflict with each other?

And, I tried to just use "graph_store" with ollama locally as below:

config = {
"graph_store": {
"provider": "neo4j",
"config": {
"url": "neo4j://10.162,xxx,xxx:7687",
"username": "neo4j",
"password": "neo4j"
},
"llm" : {
"provider": "ollama",
"config": {
"model": "llama3.1:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.162.xxx.xxx:11434/",
},
}
},
"llm": {
"provider": "ollama",
"config": {
"model": "llama3.1:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.162.xxx.xxx:11434/",
},
},
"version": "v1.1"
}

But, it failed to get the embedding from the open ai due to no internet.

Finally, I tried to just use the "vector_store" only as below, it ran successfully:

config = {
"vector_store": {
"provier": "qdrant",
"config": {
"host": "10.162.xxx.xxx",
"port": "6333",
"embedding_model_dims": 768,
"collection_name": "mem0"
}
},
"llm": {
"provider": "ollama",
"config": {
"model": "llama3.1:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.162.xxx.xxx:11434/",
},
},
"embedder": {
"provider": "ollama",
"config": {
"model": "bge-m3:latest",
"ollama_base_url": "http://10.162.xxx.xxx:11434/",
"embedding_dims": 768
},
},
"version": "v1.1"
}

BR
Kimi

[Dev-Khant]

Hey @quzhixue-Kimi I tried and it was working for me.

Can you please try and let us know if this snippet works for you?

import os
from mem0 import Memory

config = {
    "llm": {
        "provider": "ollama",
        "config": {
            "model": "llama3.1:8b",
            "temperature": 0.1,
            "max_tokens": 2000,
        }
    },
    "embedder": {
        "provider": "ollama",
        "config": {
            "model": "nomic-embed-text:latest"
        }
    }
}

m = Memory.from_config(config)
m.add("Likes to play cricket on weekends", user_id="alice", metadata={"category": "hobbies"})

[l-baldassini]

Hi,

I have the same issue, I am running the model locally using Ollama but the _ensure_model_exists() function in ollama.py still tries to pull the model. I looked at the code of this function (see below):

   def _ensure_model_exists(self):
        """
        Ensure the specified model exists locally. If not, pull it from Ollama.
        """
        local_models = self.client.list()["models"]
        if not any(model.get("name") == self.config.model for model in local_models):
            self.client.pull(self.config.model)

and printed local_models at which point I saw that the ollama Model class does not have any name attribute but has model instead. Replacing model.get("name") with model.get("model") seems to solve the issue.

Am I understanding the function wrong or should name be changed to model for this function to work correctly?

For reference, this is my config:

config = {
    "embedder": {
        "provider": "ollama",
        "config": {
            "model": "nomic-embed-text:latest",
            "ollama_base_url": "http://localhost:11434",
        }
    },
    "vector_store": {
        "provider": "elasticsearch",
        "config": {
            "collection_name": "mem0",
            "host": "http://localhost",
            "port": 9200,
            "embedding_model_dims": 768,
            "user": os.getenv("ES_USER"),
            "password": os.getenv("ES_PASSWORD"),
        }
    },
    "llm": {
        "provider": "ollama",
        "config": {
            "model": "qwen2.5-coder-32b-32k:latest",
            "api_key" : os.getenv("LLM_API_KEY"),
            "ollama_base_url" : os.getenv("LLM_API_URL"),
            "temperature": 0.2,
        }
    },
    "version": "v1.1"
}

Thank you!

[quzhixue-Kimi]

hi @Dev-Khant
Thank you for your feedback.

I am interested in graph_store with neo4j. And, the neo4j and ollama docker container started successfully. Here is my sample code as below:

Please disable the internet and run this code. It will fail to download models. There should be one error as below:

"""ollama._types.ResponseError: pull model manifest: Get "https://registry.ollama.ai/v2/library/bge-m3/manifests/latest";: dial tcp: lookup registry.ollama.ai on 127.0.0.53:53: server misbehaving"""

from mem0 import Memory

config = {
"graph_store": {
"provider": "neo4j",
"config": {
"url": "neo4j://10.162.5.211:7687",
"username": "neo4j",
"password": "neo4j"
},
"llm" : {
"provider": "ollama",
"config": {
"model": "qwq:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.162.5.211:11434",
},
}
},
"llm": {
"provider": "ollama",
"config": {
"model": "qwq:latest",
"temperature": 0.2,
"max_tokens": 8000,
"ollama_base_url": "http://10.162.5.211:11434",
},
},
"embedder": {
"provider": "ollama",
"config": {
"model": "bge-m3:latest",
"ollama_base_url": "http://10.162.5.211:11434",
},
},
"version": "v1.1"
}

Initialize Memory with the configuration

m = Memory.from_config(config)

Add a memory

m.add("I'm visiting Paris", user_id="john")

Retrieve memories

memories = m.get_all(user_id="john")
print(memories)

m.delete_all(user_id="alice123")

m.add("I like going to hikes", user_id="alice123")
m.add("I love to play badminton", user_id="alice123")
m.add("I hate playing badminton", user_id="alice123")
m.add("My friend name is john and john has a dog named tommy", user_id="alice123")
m.add("My name is Alice", user_id="alice123")
m.add("John loves to hike and Harry loves to hike as well", user_id="alice123")

m.add("My friend name is kimi and kimi is a F1 fans.", user_id="alice123")
m.add("So, kimi and john are friends.", user_id="alice123")

friends_result = m.search("Who are my friends?", user_id="alice123")
print(friends_result)

Currently, I tried to modify the mem0 code to disable the _ensure_model_exists method code.

But, got another issue. I am trying to set up a new environment with internet to test it.

A new issue posted:
#2223

BR
Kimi