Transformers gets killed for no reason on linux

Question

So i'm trying to run inference on a Huggingface model, the model is 6.18gb. This morning I was on Windows and it was possible to load the model, but inference was very slow so I took a look at DeepSpeed but only available on linux so I switched to Zorin OS. Now the exact same script gets killed when running

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Cedille/fr-boris", device_map = "auto")

What is going on ?

Answer 1

Try to diagnose with below command:

dmesg -T| grep -E -i -B100 'killed process'

And you may find out the reason.

[Fri Feb 10 21:16:54 2023] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-14313.scope,task=python,pid=1071011,uid=1000
[Fri Feb 10 21:16:54 2023] Out of memory: Killed process 1071011 (python) total-vm:2480280kB, anon-rss:1709008kB, file-rss:4kB, shmem-rss:0kB, UID:1000 pgtables:4276kB oom_score_adj:0