Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way to limit the number of CPU cores Bazel uses?

Is there a way to tell Bazel when building how many CPU cores it can use?

TL;DR

I build TensorFlow on a VMware Workstation and being a virtual machine I can adjust the number of processors and cores for the virtual machine.

In the process of building TensorFlow I found that using only one core works. When I give the Workstation four cores and build TensorFlow it eventually halts the system to the point I have to reboot.

If I wait a few hours (leave it alone overnight) it sometimes returns with the following error:

gcc: internal compiler error: Killed (program cc1plus)

While I can change the number of cores using the virtual machines configuration options I would prefer to do it without having to shut down and restart the virtual machine.

like image 686
Guy Coder Avatar asked Jan 13 '16 00:01

Guy Coder


2 Answers

some examples for your .bazelrc

build --local_ram_resources=HOST_RAM*.5 --local_cpu_resources=HOST_CPUS-1 (leave one core free) 

or

build --local_cpu_resources=1 (use a single core) 

See https://docs.bazel.build/versions/master/command-line-reference.html#flag--local_cpu_resources

The currently accepted answer is deprecated.

like image 89
mancini0 Avatar answered Sep 26 '22 00:09

mancini0


From Bazel User Manual

--local_resources availableRAM,availableCPU,availableIO

This option, which takes three comma-separated floating point arguments, specifies the amount of local resources that Bazel can take into consideration when scheduling build and test activities. Option expects amount of available RAM (in MB), number of CPU cores (with 1.0 representing single full core) and workstation I/O capability (with 1.0 representing average workstation). By default Bazel will estimate amount of RAM and number of CPU cores directly from system configuration and will assume 1.0 I/O resource.

If this option is used, Bazel will ignore both --ram_utilization_factor.

like image 38
Guy Coder Avatar answered Sep 22 '22 00:09

Guy Coder