Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reduce gcc memory usage

Currently I have to build a quite large project on a very limited hardware (a core i5 which have only 2gb RAM). When run make with 4 thread gcc crashed due to each thread eats up to 40% RAM. I googled a bit and found this link. A few people feedback it works.However, not me.

Here is how I did.

/usr/bin/qmake-qt4 -r -spec linux-g++ $DEBUG_FLAGS -o Makefile "QMAKE_CXXFLAGS +=  --param ggc-min-expand=10 --param ggc-min-heapsize=8192"  /home/build-srv/LargeProject/largeProject.pro

make output does have the flag:

g++ -c -pipe --param ggc-min-expand=10 --param ggc-min-heapsize=8192 -std=c++11 -std=c++11 -g -Wall -W -D_REENTRANT -fPIC -D_GLIBCXX_USE_CXX11_ABI=0 -DBASE_LIBRARY -DQT_DECLARATIVE_DEBUG -DQT_DECLARATIVE_LIB -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/share/qt4/mkspecs/linux-g++  -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtNetwork -I/usr/include/qt4/QtGui -I/usr/include/qt4/QtDeclarative...

But the ram usage still the same. The only way is reduce to single thread build, which is awfully slow.

What did I do wrong? Are there any better solution for this.

Update:

  • Yes, the hardware department "upgrade" the RAM capacity to 4GB and now it's responsive when building the project.(Actually a stick broke so they need to wait the replacement). But I'm curious that in a limited resource system like a VPS how can we handle that? (Except reduce the build threads?)
  • I will search for the file that eat up a alot of RAM when compiling/linking. I dont remember the name due to that ain't my modules.
  • The build server had 2GB of swap space.
like image 722
Tiana987642 Avatar asked Apr 17 '17 03:04

Tiana987642


1 Answers

Tell your employer that the tools they provide for your job are inadequate. That's a bit of a joke of a system for what you're doing, and I assume it's provided by the employer. You need more RAM, no doubt about it. It should be simple enough to extend the RAM, there are no i5 motherboards that only take 2G as a maximum.

Otherwise - yes, you will need to limit the number of threads, e.g. use make -j2 or even make -j1. Modern compilers need lots of RAM not because they are wasteful, but because they hold the entire translation unit's worth of information in memory at once to provide global optimizations. Link-time code generation will need even more memory as it holds the information about the entire application.

like image 161
Kuba hasn't forgotten Monica Avatar answered Oct 18 '22 21:10

Kuba hasn't forgotten Monica