Menu
I discover one of my processes (~90 threads) use >8G memory and I do not think it requires such large.
Therefore, I read the /proc/{pid}/maps, /proc/{pid}/smaps and pstack {pid}. I wonder how those large 63M+508K regions come from, and what they are for, and can I reduce my memory usage by effacing them in some way?
I find out the regions are almost in three categories:
4xxx000-4xxxx000: thread stacks? 313xxxx000-313xxxx000: lib text and data? 2xxxxxxxx000-2xxxxxxxx000: many confusing (64M+...+63M+508K) blocks. The number of this kind of regions is very close to the number of the threads. So are these large regions related to threads?Here is the result after I merge the result of /proc/{pid}/maps and pstack {pid} | grep Thread:
(I place the *Thread ?? (Thread 0x@@@@@@@@@@ (LWP !!!!))* at the end of each region if 0x@@@@@@@@ locates inside this region (I guess @@@@ is the thread stack base and thus that region is the for the thread stack.)
Region 0: 18M, 00400000-01675000 r-xp 00000000 08:02 5401 /mybinary
Region 1: 236K, 01875000-018b0000 rw-p 01275000 08:02 5401 /mybinary
Region 2: 456K, 018b0000-01922000 rw-p 018b0000 00:00 0
Region 3: 741M, 11b46000-4006c000 rw-p 11b46000 00:00 0 [heap]
Region 4: 4K, 40084000-40085000 ---p 40084000 00:00 0
Region 5: 512K, 40085000-40105000 rw-p 40085000 00:00 0 *Thread 53 (Thread 0x40104940 (LWP 1281))*
Region 6: 4K, 40222000-40223000 ---p 40222000 00:00 0
Region 7: 512K, 40223000-402a3000 rw-p 40223000 00:00 0 *Thread 57 (Thread 0x402a2940 (LWP 1277))*
Region 8: 4K, 40338000-40339000 ---p 40338000 00:00 0
Region 9: 512K, 40339000-403b9000 rw-p 40339000 00:00 0 *Thread 52 (Thread 0x403b8940 (LWP 1282))*
Region 10: 4K, 403b9000-403ba000 ---p 403b9000 00:00 0
Region 11: 1M, 403ba000-404ba000 rw-p 403ba000 00:00 0 *Thread 31 (Thread 0x404b9940 (LWP 1303))*
Region 12: 4K, 404ba000-404bb000 ---p 404ba000 00:00 0
Region 13: 1M, 404bb000-405bb000 rw-p 404bb000 00:00 0 *Thread 2 (Thread 0x405ba940 (LWP 26786))*
Region 14: 4K, 40759000-4075a000 ---p 40759000 00:00 0
Region 15: 512K, 4075a000-407da000 rw-p 4075a000 00:00 0 *Thread 84 (Thread 0x407d9940 (LWP 1239))*
Region 16: 4K, 407da000-407db000 ---p 407da000 00:00 0
Region 17: 512K, 407db000-4085b000 rw-p 407db000 00:00 0 *Thread 51 (Thread 0x4085a940 (LWP 1283))*
Region 18: 4K, 4085b000-4085c000 ---p 4085b000 00:00 0
Region 19: 1M, 4085c000-4095c000 rw-p 4085c000 00:00 0 *Thread 50 (Thread 0x4095b940 (LWP 1284))*
Region 20: 4K, 4095c000-4095d000 ---p 4095c000 00:00 0
Region 21: 512K, 4095d000-409dd000 rw-p 4095d000 00:00 0 *Thread 32 (Thread 0x409dc940 (LWP 1302))*
Region 22: 4K, 409dd000-409de000 ---p 409dd000 00:00 0
Region 23: 1M, 409de000-40ade000 rw-p 409de000 00:00 0 *Thread 24 (Thread 0x40add940 (LWP 26764))*
Region 24: 4K, 40b9f000-40ba0000 ---p 40b9f000 00:00 0
Region 25: 1M, 40ba0000-40ca0000 rw-p 40ba0000 00:00 0 *Thread 83 (Thread 0x40c9f940 (LWP 1240))*
Region 26: 4K, 40ca0000-40ca1000 ---p 40ca0000 00:00 0
Region 27: 1M, 40ca1000-40da1000 rw-p 40ca1000 00:00 0 *Thread 82 (Thread 0x40da0940 (LWP 1241))*
Region 28: 4K, 40da1000-40da2000 ---p 40da1000 00:00 0
Region 29: 1M, 40da2000-40ea2000 rw-p 40da2000 00:00 0 *Thread 30 (Thread 0x40ea1940 (LWP 1304))*
Region 30: 4K, 40eea000-40eeb000 ---p 40eea000 00:00 0
Region 31: 1M, 40eeb000-40feb000 rw-p 40eeb000 00:00 0 *Thread 87 (Thread 0x40fea940 (LWP 1235))*
Region 32: 4K, 40feb000-40fec000 ---p 40feb000 00:00 0
Region 33: 512K, 40fec000-4106c000 rw-p 40fec000 00:00 0 *Thread 47 (Thread 0x4106b940 (LWP 1287))*
Region 34: 4K, 4106c000-4106d000 ---p 4106c000 00:00 0
Region 35: 1M, 4106d000-4116d000 rw-p 4106d000 00:00 0 *Thread 23 (Thread 0x4116c940 (LWP 26765))*
Region 36: 4K, 41218000-41219000 ---p 41218000 00:00 0
Region 37: 1M, 41219000-41319000 rw-p 41219000 00:00 0 *Thread 81 (Thread 0x41318940 (LWP 1242))*
Region 38: 4K, 41319000-4131a000 ---p 41319000 00:00 0
Region 39: 512K, 4131a000-4139a000 rw-p 4131a000 00:00 0 *Thread 62 (Thread 0x41399940 (LWP 1263))*
Region 40: 4K, 4139a000-4139b000 ---p 4139a000 00:00 0
Region 41: 1M, 4139b000-4149b000 rw-p 4139b000 00:00 0 *Thread 61 (Thread 0x4149a940 (LWP 1264))*
Region 42: 4K, 4154b000-4154c000 ---p 4154b000 00:00 0
Region 43: 1M, 4154c000-4164c000 rw-p 4154c000 00:00 0 *Thread 71 (Thread 0x4164b940 (LWP 1252))*
Region 44: 4K, 4168b000-4168c000 ---p 4168b000 00:00 0
Region 45: 1M, 4168c000-4178c000 rw-p 4168c000 00:00 0 *Thread 89 (Thread 0x4178b940 (LWP 1233))*
Region 46: 4K, 4178c000-4178d000 ---p 4178c000 00:00 0
Region 47: 1M, 4178d000-4188d000 rw-p 4178d000 00:00 0 *Thread 88 (Thread 0x4188c940 (LWP 1234))*
Region 48: 4K, 4188d000-4188e000 ---p 4188d000 00:00 0
Region 49: 1M, 4188e000-4198e000 rw-p 4188e000 00:00 0 *Thread 70 (Thread 0x4198d940 (LWP 1253))*
Region 50: 4K, 4198e000-4198f000 ---p 4198e000 00:00 0
Region 51: 1M, 4198f000-41a8f000 rw-p 4198f000 00:00 0 *Thread 69 (Thread 0x41a8e940 (LWP 1254))*
Region 52: 4K, 41a8f000-41a90000 ---p 41a8f000 00:00 0
Region 53: 1M, 41a90000-41b90000 rw-p 41a90000 00:00 0 *Thread 68 (Thread 0x41b8f940 (LWP 1255))*
Region 54: 4K, 41b90000-41b91000 ---p 41b90000 00:00 0
Region 55: 1M, 41b91000-41c91000 rw-p 41b91000 00:00 0 *Thread 67 (Thread 0x41c90940 (LWP 1256))*
Region 56: 4K, 41c91000-41c92000 ---p 41c91000 00:00 0
Region 57: 512K, 41c92000-41d12000 rw-p 41c92000 00:00 0 *Thread 66 (Thread 0x41d11940 (LWP 1257))*
Region 58: 4K, 41d30000-41d31000 ---p 41d30000 00:00 0
Region 59: 512K, 41d31000-41db1000 rw-p 41d31000 00:00 0 *Thread 86 (Thread 0x41db0940 (LWP 1237))*
Region 60: 4K, 41db1000-41db2000 ---p 41db1000 00:00 0
Region 61: 512K, 41db2000-41e32000 rw-p 41db2000 00:00 0 *Thread 85 (Thread 0x41e31940 (LWP 1238))*
Region 62: 4K, 41e32000-41e33000 ---p 41e32000 00:00 0
Region 63: 1M, 41e33000-41f33000 rw-p 41e33000 00:00 0 *Thread 64 (Thread 0x41f32940 (LWP 1261))*
Region 64: 4K, 41f33000-41f34000 ---p 41f33000 00:00 0
Region 65: 512K, 41f34000-41fb4000 rw-p 41f34000 00:00 0 *Thread 63 (Thread 0x41fb3940 (LWP 1262))*
Region 66: 4K, 41fb7000-41fb8000 ---p 41fb7000 00:00 0
Region 67: 1M, 41fb8000-420b8000 rw-p 41fb8000 00:00 0 *Thread 80 (Thread 0x420b7940 (LWP 1243))*
Region 68: 4K, 420b8000-420b9000 ---p 420b8000 00:00 0
Region 69: 1M, 420b9000-421b9000 rw-p 420b9000 00:00 0 *Thread 79 (Thread 0x421b8940 (LWP 1244))*
Region 70: 4K, 421b9000-421ba000 ---p 421b9000 00:00 0
Region 71: 1M, 421ba000-422ba000 rw-p 421ba000 00:00 0 *Thread 78 (Thread 0x422b9940 (LWP 1245))*
Region 72: 4K, 422ba000-422bb000 ---p 422ba000 00:00 0
Region 73: 1M, 422bb000-423bb000 rw-p 422bb000 00:00 0 *Thread 77 (Thread 0x423ba940 (LWP 1246))*
Region 74: 4K, 423bb000-423bc000 ---p 423bb000 00:00 0
Region 75: 1M, 423bc000-424bc000 rw-p 423bc000 00:00 0 *Thread 76 (Thread 0x424bb940 (LWP 1247))*
Region 76: 4K, 424bc000-424bd000 ---p 424bc000 00:00 0
Region 77: 1M, 424bd000-425bd000 rw-p 424bd000 00:00 0 *Thread 75 (Thread 0x425bc940 (LWP 1248))*
Region 78: 4K, 425bd000-425be000 ---p 425bd000 00:00 0
Region 79: 1M, 425be000-426be000 rw-p 425be000 00:00 0 *Thread 74 (Thread 0x426bd940 (LWP 1249))*
Region 80: 4K, 426be000-426bf000 ---p 426be000 00:00 0
Region 81: 1M, 426bf000-427bf000 rw-p 426bf000 00:00 0 *Thread 73 (Thread 0x427be940 (LWP 1250))*
Region 82: 4K, 427bf000-427c0000 ---p 427bf000 00:00 0
Region 83: 1M, 427c0000-428c0000 rw-p 427c0000 00:00 0 *Thread 72 (Thread 0x428bf940 (LWP 1251))*
Region 84: 4K, 428c0000-428c1000 ---p 428c0000 00:00 0
Region 85: 1M, 428c1000-429c1000 rw-p 428c1000 00:00 0 *Thread 65 (Thread 0x429c0940 (LWP 1260))*
Region 86: 4K, 429c1000-429c2000 ---p 429c1000 00:00 0
Region 87: 1M, 429c2000-42ac2000 rw-p 429c2000 00:00 0 *Thread 60 (Thread 0x42ac1940 (LWP 1265))*
Region 88: 4K, 42ac2000-42ac3000 ---p 42ac2000 00:00 0
Region 89: 1M, 42ac3000-42bc3000 rw-p 42ac3000 00:00 0 *Thread 59 (Thread 0x42bc2940 (LWP 1266))*
Region 90: 4K, 42bc3000-42bc4000 ---p 42bc3000 00:00 0
Region 91: 512K, 42bc4000-42c44000 rw-p 42bc4000 00:00 0 *Thread 58 (Thread 0x42c43940 (LWP 1267))*
Region 92: 4K, 42c44000-42c45000 ---p 42c44000 00:00 0
Region 93: 1M, 42c45000-42d45000 rw-p 42c45000 00:00 0 *Thread 56 (Thread 0x42d44940 (LWP 1278))*
Region 94: 4K, 42d45000-42d46000 ---p 42d45000 00:00 0
Region 95: 1M, 42d46000-42e46000 rw-p 42d46000 00:00 0 *Thread 55 (Thread 0x42e45940 (LWP 1279))*
Region 96: 4K, 42e46000-42e47000 ---p 42e46000 00:00 0
Region 97: 1M, 42e47000-42f47000 rw-p 42e47000 00:00 0 *Thread 54 (Thread 0x42f46940 (LWP 1280))*
Region 98: 4K, 42f47000-42f48000 ---p 42f47000 00:00 0
Region 99: 1M, 42f48000-43048000 rw-p 42f48000 00:00 0 *Thread 49 (Thread 0x43047940 (LWP 1285))*
Region 100: 4K, 43048000-43049000 ---p 43048000 00:00 0
Region 101: 1M, 43049000-43149000 rw-p 43049000 00:00 0 *Thread 48 (Thread 0x43148940 (LWP 1286))*
Region 102: 4K, 43149000-4314a000 ---p 43149000 00:00 0
Region 103: 1M, 4314a000-4324a000 rw-p 4314a000 00:00 0 *Thread 46 (Thread 0x43249940 (LWP 1288))*
Region 104: 4K, 4324a000-4324b000 ---p 4324a000 00:00 0
Region 105: 1M, 4324b000-4334b000 rw-p 4324b000 00:00 0 *Thread 45 (Thread 0x4334a940 (LWP 1289))*
Region 106: 4K, 4334b000-4334c000 ---p 4334b000 00:00 0
Region 107: 1M, 4334c000-4344c000 rw-p 4334c000 00:00 0 *Thread 44 (Thread 0x4344b940 (LWP 1290))*
Region 108: 4K, 4344c000-4344d000 ---p 4344c000 00:00 0
Region 109: 1M, 4344d000-4354d000 rw-p 4344d000 00:00 0 *Thread 43 (Thread 0x4354c940 (LWP 1291))*
Region 110: 4K, 4354d000-4354e000 ---p 4354d000 00:00 0
Region 111: 1M, 4354e000-4364e000 rw-p 4354e000 00:00 0 *Thread 42 (Thread 0x4364d940 (LWP 1292))*
Region 112: 4K, 4364e000-4364f000 ---p 4364e000 00:00 0
Region 113: 1M, 4364f000-4374f000 rw-p 4364f000 00:00 0 *Thread 41 (Thread 0x4374e940 (LWP 1293))*
Region 114: 4K, 4374f000-43750000 ---p 4374f000 00:00 0
Region 115: 1M, 43750000-43850000 rw-p 43750000 00:00 0 *Thread 40 (Thread 0x4384f940 (LWP 1294))*
Region 116: 4K, 43850000-43851000 ---p 43850000 00:00 0
Region 117: 1M, 43851000-43951000 rw-p 43851000 00:00 0 *Thread 39 (Thread 0x43950940 (LWP 1295))*
Region 118: 4K, 43951000-43952000 ---p 43951000 00:00 0
Region 119: 1M, 43952000-43a52000 rw-p 43952000 00:00 0 *Thread 38 (Thread 0x43a51940 (LWP 1296))*
Region 120: 4K, 43a52000-43a53000 ---p 43a52000 00:00 0
Region 121: 1M, 43a53000-43b53000 rw-p 43a53000 00:00 0 *Thread 37 (Thread 0x43b52940 (LWP 1297))*
Region 122: 4K, 43b53000-43b54000 ---p 43b53000 00:00 0
Region 123: 1M, 43b54000-43c54000 rw-p 43b54000 00:00 0 *Thread 36 (Thread 0x43c53940 (LWP 1298))*
Region 124: 4K, 43c54000-43c55000 ---p 43c54000 00:00 0
Region 125: 1M, 43c55000-43d55000 rw-p 43c55000 00:00 0 *Thread 35 (Thread 0x43d54940 (LWP 1299))*
Region 126: 4K, 43d55000-43d56000 ---p 43d55000 00:00 0
Region 127: 1M, 43d56000-43e56000 rw-p 43d56000 00:00 0 *Thread 34 (Thread 0x43e55940 (LWP 1300))*
Region 128: 4K, 43e56000-43e57000 ---p 43e56000 00:00 0
Region 129: 1M, 43e57000-43f57000 rw-p 43e57000 00:00 0 *Thread 33 (Thread 0x43f56940 (LWP 1301))*
Region 130: 4K, 43f57000-43f58000 ---p 43f57000 00:00 0
Region 131: 1M, 43f58000-44058000 rw-p 43f58000 00:00 0 *Thread 29 (Thread 0x44057940 (LWP 7344))*
Region 132: 4K, 44058000-44059000 ---p 44058000 00:00 0
Region 133: 1M, 44059000-44159000 rw-p 44059000 00:00 0 *Thread 28 (Thread 0x44158940 (LWP 7345))*
Region 134: 4K, 44159000-4415a000 ---p 44159000 00:00 0
Region 135: 1M, 4415a000-4425a000 rw-p 4415a000 00:00 0 *Thread 27 (Thread 0x44259940 (LWP 7346))*
Region 136: 4K, 4425a000-4425b000 ---p 4425a000 00:00 0
Region 137: 1M, 4425b000-4435b000 rw-p 4425b000 00:00 0 *Thread 26 (Thread 0x4435a940 (LWP 7347))*
Region 138: 4K, 4435b000-4435c000 ---p 4435b000 00:00 0
Region 139: 1M, 4435c000-4445c000 rw-p 4435c000 00:00 0 *Thread 25 (Thread 0x4445b940 (LWP 7348))*
Region 140: 4K, 4445c000-4445d000 ---p 4445c000 00:00 0
Region 141: 512K, 4445d000-444dd000 rw-p 4445d000 00:00 0 *Thread 22 (Thread 0x444dc940 (LWP 26766))*
Region 142: 4K, 444dd000-444de000 ---p 444dd000 00:00 0
Region 143: 512K, 444de000-4455e000 rw-p 444de000 00:00 0 *Thread 21 (Thread 0x4455d940 (LWP 26767))*
Region 144: 4K, 4455e000-4455f000 ---p 4455e000 00:00 0
Region 145: 1M, 4455f000-4465f000 rw-p 4455f000 00:00 0 *Thread 20 (Thread 0x4465e940 (LWP 26768))*
Region 146: 4K, 4465f000-44660000 ---p 4465f000 00:00 0
Region 147: 1M, 44660000-44760000 rw-p 44660000 00:00 0 *Thread 19 (Thread 0x4475f940 (LWP 26769))*
Region 148: 4K, 44760000-44761000 ---p 44760000 00:00 0
Region 149: 1M, 44761000-44861000 rw-p 44761000 00:00 0 *Thread 18 (Thread 0x44860940 (LWP 26770))*
Region 150: 4K, 44861000-44862000 ---p 44861000 00:00 0
Region 151: 512K, 44862000-448e2000 rw-p 44862000 00:00 0 *Thread 17 (Thread 0x448e1940 (LWP 26771))*
Region 152: 4K, 448e2000-448e3000 ---p 448e2000 00:00 0
Region 153: 1M, 448e3000-449e3000 rw-p 448e3000 00:00 0 *Thread 16 (Thread 0x449e2940 (LWP 26772))*
Region 154: 4K, 449e3000-449e4000 ---p 449e3000 00:00 0
Region 155: 1M, 449e4000-44ae4000 rw-p 449e4000 00:00 0 *Thread 15 (Thread 0x44ae3940 (LWP 26773))*
Region 156: 4K, 44ae4000-44ae5000 ---p 44ae4000 00:00 0
Region 157: 1M, 44ae5000-44be5000 rw-p 44ae5000 00:00 0 *Thread 14 (Thread 0x44be4940 (LWP 26774))*
Region 158: 4K, 44be5000-44be6000 ---p 44be5000 00:00 0
Region 159: 1M, 44be6000-44ce6000 rw-p 44be6000 00:00 0 *Thread 13 (Thread 0x44ce5940 (LWP 26775))*
Region 160: 4K, 44ce6000-44ce7000 ---p 44ce6000 00:00 0
Region 161: 1M, 44ce7000-44de7000 rw-p 44ce7000 00:00 0 *Thread 12 (Thread 0x44de6940 (LWP 26776))*
Region 162: 4K, 44de7000-44de8000 ---p 44de7000 00:00 0
Region 163: 1M, 44de8000-44ee8000 rw-p 44de8000 00:00 0 *Thread 11 (Thread 0x44ee7940 (LWP 26777))*
Region 164: 4K, 44ee8000-44ee9000 ---p 44ee8000 00:00 0
Region 165: 1M, 44ee9000-44fe9000 rw-p 44ee9000 00:00 0 *Thread 10 (Thread 0x44fe8940 (LWP 26778))*
Region 166: 4K, 44fe9000-44fea000 ---p 44fe9000 00:00 0
Region 167: 1M, 44fea000-450ea000 rw-p 44fea000 00:00 0 *Thread 9 (Thread 0x450e9940 (LWP 26779))*
Region 168: 4K, 450ea000-450eb000 ---p 450ea000 00:00 0
Region 169: 1M, 450eb000-451eb000 rw-p 450eb000 00:00 0 *Thread 8 (Thread 0x451ea940 (LWP 26780))*
Region 170: 4K, 451eb000-451ec000 ---p 451eb000 00:00 0
Region 171: 1M, 451ec000-452ec000 rw-p 451ec000 00:00 0 *Thread 7 (Thread 0x452eb940 (LWP 26781))*
Region 172: 4K, 452ec000-452ed000 ---p 452ec000 00:00 0
Region 173: 1M, 452ed000-453ed000 rw-p 452ed000 00:00 0 *Thread 6 (Thread 0x453ec940 (LWP 26782))*
Region 174: 4K, 453ed000-453ee000 ---p 453ed000 00:00 0
Region 175: 1M, 453ee000-454ee000 rw-p 453ee000 00:00 0 *Thread 5 (Thread 0x454ed940 (LWP 26783))*
Region 176: 4K, 454ee000-454ef000 ---p 454ee000 00:00 0
Region 177: 1M, 454ef000-455ef000 rw-p 454ef000 00:00 0 *Thread 4 (Thread 0x455ee940 (LWP 26784))*
Region 178: 4K, 455ef000-455f0000 ---p 455ef000 00:00 0
Region 179: 1M, 455f0000-456f0000 rw-p 455f0000 00:00 0 *Thread 3 (Thread 0x456ef940 (LWP 26785))*
Region 180: 112K, 3138200000-313821c000 r-xp 00000000 08:03 28509165 /lib64/ld-2.5.so
Region 181: 4K, 313841b000-313841c000 r--p 0001b000 08:03 28509165 /lib64/ld-2.5.so
Region 182: 4K, 313841c000-313841d000 rw-p 0001c000 08:03 28509165 /lib64/ld-2.5.so
Region 183: 1M, 3138600000-313874d000 r-xp 00000000 08:03 28509166 /lib64/libc-2.5.so
Region 184: 2M, 313874d000-313894d000 ---p 0014d000 08:03 28509166 /lib64/libc-2.5.so
Region 185: 16K, 313894d000-3138951000 r--p 0014d000 08:03 28509166 /lib64/libc-2.5.so
Region 186: 4K, 3138951000-3138952000 rw-p 00151000 08:03 28509166 /lib64/libc-2.5.so
Region 187: 20K, 3138952000-3138957000 rw-p 3138952000 00:00 0
Region 188: 8K, 3138a00000-3138a02000 r-xp 00000000 08:03 28509167 /lib64/libdl-2.5.so
Region 189: 2M, 3138a02000-3138c02000 ---p 00002000 08:03 28509167 /lib64/libdl-2.5.so
Region 190: 4K, 3138c02000-3138c03000 r--p 00002000 08:03 28509167 /lib64/libdl-2.5.so
Region 191: 4K, 3138c03000-3138c04000 rw-p 00003000 08:03 28509167 /lib64/libdl-2.5.so
Region 192: 88K, 3138e00000-3138e16000 r-xp 00000000 08:03 28509171 /lib64/libpthread-2.5.so
Region 193: 1M, 3138e16000-3139015000 ---p 00016000 08:03 28509171 /lib64/libpthread-2.5.so
Region 194: 4K, 3139015000-3139016000 r--p 00015000 08:03 28509171 /lib64/libpthread-2.5.so
Region 195: 4K, 3139016000-3139017000 rw-p 00016000 08:03 28509171 /lib64/libpthread-2.5.so
Region 196: 16K, 3139017000-313901b000 rw-p 3139017000 00:00 0
Region 197: 520K, 3139200000-3139282000 r-xp 00000000 08:03 28509172 /lib64/libm-2.5.so
Region 198: 1M, 3139282000-3139481000 ---p 00082000 08:03 28509172 /lib64/libm-2.5.so
Region 199: 4K, 3139481000-3139482000 r--p 00081000 08:03 28509172 /lib64/libm-2.5.so
Region 200: 4K, 3139482000-3139483000 rw-p 00082000 08:03 28509172 /lib64/libm-2.5.so
Region 201: 80K, 3139600000-3139614000 r-xp 00000000 08:03 27531228 /usr/lib64/libz.so.1.2.3
Region 202: 1M, 3139614000-3139813000 ---p 00014000 08:03 27531228 /usr/lib64/libz.so.1.2.3
Region 203: 4K, 3139813000-3139814000 rw-p 00013000 08:03 27531228 /usr/lib64/libz.so.1.2.3
Region 204: 236K, 3139a00000-3139a3b000 r-xp 00000000 08:03 28509176 /lib64/libsepol.so.1
Region 205: 2M, 3139a3b000-3139c3b000 ---p 0003b000 08:03 28509176 /lib64/libsepol.so.1
Region 206: 4K, 3139c3b000-3139c3c000 rw-p 0003b000 08:03 28509176 /lib64/libsepol.so.1
Region 207: 40K, 3139c3c000-3139c46000 rw-p 3139c3c000 00:00 0
Region 208: 28K, 3139e00000-3139e07000 r-xp 00000000 08:03 28509173 /lib64/librt-2.5.so
Region 209: 2M, 3139e07000-313a007000 ---p 00007000 08:03 28509173 /lib64/librt-2.5.so
Region 210: 4K, 313a007000-313a008000 r--p 00007000 08:03 28509173 /lib64/librt-2.5.so
Region 211: 4K, 313a008000-313a009000 rw-p 00008000 08:03 28509173 /lib64/librt-2.5.so
Region 212: 84K, 313a200000-313a215000 r-xp 00000000 08:03 28509177 /lib64/libselinux.so.1
Region 213: 2M, 313a215000-313a415000 ---p 00015000 08:03 28509177 /lib64/libselinux.so.1
Region 214: 8K, 313a415000-313a417000 rw-p 00015000 08:03 28509177 /lib64/libselinux.so.1
Region 215: 4K, 313a417000-313a418000 rw-p 313a417000 00:00 0
Region 216: 52K, 313a600000-313a60d000 r-xp 00000000 08:03 28508215 /lib64/libgcc_s-4.1.2-20080825.so.1
Region 217: 2M, 313a60d000-313a80d000 ---p 0000d000 08:03 28508215 /lib64/libgcc_s-4.1.2-20080825.so.1
Region 218: 4K, 313a80d000-313a80e000 rw-p 0000d000 08:03 28508215 /lib64/libgcc_s-4.1.2-20080825.so.1
Region 219: 920K, 313aa00000-313aae6000 r-xp 00000000 08:03 27533465 /usr/lib64/libstdc++.so.6.0.8
Region 220: 1M, 313aae6000-313ace5000 ---p 000e6000 08:03 27533465 /usr/lib64/libstdc++.so.6.0.8
Region 221: 24K, 313ace5000-313aceb000 r--p 000e5000 08:03 27533465 /usr/lib64/libstdc++.so.6.0.8
Region 222: 12K, 313aceb000-313acee000 rw-p 000eb000 08:03 27533465 /usr/lib64/libstdc++.so.6.0.8
Region 223: 72K, 313acee000-313ad00000 rw-p 313acee000 00:00 0
Region 224: 1M, 313b200000-313b32d000 r-xp 00000000 08:03 28509179 /lib64/libcrypto.so.0.9.8e
Region 225: 1M, 313b32d000-313b52c000 ---p 0012d000 08:03 28509179 /lib64/libcrypto.so.0.9.8e
Region 226: 132K, 313b52c000-313b54d000 rw-p 0012c000 08:03 28509179 /lib64/libcrypto.so.0.9.8e
Region 227: 16K, 313b54d000-313b551000 rw-p 313b54d000 00:00 0
Region 228: 176K, 313ba00000-313ba2c000 r-xp 00000000 08:03 27542541 /usr/lib64/libgssapi_krb5.so.2.2
Region 229: 2M, 313ba2c000-313bc2c000 ---p 0002c000 08:03 27542541 /usr/lib64/libgssapi_krb5.so.2.2
Region 230: 8K, 313bc2c000-313bc2e000 rw-p 0002c000 08:03 27542541 /usr/lib64/libgssapi_krb5.so.2.2
Region 231: 276K, 313be00000-313be45000 r-xp 00000000 08:03 28508219 /lib64/libssl.so.0.9.8e
Region 232: 1M, 313be45000-313c044000 ---p 00045000 08:03 28508219 /lib64/libssl.so.0.9.8e
Region 233: 24K, 313c044000-313c04a000 rw-p 00044000 08:03 28508219 /lib64/libssl.so.0.9.8e
Region 234: 12K, 313ce00000-313ce03000 r-xp 00000000 08:03 28509182 /lib64/libcom_err.so.2.1
Region 235: 1M, 313ce03000-313d002000 ---p 00003000 08:03 28509182 /lib64/libcom_err.so.2.1
Region 236: 4K, 313d002000-313d003000 rw-p 00002000 08:03 28509182 /lib64/libcom_err.so.2.1
Region 237: 68K, 313d200000-313d211000 r-xp 00000000 08:03 28508365 /lib64/libresolv-2.5.so
Region 238: 2M, 313d211000-313d411000 ---p 00011000 08:03 28508365 /lib64/libresolv-2.5.so
Region 239: 4K, 313d411000-313d412000 r--p 00011000 08:03 28508365 /lib64/libresolv-2.5.so
Region 240: 4K, 313d412000-313d413000 rw-p 00012000 08:03 28508365 /lib64/libresolv-2.5.so
Region 241: 8K, 313d413000-313d415000 rw-p 313d413000 00:00 0
Region 242: 144K, 313e200000-313e224000 r-xp 00000000 08:03 27542539 /usr/lib64/libk5crypto.so.3.1
Region 243: 1M, 313e224000-313e423000 ---p 00024000 08:03 27542539 /usr/lib64/libk5crypto.so.3.1
Region 244: 8K, 313e423000-313e425000 rw-p 00023000 08:03 27542539 /usr/lib64/libk5crypto.so.3.1
Region 245: 580K, 313e600000-313e691000 r-xp 00000000 08:03 27542540 /usr/lib64/libkrb5.so.3.3
Region 246: 2M, 313e691000-313e891000 ---p 00091000 08:03 27542540 /usr/lib64/libkrb5.so.3.3
Region 247: 16K, 313e891000-313e895000 rw-p 00091000 08:03 27542540 /usr/lib64/libkrb5.so.3.3
Region 248: 32K, 313ea00000-313ea08000 r-xp 00000000 08:03 27542538 /usr/lib64/libkrb5support.so.0.1
Region 249: 1M, 313ea08000-313ec07000 ---p 00008000 08:03 27542538 /usr/lib64/libkrb5support.so.0.1
Region 250: 4K, 313ec07000-313ec08000 rw-p 00007000 08:03 27542538 /usr/lib64/libkrb5support.so.0.1
Region 251: 8K, 313ee00000-313ee02000 r-xp 00000000 08:03 28509181 /lib64/libkeyutils-1.2.so
Region 252: 1M, 313ee02000-313f001000 ---p 00002000 08:03 28509181 /lib64/libkeyutils-1.2.so
Region 253: 4K, 313f001000-313f002000 rw-p 00001000 08:03 28509181 /lib64/libkeyutils-1.2.so
Region 254: 4K, 2aaaaaaac000-2aaaaaaad000 rw-p 2aaaaaaac000 00:00 0
Region 255: 64M, 2aaaac000000-2aaab0000000 rw-p 2aaaac000000 00:00 0
Region 256: 40K, 2aaab0012000-2aaab001c000 r-xp 00000000 08:03 28508187 /lib64/libnss_files-2.5.so
Region 257: 1M, 2aaab001c000-2aaab021b000 ---p 0000a000 08:03 28508187 /lib64/libnss_files-2.5.so
Region 258: 4K, 2aaab021b000-2aaab021c000 r--p 00009000 08:03 28508187 /lib64/libnss_files-2.5.so
Region 259: 4K, 2aaab021c000-2aaab021d000 rw-p 0000a000 08:03 28508187 /lib64/libnss_files-2.5.so
Region 260: 64M, 2aaab4000000-2aaab8000000 rw-p 2aaab4000000 00:00 0
Region 261: 64M, 2aaab8000000-2aaabc000000 rw-p 2aaab8000000 00:00 0
Region 262: 63M, 2aaabc000000-2aaabfffa000 rw-p 2aaabc000000 00:00 0
Region 263: 24K, 2aaabfffa000-2aaac0000000 ---p 2aaabfffa000 00:00 0
Region 264: 63M, 2aaac0000000-2aaac3ff9000 rw-p 2aaac0000000 00:00 0
Region 265: 28K, 2aaac3ff9000-2aaac4000000 ---p 2aaac3ff9000 00:00 0
Region 266: 64M, 2aaac4000000-2aaac8000000 rw-p 2aaac4000000 00:00 0
Region 267: 63M, 2aaac8000000-2aaacbfed000 rw-p 2aaac8000000 00:00 0
Region 268: 76K, 2aaacbfed000-2aaacc000000 ---p 2aaacbfed000 00:00 0
Region 269: 63M, 2aaacc000000-2aaacfffe000 rw-p 2aaacc000000 00:00 0
Region 270: 8K, 2aaacfffe000-2aaad0000000 ---p 2aaacfffe000 00:00 0
Region 271: 63M, 2aaad0000000-2aaad3fe5000 rw-p 2aaad0000000 00:00 0
Region 272: 108K, 2aaad3fe5000-2aaad4000000 ---p 2aaad3fe5000 00:00 0
Region 273: 64M, 2aaad4000000-2aaad8000000 rw-p 2aaad4000000 00:00 0
Region 274: 64M, 2aaad8000000-2aaadc000000 rw-p 2aaad8000000 00:00 0
Region 275: 64M, 2aaadc000000-2aaae0000000 rw-p 2aaadc000000 00:00 0
Region 276: 64M, 2aaae0000000-2aaae4000000 rw-p 2aaae0000000 00:00 0
Region 277: 64M, 2aaae8000000-2aaaec000000 rw-p 2aaae8000000 00:00 0
Region 278: 64M, 2aaaec000000-2aaaf0000000 rw-p 2aaaec000000 00:00 0
Region 279: 63M, 2aaaf0000000-2aaaf3ff6000 rw-p 2aaaf0000000 00:00 0
Region 280: 40K, 2aaaf3ff6000-2aaaf4000000 ---p 2aaaf3ff6000 00:00 0
Region 281: 63M, 2aaaf8000000-2aaafbfff000 rw-p 2aaaf8000000 00:00 0
Region 282: 4K, 2aaafbfff000-2aaafc000000 ---p 2aaafbfff000 00:00 0
Region 283: 63M, 2aaafc000000-2aaaffffb000 rw-p 2aaafc000000 00:00 0
Region 284: 20K, 2aaaffffb000-2aab00000000 ---p 2aaaffffb000 00:00 0
Region 285: 63M, 2aab00000000-2aab03fff000 rw-p 2aab00000000 00:00 0
Region 286: 4K, 2aab03fff000-2aab04000000 ---p 2aab03fff000 00:00 0
Region 287: 63M, 2aab04000000-2aab07f73000 rw-p 2aab04000000 00:00 0
Region 288: 564K, 2aab07f73000-2aab08000000 ---p 2aab07f73000 00:00 0
Region 289: 63M, 2aab08000000-2aab0bfac000 rw-p 2aab08000000 00:00 0
Region 290: 336K, 2aab0bfac000-2aab0c000000 ---p 2aab0bfac000 00:00 0
(Omit Region from 291-425 since the post length limitation)
Region 426: 508K, 2aac27f81000-2aac28000000 ---p 2aac27f81000 00:00 0
Region 427: 63M, 2aac28000000-2aac2bf81000 rw-p 2aac28000000 00:00 0
Region 428: 508K, 2aac2bf81000-2aac2c000000 ---p 2aac2bf81000 00:00 0
Region 429: 64M, 2aac2c000000-2aac30000000 rw-p 2aac2c000000 00:00 0
Region 430: 512M, 2aac34000000-2aac54000000 rw-p 2aac34000000 00:00 0
Region 431: 502M, 2aac5c30b000-2aac7b90b000 rw-p 2aac5c30b000 00:00 0
Region 432: 4K, 2b5d0b054000-2b5d0b055000 rw-p 2b5d0b054000 00:00 0
Region 433: 36K, 2b5d0b067000-2b5d0b070000 rw-p 2b5d0b067000 00:00 0 *Thread 1 (Thread 0x2b5d0b06f010 (LWP 1232))*
Region 434: 116K, 7fffe2668000-7fffe2685000 rw-p 7ffffffe2000 00:00 0 [stack]
Region 435: 116K, ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
892
Clearly most of your 8G is due to the 3rd category of memory regions ("many confusing (64M+...+63M+508K) blocks"). We found similar behaviour after porting our multithreaded app from 32-bit linux (where it used 200M virtual memory) to 64-bit linux (where it used 7G virtual memory): in our case, we found pairs of 132K rw-p and 64M ---p blocks, one pair for (nearly) each thread.
I found a similar issue discussed here: http://www.experts-exchange.com/Programming/System/Linux/Q_27033610.html. Here's the poster's conclusion:
OK, I browsed glibc sources, and glibc does that.
It defines HEAP_MAX_SIZE as 2 * DEFAULT_MMAP_THRESHOLD_MAX where DEFAULT_MMAP_THRESHOLD_MAX is 512K on 32bit and 32M on 64.
Then it wants a heap per thread - if no spare heap allocates new one[new_heap()]. The new heap has to be HEAP_MAX_SIZE aligned so next heaps can fit into possibly holes(to save vsize - so ironic!). To align, it simply allocates doubled size then unmaps spare part.
Finally it changes the protection for wanted region only leaving majority of the new heap without any rights. If more heap is needed changes protection on next chunk[grow_heap()] and so on.
To me - it looks stupid and vsize wasteful. On the other hand why someone would limit vspace on 64bit? Good Q, isn't it?
Also, it cannot be changed - it's compile time constants.
All in all, I'll have to live with this huge vsize on multithreaded applications or link against different memory allocator.
In our case, each thread has a 132K heap, with 64M available for grow_heap(); in your case, it looks to me as if each heap has already grown to (or near to) 64M, e.g. region 271 is a 63M heap (rw-p), and region 272 the as-yet unused remaining 108K (---p). Since you don't know why your process is using 8G in the first place, I don't know how to explain that.
I also don't know why glibc creates a heap for each thread (if, indeed, that is the right explanation). Here's what confuses me:
174
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
