Posts tagged multiprocessing
Multiprocessing in Python and garbage collection
- 14 June 2014
Working with multiple threads in Python often leads to high RAM consumption. Unfortunately, automatic garbage collection in child processes isn’t working well. But there are two alternatives:
When using Pool(), you can specify number of task after which the child will be restarted resulting in memory release.
If you use Process(), you can simply delete unwanted objects and call gc.collect() inside the child.
Note, this may slow down your child process substantially!