Re: [Boost-bugs] [Boost C++ Libraries] #290: perfomance: memory cleanup for pool takes too long

Subject: Re: [Boost-bugs] [Boost C++ Libraries] #290: perfomance: memory cleanup for pool takes too long
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2007-11-21 01:19:09

#290: perfomance: memory cleanup for pool takes too long
---------------------+------------------------------------------------------
  Reporter:  nobody  |       Owner:  shammah 
      Type:  Bugs    |      Status:  assigned
 Milestone:          |   Component:  pool    
   Version:  None    |    Severity:  Problem 
Resolution:  None    |    Keywords:          
---------------------+------------------------------------------------------
Changes (by marshall):

  * component:  None => pool
  * severity:  => Problem

Old description:

> {{{
> hi,
>
> In my project I use boost::pool because I have to
> instantiate millions of object at the beginning of the
> application, and only get rid of the at termination time.
>
> I noticed that when processing large data
> (which may require 1Gb memory for the application while
> it is running)
> the application may finish its processing approx after 10
> seconds.
> the cleanup part (which consists only of calling the pool
> release methods on allocated objects)
> on the other hand, takes many minutes
> (I usually give up after 5 minutes)
>
> debugging the code i noticed that the while loop in
> the nextof function
> (of SIMPLE SEGREGATED STORAGE.HPP)
>
> takes many iterations.
> this leads me to the belief that the reason the cleanup
> takes too long is because trying to cleanup 1GB memory
> of million of objects at once causes
> pool to retain all its chunks in the hope of more
> allocations in the future
> which in turns causes it to waste all computation on
> running to the end of (endless) lists in the nextof
> function
>
> I will appreciate a suggestion on how to fix this
> until a more permanent solution is applied to the pool
> library
>
> thanks
> aviad
> aviadr_at_[hidden]
>
> }}}

New description:

 {{{
 hi,

 In my project I use boost::pool because I have to
 instantiate millions of object at the beginning of the
 application, and only get rid of the at termination time.

 I noticed that when processing large data
 (which may require 1Gb memory for the application while
 it is running)
 the application may finish its processing approx after 10
 seconds.
 the cleanup part (which consists only of calling the pool
 release methods on allocated objects)
 on the other hand, takes many minutes
 (I usually give up after 5 minutes)

 debugging the code i noticed that the while loop in
 the nextof function
 (of SIMPLE SEGREGATED STORAGE.HPP)

 takes many iterations.
 this leads me to the belief that the reason the cleanup
 takes too long is because trying to cleanup 1GB memory
 of million of objects at once causes
 pool to retain all its chunks in the hope of more
 allocations in the future
 which in turns causes it to waste all computation on
 running to the end of (endless) lists in the nextof
 function

 I will appreciate a suggestion on how to fix this
 until a more permanent solution is applied to the pool
 library

 thanks
 aviad
 aviadr_at_[hidden]

 }}}

-- 
Ticket URL: <http://svn.boost.org/trac/boost/ticket/290#comment:2>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.