Re: [Boost-users] shared_ptr segfaults on 64bit AMD

From: Paul Davis (pjdavis_at_[hidden])
Date: 2006-08-27 12:28:12

Peter Dimov wrote:
> Paul Davis wrote:
>   
>> Howdy,
>>
>> I've come across an odd segfault that originates in some of the boost
>> code. (By originate, meaning thats where the stack trace points, I'm
>> not sure if its me or boost thats wrong)
>>
>> Anyway, the weird part is that its only on my 64 bit machine.
>>
>> I took a look at where its segfaulting in
>> boost::detail::atomic_exchange_and_add(). Its scary inline assembly
>> stuff.  Well, mostly I just don't know assembly so I haven't the
>> slightest idea if its right or wrong.  And obviously, its a platform
>> specific header and what not so I imagine its limited to this area.  I
>> have had other weird segfaults that come and go from the atomic_* set
>> of methods.  I can't pin down exactly whats causing it.  They mostly
>> seem
>> to be coming from storing shared_ptr's in STL containers.  I've never
>> had any problems with it before so I'm assuming its just a relatively
>> untested section of code.
>>     
>
> It's being tested quite extensively, but some problems are triggered only in 
> very rare circumstances depending on the optimization level and the specific 
> compiler backend. This will be pretty hard to pin down.
>
> We can start with sanity checking whether int is 32 bits or 64 bits on this 
> platform. You should also try different optimization levels and see whether 
> this makes a difference. It would help a lot if you can trim the failing 
> example to a small snippet; we can examine the generated assembly (g++ -S) 
> then and see the atomic_* portions in context (they are usually marked with 
> #APP in the .s file.)
>   
[snip]

sizeof( int ) == 4 on this platform.

I've done a bit of looking to see if I can't find anywhere that I'm just 
wildly screwing up memory access.

I've tried valgrind and efence both.

I can't get efence to give me any information because it keeps detecting 
a malloc of size 0 in the postgres library, which the program doesn't 
run without.

Valgrind gave some errors emanating from the boost pointers reading 
invalid memory reads.  I've attached the valgrind output.

Now, I don't have the greatest experience with valgrind, but I'm 
reasonably sure that its not pointing directly at boost.  I could've 
overrun an array boundary, or somehow managed to write to my own memory 
segment and screwed with the internals of the ptr in which case it would 
still be my fault.

But I'm at a loss as what else to try.  I've tried recreating a similar 
situation in a simple testcase, but I can't manage to get the bug to 
come out.  I'm leaning toward it being my fault, but I'm not exactly 
sure what to try next to figure out where it is.

I'll keep looking.

Paul


==993== Memcheck, a memory error detector.
==993== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==993== Using LibVEX rev 1471, a library for dynamic binary translation.
==993== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==993== Using valgrind-3.1.0-Debian, a dynamic binary instrumentation framework.
==993== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==993== For more details, rerun with: -v
==993== 
==993== Invalid read of size 8
==993==    at 0x4010664: (within /lib/ld-2.3.6.so)
==993==    by 0x40089BC: (within /lib/ld-2.3.6.so)
==993==    by 0x4004DF3: (within /lib/ld-2.3.6.so)
==993==    by 0x4006612: (within /lib/ld-2.3.6.so)
==993==    by 0x576C51B: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576D0C9: _dl_open (in /lib/libc-2.3.6.so)
==993==    by 0x576E627: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576E6D2: __libc_dlopen_mode (in /lib/libc-2.3.6.so)
==993==    by 0x574A0F1: __nss_lookup_function (in /lib/libc-2.3.6.so)
==993==    by 0x574A253: (within /lib/libc-2.3.6.so)
==993==  Address 0x69E8528 is 16 bytes inside a block of size 21 alloc'd
==993==    at 0x4A19A16: malloc (vg_replace_malloc.c:149)
==993==    by 0x4006A00: (within /lib/ld-2.3.6.so)
==993==    by 0x576C51B: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576D0C9: _dl_open (in /lib/libc-2.3.6.so)
==993==    by 0x576E627: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576E6D2: __libc_dlopen_mode (in /lib/libc-2.3.6.so)
==993==    by 0x574A0F1: __nss_lookup_function (in /lib/libc-2.3.6.so)
==993==    by 0x574A253: (within /lib/libc-2.3.6.so)
==993==    by 0x5702A14: getpwuid_r (in /lib/libc-2.3.6.so)
==993==    by 0x4B37E18: pqGetpwuid (in /usr/lib/libpq.so.4.1)
==993== 
==993== Invalid read of size 8
==993==    at 0x401067E: (within /lib/ld-2.3.6.so)
==993==    by 0x40089BC: (within /lib/ld-2.3.6.so)
==993==    by 0x4004DF3: (within /lib/ld-2.3.6.so)
==993==    by 0x4006612: (within /lib/ld-2.3.6.so)
==993==    by 0x4009C2C: (within /lib/ld-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x4009F32: (within /lib/ld-2.3.6.so)
==993==    by 0x576C57A: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576D0C9: _dl_open (in /lib/libc-2.3.6.so)
==993==    by 0x58AD043: (within /lib/libdl-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==  Address 0x69F7A90 is 24 bytes inside a block of size 26 alloc'd
==993==    at 0x4A19A16: malloc (vg_replace_malloc.c:149)
==993==    by 0x4006A00: (within /lib/ld-2.3.6.so)
==993==    by 0x4009C2C: (within /lib/ld-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x4009F32: (within /lib/ld-2.3.6.so)
==993==    by 0x576C57A: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576D0C9: _dl_open (in /lib/libc-2.3.6.so)
==993==    by 0x58AD043: (within /lib/libdl-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x58AD541: (within /lib/libdl-2.3.6.so)
==993==    by 0x58AD081: dlopen (in /lib/libdl-2.3.6.so)
==993== 
==993== Conditional jump or move depends on uninitialised value(s)
==993==    at 0x4008F11: (within /lib/ld-2.3.6.so)
==993==    by 0x576C666: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576D0C9: _dl_open (in /lib/libc-2.3.6.so)
==993==    by 0x58AD043: (within /lib/libdl-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x58AD541: (within /lib/libdl-2.3.6.so)
==993==    by 0x58AD081: dlopen (in /lib/libdl-2.3.6.so)
==993==    by 0x407683: GetModule(std::string) (dds.cc:46)
==993==    by 0x4094BB: end_elem(void*, char const*) (dds.cc:196)
==993==    by 0x4C4C29A: (within /usr/lib/libexpat.so.1.0.0)
==993==    by 0x4C4D064: (within /usr/lib/libexpat.so.1.0.0)
==993== 
==993== Conditional jump or move depends on uninitialised value(s)
==993==    at 0x4008F51: (within /lib/ld-2.3.6.so)
==993==    by 0x576C666: (within /lib/libc-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x576D0C9: _dl_open (in /lib/libc-2.3.6.so)
==993==    by 0x58AD043: (within /lib/libdl-2.3.6.so)
==993==    by 0x400B13F: (within /lib/ld-2.3.6.so)
==993==    by 0x58AD541: (within /lib/libdl-2.3.6.so)
==993==    by 0x58AD081: dlopen (in /lib/libdl-2.3.6.so)
==993==    by 0x407683: GetModule(std::string) (dds.cc:46)
==993==    by 0x4094BB: end_elem(void*, char const*) (dds.cc:196)
==993==    by 0x4C4C29A: (within /usr/lib/libexpat.so.1.0.0)
==993==    by 0x4C4D064: (within /usr/lib/libexpat.so.1.0.0)

========================
| DDS Parameter Report |
========================

    Marching
    ########

        Previously Calculated
        ---------------------

        Generation in Progress
        ----------------------
            border_points

==993== 
==993== Invalid read of size 1
==993==    at 0x4B2CDA7: PQcmdTuples (in /usr/lib/libpq.so.4.1)
==993==    by 0x4FD39BA: dds::client::Command(std::string) (client.cc:121)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993==  Address 0x69FA098 is 32 bytes inside a block of size 160 free'd
==993==    at 0x4A1A5B3: free (vg_replace_malloc.c:235)
==993==    by 0x4FD39B1: dds::client::Command(std::string) (client.cc:119)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993== 
==993== Invalid read of size 1
==993==    at 0x4B2CDAB: PQcmdTuples (in /usr/lib/libpq.so.4.1)
==993==    by 0x4FD39BA: dds::client::Command(std::string) (client.cc:121)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993==  Address 0x69FA09F is 39 bytes inside a block of size 160 free'd
==993==    at 0x4A1A5B3: free (vg_replace_malloc.c:235)
==993==    by 0x4FD39B1: dds::client::Command(std::string) (client.cc:119)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993== 
==993== Invalid read of size 1
==993==    at 0x4B2CDD7: PQcmdTuples (in /usr/lib/libpq.so.4.1)
==993==    by 0x4FD39BA: dds::client::Command(std::string) (client.cc:121)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993==  Address 0x69FA0A0 is 40 bytes inside a block of size 160 free'd
==993==    at 0x4A1A5B3: free (vg_replace_malloc.c:235)
==993==    by 0x4FD39B1: dds::client::Command(std::string) (client.cc:119)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993== 
==993== Invalid read of size 1
==993==    at 0x4B2CDC2: PQcmdTuples (in /usr/lib/libpq.so.4.1)
==993==    by 0x4FD39BA: dds::client::Command(std::string) (client.cc:121)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993==  Address 0x69FA0A1 is 41 bytes inside a block of size 160 free'd
==993==    at 0x4A1A5B3: free (vg_replace_malloc.c:235)
==993==    by 0x4FD39B1: dds::client::Command(std::string) (client.cc:119)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993== 
==993== Invalid read of size 1
==993==    at 0x56A5CF1: (within /lib/libc-2.3.6.so)
==993==    by 0x56A3721: atoi (in /lib/libc-2.3.6.so)
==993==    by 0x4FD39C2: dds::client::Command(std::string) (client.cc:121)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993==  Address 0x69FA0A1 is 41 bytes inside a block of size 160 free'd
==993==    at 0x4A1A5B3: free (vg_replace_malloc.c:235)
==993==    by 0x4FD39B1: dds::client::Command(std::string) (client.cc:119)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993== 
==993== Invalid read of size 1
==993==    at 0x56A5E77: (within /lib/libc-2.3.6.so)
==993==    by 0x56A3721: atoi (in /lib/libc-2.3.6.so)
==993==    by 0x4FD39C2: dds::client::Command(std::string) (client.cc:121)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993==  Address 0x69FA0A1 is 41 bytes inside a block of size 160 free'd
==993==    at 0x4A1A5B3: free (vg_replace_malloc.c:235)
==993==    by 0x4FD39B1: dds::client::Command(std::string) (client.cc:119)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993== 
==993== Invalid read of size 1
==993==    at 0x56A5E0D: (within /lib/libc-2.3.6.so)
==993==    by 0x56A3721: atoi (in /lib/libc-2.3.6.so)
==993==    by 0x4FD39C2: dds::client::Command(std::string) (client.cc:121)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
==993==  Address 0x69FA0A2 is 42 bytes inside a block of size 160 free'd
==993==    at 0x4A1A5B3: free (vg_replace_malloc.c:235)
==993==    by 0x4FD39B1: dds::client::Command(std::string) (client.cc:119)
==993==    by 0x4FD39F8: dds::client::Insert(std::string) (client.cc:127)
==993==    by 0x408A92: main (dds.cc:489)
=============
| Arguments |
=============

    experiment_id = 5053
    field         = 2
    sequence      = 0

============
| Marching |
============
==993== 
==993== Invalid read of size 8
==993==    at 0x40A4BC: boost::detail::shared_count::~shared_count() (shared_count.hpp:159)
==993==    by 0x6D0B492: boost::shared_ptr<min_heap>::~shared_ptr() (shared_ptr.hpp:106)
==993==    by 0x6D0B8D8: __gnu_cxx::new_allocator<boost::shared_ptr<min_heap> >::destroy(boost::shared_ptr<min_heap>*) (new_allocator.h:107)
==993==    by 0x6D0CF5D: std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > >::erase(__gnu_cxx::__normal_iterator<boost::shared_ptr<min_heap>*, std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > > >) (vector.tcc:115)
==993==    by 0x6D0AB00: label_map::CloseToAlive(boost::shared_ptr<point>) (label_map.cc:276)
==993==    by 0x6D0626E: fast_march::march(cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&, cas::Imaging::Image<unsigned>&) (fast_march.cc:76)
==993==    by 0x6D11C5B: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:110)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993==  Address 0x6A066B8 is 8 bytes before a block of size 128 alloc'd
==993==    at 0x4A19DE3: operator new(unsigned long) (vg_replace_malloc.c:168)
==993==    by 0x6D0CFC3: __gnu_cxx::new_allocator<boost::shared_ptr<min_heap> >::allocate(unsigned long, void const*) (new_allocator.h:88)
==993==    by 0x6D0CFEA: std::_Vector_base<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > >::_M_allocate(unsigned long) (stl_vector.h:117)
==993==    by 0x6D0D8B2: std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<boost::shared_ptr<min_heap>*, std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > > >, boost::shared_ptr<min_heap> const&) (vector.tcc:275)
==993==    by 0x6D0DB32: std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > >::push_back(boost::shared_ptr<min_heap> const&) (stl_vector.h:610)
==993==    by 0x6D0A94F: label_map::CloseToAlive(boost::shared_ptr<point>) (label_map.cc:263)
==993==    by 0x6D0626E: fast_march::march(cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&, cas::Imaging::Image<unsigned>&) (fast_march.cc:76)
==993==    by 0x6D11C5B: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:110)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993== 
==993== Invalid read of size 8
==993==    at 0x40A4C8: boost::detail::shared_count::~shared_count() (shared_count.hpp:159)
==993==    by 0x6D0B492: boost::shared_ptr<min_heap>::~shared_ptr() (shared_ptr.hpp:106)
==993==    by 0x6D0B8D8: __gnu_cxx::new_allocator<boost::shared_ptr<min_heap> >::destroy(boost::shared_ptr<min_heap>*) (new_allocator.h:107)
==993==    by 0x6D0CF5D: std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > >::erase(__gnu_cxx::__normal_iterator<boost::shared_ptr<min_heap>*, std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > > >) (vector.tcc:115)
==993==    by 0x6D09BF2: label_map::NextClosePoint() (label_map.cc:183)
==993==    by 0x6D05FC4: fast_march::march(cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&, cas::Imaging::Image<unsigned>&) (fast_march.cc:42)
==993==    by 0x6D11C5B: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:110)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993==  Address 0x6A066A8 is not stack'd, malloc'd or (recently) free'd
==993== 
==993== Invalid read of size 4
==993==    at 0x40A2F8: boost::detail::atomic_exchange_and_add(int*, int) (sp_counted_base_gcc_x86.hpp:50)
==993==    by 0x40A46B: boost::detail::sp_counted_base::release() (sp_counted_base_gcc_x86.hpp:143)
==993==    by 0x40A4CF: boost::detail::shared_count::~shared_count() (shared_count.hpp:159)
==993==    by 0x6D0B492: boost::shared_ptr<min_heap>::~shared_ptr() (shared_ptr.hpp:106)
==993==    by 0x6D0B8D8: __gnu_cxx::new_allocator<boost::shared_ptr<min_heap> >::destroy(boost::shared_ptr<min_heap>*) (new_allocator.h:107)
==993==    by 0x6D0CF5D: std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > >::erase(__gnu_cxx::__normal_iterator<boost::shared_ptr<min_heap>*, std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > > >) (vector.tcc:115)
==993==    by 0x6D09BF2: label_map::NextClosePoint() (label_map.cc:183)
==993==    by 0x6D05FC4: fast_march::march(cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&, cas::Imaging::Image<unsigned>&) (fast_march.cc:42)
==993==    by 0x6D11C5B: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:110)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993==  Address 0xC8 is not stack'd, malloc'd or (recently) free'd
==993== 
==993== Process terminating with default action of signal 11 (SIGSEGV)
==993==  Access not within mapped region at address 0xC8
==993==    at 0x40A2F8: boost::detail::atomic_exchange_and_add(int*, int) (sp_counted_base_gcc_x86.hpp:50)
==993==    by 0x40A46B: boost::detail::sp_counted_base::release() (sp_counted_base_gcc_x86.hpp:143)
==993==    by 0x40A4CF: boost::detail::shared_count::~shared_count() (shared_count.hpp:159)
==993==    by 0x6D0B492: boost::shared_ptr<min_heap>::~shared_ptr() (shared_ptr.hpp:106)
==993==    by 0x6D0B8D8: __gnu_cxx::new_allocator<boost::shared_ptr<min_heap> >::destroy(boost::shared_ptr<min_heap>*) (new_allocator.h:107)
==993==    by 0x6D0CF5D: std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > >::erase(__gnu_cxx::__normal_iterator<boost::shared_ptr<min_heap>*, std::vector<boost::shared_ptr<min_heap>, std::allocator<boost::shared_ptr<min_heap> > > >) (vector.tcc:115)
==993==    by 0x6D09BF2: label_map::NextClosePoint() (label_map.cc:183)
==993==    by 0x6D05FC4: fast_march::march(cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&, cas::Imaging::Image<unsigned>&) (fast_march.cc:42)
==993==    by 0x6D11C5B: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:110)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993== 
==993== ERROR SUMMARY: 27 errors from 14 contexts (suppressed: 8 from 1)
==993== malloc/free: in use at exit: 43,435,937 bytes in 690,554 blocks.
==993== malloc/free: 851,137 allocs, 160,583 frees, 9,514,559,106 bytes allocated.
==993== For counts of detected errors, rerun with: -v
==993== searching for pointers to 690,554 not-freed blocks.
==993== checked 43,318,680 bytes.
==993== 
==993== 
==993== 24 bytes in 1 blocks are possibly lost in loss record 6 of 64
==993==    at 0x4A1A251: operator new[](unsigned long) (vg_replace_malloc.c:201)
==993==    by 0x4D7E1C3: cas::Fields::FieldIO::InitRead(int) (FieldIO.cc:234)
==993==    by 0x4D7E985: cas::Fields::FieldIO::ReadFrame(int, int, int, cas::Imaging::Image<unsigned char>&, cas::Fields::_MachineCoord&, long&) (FieldIO.cc:73)
==993==    by 0x4D83E0A: cas::Movie::ReadFrame(int, int, int, cas::Imaging::Image<unsigned char>&, cas::Fields::_MachineCoord&, long&) (Movie.cc:172)
==993==    by 0x6D11B08: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:103)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993== 
==993== 
==993== 34,608 bytes in 1,059 blocks are possibly lost in loss record 54 of 64
==993==    at 0x4A19DE3: operator new(unsigned long) (vg_replace_malloc.c:168)
==993==    by 0x527CC7C: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /usr/lib/libstdc++.so.6.0.7)
==993==    by 0x527CD1A: std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long) (in /usr/lib/libstdc++.so.6.0.7)
==993==    by 0x527DE05: std::string::reserve(unsigned long) (in /usr/lib/libstdc++.so.6.0.7)
==993==    by 0x5277B16: std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::overflow(int) (in /usr/lib/libstdc++.so.6.0.7)
==993==    by 0x527B67C: std::basic_streambuf<char, std::char_traits<char> >::xsputn(char const*, long) (in /usr/lib/libstdc++.so.6.0.7)
==993==    by 0x52747DE: std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*) (in /usr/lib/libstdc++.so.6.0.7)
==993==    by 0x40792C: main (dds.cc:284)
==993== 
==993== 
==993== 60,824 bytes in 1 blocks are possibly lost in loss record 55 of 64
==993==    at 0x4A1A251: operator new[](unsigned long) (vg_replace_malloc.c:201)
==993==    by 0x6D181B5: min_heap::Size(int) (min_heap.cc:34)
==993==    by 0x6D0A7BC: label_map::CloseToAlive(boost::shared_ptr<point>) (label_map.cc:251)
==993==    by 0x6D0626E: fast_march::march(cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&, cas::Imaging::Image<unsigned>&) (fast_march.cc:76)
==993==    by 0x6D11C5B: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:110)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993== 
==993== 
==993== 5,505,032 bytes in 1 blocks are possibly lost in loss record 61 of 64
==993==    at 0x4A1A251: operator new[](unsigned long) (vg_replace_malloc.c:201)
==993==    by 0x6D09040: label_map::Init(float, cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&) (label_map.cc:49)
==993==    by 0x6D05F76: fast_march::march(cas::Imaging::Image<unsigned short>&, cas::Imaging::Image<float>&, cas::Imaging::Image<unsigned>&) (fast_march.cc:33)
==993==    by 0x6D11C5B: marching::run(boost::shared_ptr<dds::client>, boost::shared_ptr<dds::item>, boost::shared_ptr<dds::item>) (marching.cc:110)
==993==    by 0x4074BC: process(boost::shared_ptr<dds::item>) (dds.cc:259)
==993==    by 0x408FFC: main (dds.cc:531)
==993== 
==993== LEAK SUMMARY:
==993==    definitely lost: 0 bytes in 0 blocks.
==993==      possibly lost: 5,600,488 bytes in 1,062 blocks.
==993==    still reachable: 37,835,449 bytes in 689,492 blocks.
==993==         suppressed: 0 bytes in 0 blocks.
==993== Reachable blocks (those to which a pointer was found) are not shown.
==993== To see them, rerun with: --show-reachable=yes