September 2004 - Boost - lists.boost.org

Re: IOStream mmap too restrictive?
by Jonathan Graehl 03 Sep '04

03 Sep '04

> > >After reviewing this, I think I spoke too soon. The above restrictions are >consistent with Posix. I had thought that mmap would allow mapping beyond >the file current size, but it seems not. One should use ftruncate to set >the size before mmap instead. > Since ftruncate isn't portable, it seems something like this (warning: untested) would be appropriate for boost::filesystem (perhaps taking a path object as an argument instead of a string - and shouldn't the Boost IO library do the same?) #ifdef _WIN32 #include <io.h> #else #include <unistd.h> #endif bool create_file(const std::string& path,std::size_t size) { #ifdef _WIN32 int fh=::_open(path.c_str(),_O_CREAT|_O_SHORT_LIVED); if (fh == -1) return false; if (::_chsize(fh,size) == -1) return false; return ::_close(fh) != -1; #else return ::truncate(path.c_str(),size) != -1; #endif } I don't see how, in Windows, to atomically keep the file open before it's mmapped, because chsize takes an int fh and the (equivalent of) mmap takes a HANDLE (maybe there is some Win32 call to get the HANDLE from the int fh?). The abstraction/interface that seems to be missing here is a low level (operating system level) file descriptor that can be passed to platform specific functions like ftruncate or _chsize, which weren't anticipated and wrapped by the higher-level libraries. I'd also like to see a constructor for mapped_file that creates a new file of exactly the required size (this might be a reasonable request for mapped_file_sink as well). It might also be nice to be able to set MAP_SHARED vs MAP_PRIVATE, or to be allowed to atomically deactivate an existing mapped_file (substituting a new backing file), guaranteeing the same address space (this is, I think, feasible if you do no allocation between munmap and mmap, and you can always detect failure by setting MAP_FIXED). I guess there's enough work left to be done for a "proper" Windows/POSIX mmap wrapper that the IO library shouldn't attempt to promote it yet.

2 4

RE: [boost] Checking if RTTI is available.
by Stefan Seefeld 02 Sep '04

02 Sep '04

> From: brangdon(a)cix.compulink.co.uk > [mailto:brangdon@cix.compulink.co.uk] > Sent: September 2, 2004 17:05 > In particular, VC++ ships with MFC, which has its own hand-rolled > solution. If you use that, you may not want the overhead of RTTI. Yeah, I noticed, MSVC is full of 'hand-rolled' 'solutions'. One more reason to avoid it. Sorry, couldn't resist. Regards, Stefan

1 0

Re: [boost] Re: [1.32 Release] MPL has been merged into the main trunk
by JOAQUIN LOPEZ MU?Z 02 Sep '04

02 Sep '04

----- Mensaje original ----- De: Eric Niebler <eric(a)boost-consulting.com> Fecha: Jueves, Septiembre 2, 2004 8:11 pm Asunto: [boost] Re: [1.32 Release] MPL has been merged into the main trunk > > > Jonathan Turkanis wrote: > > > "Eric Niebler" <eric(a)boost-consulting.com> wrote in message > > news:41374F88.9010100@boost-consulting.com... > > > >>Aleksey Gurtovoy wrote: > >> > >>>Everything has been tested locally, so the disturbance should > be minimal. > >>>I'll post a detailed changelog later today. Metacomm > regressions have > >>>been restored to run against the main trunk. > >>> > >> > >>Yikes! Where did apply_if go? The docs still talk about > apply_if. Is its > >>absence an oversight or by-design? > > > > > > Can I assume it's now 'eval_if'? > > > > If so, I like it better, since there's no temptation to confuse > apply_if and > > apply. > > > > Jonathan > > > > Oh. How do I make my code work with old versions of MPL and new? > There > is no mpl/version.hpp or a macro I can test for conditional > compilation > AFAICT. I'll have to avoid both mpl::apply_if and mpl::eval_if and > roll > my own. :-( I think you can use BOOST_VERSION in boost/version.hpp for this purpose. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo

1 0

RE: [boost] Checking if RTTI is available.
by Stefan Seefeld 02 Sep '04

02 Sep '04

> From: Jurko Gospodnetiæ [mailto:jurko@zse.hr] > Sent: September 2, 2004 04:14 > Hi all. > > I was wondering if boost has any portable way for detecting whether > RTTI is enabled? (For instance a macro expanding to _CPPRTTI in > Visual Studio). If there is, I have not been able to find it. Out of curiosity: why would anybody not use RTTI ? Why shouldn't compilers without RTTI just considered broken ? Being able to switch RTTI support off just introduces binary incompatibilities and makes some code illegal (you can't cast between classes with a virtual inheritance relationship without dynamic_cast, etc.) Regards, Stefan

2 1

RE: [boost] Re: Porting to SCO Unix
by Glen Knowles 02 Sep '04

02 Sep '04

> From: David Abrahams > "Peter Dimov" <pdimov(a)mmltd.net> writes: > > > John Maddock wrote: > >>> ** checking pthread_yield... no > >>> This is correct, but I do have a pthread_yield_np() > function, maybe > >>> we should look for it, too. > >> > >> It's up to the thread lib guys to use it (or not), maybe you could > >> hassle the authors of the thread lib you are using to provide a > >> standard function signature? > > > > There is no pthread_yield in POSIX. Yields are evil anyway. ;-) > > Why evil? There's a lot of backing for this, and the best single comment is probably: ....Similarly, sched_yield() can be used to resolve some problems on a uniprocessor, though such problems can usually be solved more cleanly in other ways. Using sched_yield() will never solve a problem (unless the problem statement is "the performance is too good") on a multiprocessor, and you should never use it there. - Butenhof Glen

1 0

RE: [boost] Re: IOStreams formal review start
by Reece Dunn 02 Sep '04

02 Sep '04

Jonathan Turkanis wrote: >"Daryle Walker" <darylew(a)hotmail.com> wrote: > > On 8/28/04 8:09 PM, "Jeff Garland" <jeff(a)crystalclearsoftware.com> >wrote: > > 2. This library does what a lot of other text-I/O libraries do, try to >fit > > in "kewl" compression schemes. The problem is that the types of >compression > > here are binary oriented; they convert between sets of byte streams. > > However, characters are not bytes (although characters, like other >types, > > are stored as bytes). > >Are you saying there are problems with the implementation of the >compression >filters, e.g., that they make unwarranted assumptions about 'char'? If so, >please let me know. I'm sure it can be fixed. If you want to read a bzip2 file (for example), the bzip2 file will be a binary stream and the resulting stream will be either binary or text depending on what the compressed file stores. You can tell std::ifstream to read the stream as binary, so what is the problem? Also, you might want to convert between text and binary modes: file > bzip2 (binary) > text > (process) > bzip2 (binary) > file > > 1. Actual code using this library is very slick and easy to set up. >This > > ease of use/set-up also applies to the plug-in filters and/or resources. I also like the ability to chain filters. I have a program that uses a similar mechanism on a character tape stream that allows text to be split to a certain character length and/or merged back into one block. It would be interesting to compare the performance of using my program verses an implementation using this library. >Can I rephrase this as follows: InputFilters and OutputFilters are a useful >addition to the standard library, but Sources and Sinks just duplicate >functionality alread present? If this is not your point please correct me. > >There are two main resons to write Sources and Sinks instead of stream >buffers: > >1. Sources and Sinks and sinks express just the core functionality of a >component. Usually you have to implement just one or two functions with >very >natural interfaces. You don't have to worry about buffering or about >putting >back characters. This is similar to what iterator adaptors and flex_string do for iterators and std::strings respectively. This makes it easy to write standard conforming components. For example, if you wanted to add support for a COM-based IStream, it would be easy to implement a source and/or sink. I will post a more detailed review once I have had a better look through the documentation and library, but at the moment it looks good. Regards, Reece _________________________________________________________________ Use MSN Messenger to send music and pics to your friends http://www.msn.co.uk/messenger

4 14

serialization, pickling, and mersenne_twister
by Neal D. Becker 02 Sep '04

02 Sep '04

I propose to add the following to mersenne_twister, which will support both the [proposed] boost::serialization as well as any other use: Index: mersenne_twister.hpp =================================================================== RCS file: /cvsroot/boost/boost/boost/random/mersenne_twister.hpp,v retrieving revision 1.16 diff -c -r1.16 mersenne_twister.hpp *** mersenne_twister.hpp 27 Jul 2004 03:43:32 -0000 1.16 --- mersenne_twister.hpp 2 Sep 2004 16:44:07 -0000 *************** *** 133,138 **** --- 133,145 ---- return os; } + template<class Archive> + void serialize(Archive & ar, const unsigned int version) + { + ar & x; + ar & i; + } + template<class CharT, class Traits> friend std::basic_istream<CharT,Traits>& operator>>(std::basic_istream<CharT,Traits>& is, mersenne_twister& mt) For use with boost::python pickling, I considered 2 approaches. 1) Use boost::serialization. This would require providing an archive format that would archive via pickling. 2) Use an independent technique, but share the same interface "serialize". I believe #2 is the best choice. Layering pickling on top of serialization would result in duplication and inefficiency. Although #2 is a bit awkward, it is feasible. Here is an example of how to serialize mersenne_twister via the above interface, using boost::python pickle. Perhaps this technique could be made more generic. struct mt_pickle_suite : python::pickle_suite { struct Archive { struct Save { python::list result; void operator & (rng_t::result_type x[]) { python::list l; for (int i = 0; i < 2*rng_t::state_size; i++) l.append (x[i]); result.append (l); } template<typename T> void operator & (const T& x) { result.append (x); } }; struct Load { python::object entries; Load (const python::object& _entries) : entries (_entries) {} void operator & (rng_t::result_type x[]) { python::list l = python::extract<python::list> (entries[0]); for (int i = 0; i < 2*rng_t::state_size; i++) x[i] = python::extract<rng_t::result_type> (l[i]); } template<typename T> void operator & (T& x) { x = (python::extract<T>(entries[1])); } }; }; static python::object getstate (rng_t& rng) { Archive::Save ar; rng.serialize (ar, 0); return ar.result; } static void setstate(rng_t& rng, python::object entries) { Archive::Load ar (entries); rng.serialize (ar, 0); } };

1 0

Changes to numeric/ublas/vector.hpp break ublas bindings
by Matthias Troyer 02 Sep '04

02 Sep '04

Yesterday's change in numeric/ublas/vector.hp in the Boost CVS (making the pointer typedefs private) breaks the ublas LAPACK and BLAS bindings which need access to this type. Is it possible to make these types public again? Matthias

1 0

RE: [boost] Re: New design proposal for boost::filesystem
by Ben Hutchings 02 Sep '04

02 Sep '04

Beman Dawes <bdawes(a)acm.org> wrote: > At 01:53 PM 8/30/2004, Carlo Wood wrote: > > >My idea on this was (and still is) that it is possible to convert > >a relative path to an absolute path (automatically) by adding an > >absolute path object reference to the relative path object. > > Yes, as long as you know the absolute path reference. Is the absolute > path reference known for all operating systems? I'm not sure about > that. For example, there have been multi-rooted systems that didn't > have a concept of "current drive". How would that work? <snip> Further, I know of one OS that supports directories but not a current directory: Windows CE. All paths passed to its filesystem functions must be absolute.

2 1

RE: [boost] Re: IOStream mmap too restrictive?
by Ben Hutchings 02 Sep '04

02 Sep '04

Jonathan Graehl <jonathan(a)graehl.org> wrote: <snip> > I don't see how, in Windows, to atomically keep the file open before > it's mmapped, because chsize takes an int fh and the (equivalent of) > mmap takes a HANDLE (maybe there is some Win32 call to get the HANDLE > from the int fh?). <snip> You mean file descriptor, not "int fh". "File handle" usually means a FILE or FILE pointer. Anyway, file descriptors are a Unix feature emulated by many C libraries and not by Win32. The VC++ library provides the function _get_osfhandle which returns the underlying Win32 file handle for a file descriptor. Other C/C++ libraries for Windows may provide similar functions.

1 0