Re: [boost] Re: generic polyvalent input/output ['pickle']

From: Beman Dawes (beman_at_[hidden])
Date: 2000-11-10 09:15:44

At 01:11 AM 11/10/2000 +0000, krempp_at_[hidden] wrote:

 >I had not noticed persistence in my exploration of the Files sections
 >in search of things that would interest me..
 >Now I just had a look at it.
 >It seems to aim at the same goal as what I had in mind with pickle.
 >with some differences..

It is clear there is demand for light-weight persistence; it was on the 
original list of desired boost libraries.  ("light-weight" is probably the 
wrong term; I'm trying to differentiate this type of persistence from 
object database style heavy-weight persistence.)

 >>Several of us have been hoping that Jens will restart his
 >development
 >>effort when he gets back from vacation in a few days.
 >
 >humm.. persistence files are dated June 2000. Impressive vacations..
 >What kind of job does he have ? maybe it's not too late for me to take
 >the right career directions...  ;-)

Yes, well....  Seriously, Jens was at the Boost and C++ Committee meetings 
in Toronto a couple of weeks ago, and he and I discussed finishing 
persistence.  He went on vacation after that, and won't be home until 
Sunday, IIRC.

I'm hoping he will respond to your comments early next week.

 >
 >I also dicovered the few (4) messages in the list about
 >persistence. including the one with this link
 >http://x42.deja.com/getdoc.xp?AN=577017164&CONTEXT=952360117.1170735127&hit=
 >
 >num=0
 >which also brings good ideas.
 >Did anybody get more details on its approach ? I like the fact that
 >you just describe what are the members, not implement write + load ..
 >(it only doubles the code, but..)
 >And also GetName() being specifiable instead of always relying on RTTI
 >names , is similar to what I considered.
 >[ Dumping the mangled typeid(T).name() is unreadable...]
 >
 >BTW, why printing type name only when writing to file ? it is always a
 >very valuable information about the written object.
 >shift_writer should do it by default..

Actually, I think we were leaning the other way; not writing the name at 
all.

 >
 >Here are the first critics I could thought of :
 >
 >1_ Persistence lacks nesting of data within convenient block markers.

Yes, that has been the primary hold up.

 > saving a vector<vector<float> >  creates output unreadable by human
 >without nesting.
 >Maybe human readability is not a concern for save_file.
 >But I take it as a primary goal..
 >OTOH, implementing a full XML-like nesting system might be too verbose
 >in some situations.
 >But we may find a way to specify the desired level of decoration.
 >Maybe through a template parameter class 'Decorater' ?  It would hold
 >methods for the different occasions when printing additional
 >'human-helpers' characters makes sense
 >eg :
 >-iteration in a container.
 >  nth iteration could print '#n'
 >-begin/end of a container
 > could print { and },
 > or XML-like "</"+description_name()+">.
 >maybe some "\n" from time to time..

Yes, some of the hacked up versions of persistence circulating privately do 
that.

Basically, you are going over the much the same ground as I and Jens did in 
our private discussions.  But I encourage you to keep at it; clearly you 
have some good ideas that we (or at least I) hadn't thought of.

 > ...

 >3_
 >binary could be made a bit portable...

By "bit portable..." I assume you mean with a specified bit order, so that 
the resulting file is reasonably portable.

A better solution might be to supply additional "portable_binary" readers 
and writers.

1) That still leaves the more efficient binary_reader and binary_writer 
available for those who prefer that approach.

2) portable_binary_reader and portable_binary_writer would almost surely 
rely on unspecified behavior (things like the order bits are actually 
written to external memory), even though in practice they would work as 
intended for virtually all modern hosted systems.

3) Both big-endian and little-endian schemes need to be supported, to 
interface with other software, as previously pointed out by Darin Adler and 
others.

 >
 >4_
 >I prefer loading/saving via calling operator << and >> on an object.
 >I find that
 >boost::ipickle<boost::shift_reader>  pic(std::ifstream("data.txt"));
 >pic>>database;
 >
 >makes nicer code (without even needing a wrapper) than
 >  boost::load_file<boost::shift_reader>(database, "data.txt");
 >even more when you use pic several times consecutively...
 >
 >[ supposing shift_reader is modified to always print some
 >implementation of name_of_type<T>()  as a first word . ]

Thanks for the interesting comments.  Hopefully they will help rejuvenate 
the boost persistence effort.  But give Jens a few days to get home and 
respond.

--Beman