Re: [Boost-build] Bjam performance commentary.

From: Reece Dunn (msclrhd_at_[hidden])
Date: 2006-06-16 18:53:19

Sohail Somani wrote:
> > [mailto:boost-build-bounces_at_[hidden]] On Behalf Of Rene Rivera
> > 
> > For those who are posting about the performance of bjam, and ways to 
> > "solve" the problems. I need to point out that you need to 
> > ensure that 
> > what you are proposing to fix is actually broken. Specifically at 
> > minimum I expect people to look at the performance metrics 
> > before saying 
> > that some particular aspect of bjam is slow. Those are available, as 
> > documented, by adding "-d+10" option. For example:
> > 
> >    bjam -n -d+10 -a --v2 --builddir=C:/DevRoots/Boost/build 
> > --prefix=C:/DevRoots/Boost/install "toolset=gcc-3.4.5_mingw_st
> > lport51" "stdlib=stlport-5.1_gcc345" "variant=debug" 
> > "threading=multi" stage
> 
> Still seems to be file system related though.

Time-wise, the slowest performers from a run I did were:

3783   ==> boost-build
3783   ==> modules.import
3783   ==> modules.load
120721   ==> MAIN_MAKE
124754   ==> MAIN

And for memory:

95856   ==> class_at_property-map.find-replace
97388   ==> path.native-NT
102384   ==> property.validate-property-sets
109319   ==> class.new
111850   ==> toolset.add-flag
115450   ==> class_at_file-target.path
121739   ==> class_at_abstract-file-target.actual-name
136289   ==> class_at_file-target.actualize-location
142555   ==> virtual-target.register
153961   ==> BINDMODULE
163276   ==> class_at_action.actualize
194811   ==> generators.try-one-generator
195848   ==> class.xinit
199740   ==> generators.construct-really
208397   ==> class_at_virtual-target.actualize
216968   ==> INCLUDES
224948   ==> NORMALIZE_PATH
263471   ==> toolset.flags
293306   ==> class_at_abstract-file-target.actualize-no-scanner
297869   ==> toolset.normalize-condition
309497   ==> timestamp
310917   ==> generators.viable-source-types-for-generator
319717   ==> MATCH
324096   ==> MAIN
341462   ==> FILE_DIRSCAN
454081   ==> feature.split
513771   ==> MAIN_MAKE
671364   ==> scanner.hdrrule
679758   ==> generators.find-viable-generators
828412   ==> toolset.find-property-subset
872041   ==> toolset.set-target-variables-aux
1130129   ==> toolset.set-target-variables
2321852   ==> scanner.propagate
3263234   ==> regex.split
3267496   ==> modules.load
3996980   ==> class_at_c-scanner.process

So modules.load seems like a good place to optimise :). Reducing the
memory overhead will also gain speed -- I suspect that this is where
a lot of time is going. That means:

   toolset.set-target-variables
   scanner.propagate
   regex.split
   modules.load
   class_at_c-scanner.process

are good candidates for optimising. It will be interesting to see what
impact the Python port will have on these figures as Python has its
own regex module, deals natively with module imports and has very
good string split/join facilities.

- Reece
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/