[Boost-bugs] [Boost C++ Libraries] #10718: program_options: Multitoken option is limited to 32000 token

Subject: [Boost-bugs] [Boost C++ Libraries] #10718: program_options: Multitoken option is limited to 32000 token
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2014-10-30 10:08:19

#10718: program_options: Multitoken option is limited to 32000 token
 Reporter: hans.hohenfeld@… | Owner: vladimir_prus
     Type: Bugs | Status: new
Milestone: To Be Determined | Component: program_options
  Version: Boost 1.55.0 | Severity: Problem
 Keywords: |
 Dear all,

 I've created an application using boost::program_options to handle command
 line parameters. Beyond several other parameters, it uses a multitoken
 parameter to handle input file names passed. The application is basically
 executed like:

 ./prog -i *.nii

 where *.nii resolves to a substantial number of files.

 The relevant part of my application's code is

 typedef std::vector<std::string> FileContainer;
 FileContainer files;
 namespace po = boost::program_options;


 po::options_description opts("Test");
 opts.add_options() ("input,i",
                     "The input file(s)");

 po::variables_map vm;
 po::store(po::parse_command_line(ac, av, opts), vm);

 So far it works very well. A problem occurs, as soon as the number of
 input files and therefore the number of tokens exceeds 32000 (which is a
 regular scenario, as the application is running on a HPC cluster and is
 working with massive amounts of data). In such cases, the first 32000 file
 names are parsed, the remaining ones are skipped silently.

 I searched a little in the program_options code and found the reason for
 this behavior in


 from line 315

        unsigned max_tokens() const {
             if (m_multitoken) {
                 return 32000;
             } else if (m_zero_tokens) {
                 return 0;
             } else {
                 return 1;

 This is the case for all recent boost versions, including the latest from

 I replaced the 32000 with UINT_MAX from <climits> and it seems to work.
 Now my questions:

 1. Is 32000 intentional? UINT_MAX would also create an upper limit, but a
 much larger one.
 2. Is there a reason to "fail" silently, when the limit is reached? If
 not, I'd suggest to add an error/warning or something.

 I assume this is a rare use case and therefor won't cause any problems for
 the majority of applications and furthermore there are plenty of
 workaround (including simply not using boost::program_options), but

 If you need any further details, just let me know.

 Thank you.

 best regards,


Ticket URL: <https://svn.boost.org/trac/boost/ticket/10718>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.

This archive was generated by hypermail 2.1.7 : 2017-02-16 18:50:17 UTC