[program options] vectro from config file
Hi All, I can't find an answer to this before, so I'll ask here. I would like to read a vector from a config file using program_options. example definition of option: ("Expt.ControlOrd", po::value<vector<int>>(&_pcData->SetExptOrders()), "Expt.ControlOrd") In the config file, the following works: ControlOrd = 1 ControlOrd = 2 But i would much prefer to use: ControlOrd = 1 2 or ControlOrd = (1 2) or something like that. I have made this work by editing the sourcecode, diff: ---- Index: config_file.cpp =================================================================== --- config_file.cpp (revision 54915) +++ config_file.cpp (working copy) @@ -12,6 +12,8 @@ #include <boost/program_options/detail/convert.hpp> #include <boost/throw_exception.hpp> +#include <boost/algorithm/string.hpp> + #include <iostream> #include <fstream> #include <cassert> @@ -112,8 +114,17 @@ found = true; this->value().string_key = name; this->value().value.clear(); - this->value().value.push_back(value); this->value().unregistered = !registered; + + if (*value.begin() == '(' && *value.rbegin() == ')') + { + value = value.substr(1, value.size()-2); + vector<string> SplitVec; + split( SplitVec, value, is_any_of(" ") ); + this->value().value.swap(SplitVec); + } + else + this->value().value.push_back(value); break; } else { ---- Of course, this is not a great solution as it doesn't allow you to read strings starting with ( and ending with ). Well, it is missing one thing. If we could somehow check that the datatype requested for this options is a vector, we could add that check to the if. I think that leaves only one problem, that of reading a vector of strings. Please tell me if it is possible to check for the requested type at this point in the code and how to do it. Any other comments also very welcome! I'll suggest it to the author once it works well. Thank you, Diederick
Diederick C. Niehorster wrote:
Hi All,
I can't find an answer to this before, so I'll ask here.
I would like to read a vector from a config file using program_options.
example definition of option: ("Expt.ControlOrd", po::value<vector<int>>(&_pcData->SetExptOrders()), "Expt.ControlOrd")
In the config file, the following works: ControlOrd = 1 ControlOrd = 2
But i would much prefer to use: ControlOrd = 1 2 or ControlOrd = (1 2) or something like that.
I have made this work by editing the sourcecode, diff: ---- Index: config_file.cpp =================================================================== --- config_file.cpp (revision 54915) +++ config_file.cpp (working copy) @@ -12,6 +12,8 @@ #include <boost/program_options/detail/convert.hpp> #include <boost/throw_exception.hpp>
+#include <boost/algorithm/string.hpp> + #include <iostream> #include <fstream> #include <cassert> @@ -112,8 +114,17 @@ found = true; this->value().string_key = name; this->value().value.clear(); - this->value().value.push_back(value); this->value().unregistered = !registered; + + if (*value.begin() == '(' && *value.rbegin() == ')') + { + value = value.substr(1, value.size()-2); + vector<string> SplitVec; + split( SplitVec, value, is_any_of(" ") ); + this->value().value.swap(SplitVec); + } + else + this->value().value.push_back(value); break;
} else { ----
Of course, this is not a great solution as it doesn't allow you to read strings starting with ( and ending with ). Well, it is missing one thing. If we could somehow check that the datatype requested for this options is a vector, we could add that check to the if. I think that leaves only one problem, that of reading a vector of strings.
Please tell me if it is possible to check for the requested type at this point in the code and how to do it. Any other comments also very welcome!
It's not possible at this point of code. Config file parser is purely syntactic, so it does not know about types of options. Overriding 'validate' to handle () might be easier solution, though current design with validate as global function does not make it very convenient. There's proposal, on boost-devel, to allow specifying validate function per-option, that will help. - Volodya
Hi Volodya, Thank you for your reply. I have placed my code, improved, into the validate() function now (see below). Thats a good idea anyway, this way it works on input from any source. I am still stuck with the issue of how to test whether users specified a vector or not as return target in their options_description, that is: ("Tex.GaussSigma", po::value<float>(&fTexGaussSigma) , "Tex.GaussSigma") or ("Tex.Color" , po::value<vector<float>>(&vfTexColor), "Expt.Color") Due to the very complicated flow (if you've just looked at the code for a few hours) I find it hard to figure out. I suppose i have to run typeid(*something*).name() to get the type of some variable and then check if its a vector or not. But what should be in the place of *something*? As a general point, do you think this would be a good addition to the parser, that is, would you want to incorporate this kind of extention into the Boost release? It is a much easier way of specifying a vector, whether on the command line or from another source without having to resort to positional options, which are not a good solution for every source and situation. Compare (for command line) [--test "(1 2)"] with [--test 1 --test 2]. Lastly, in an old post (cant find it now, was 2005 or 2006) a user asked whether it was possible to do po::parse_config_file(InitFileName,poInit,true); instead of po::parse_config_file(ifs,poInit,true); as the documentation apparently reflected this at that point. You replied that you'd fix that at that time (I believe it was you, doesn't really matter), but when i just tried it didnt work. Have you decided against it? Best, Diederick ----- Index: detail/value_semantic.hpp =================================================================== --- detail/value_semantic.hpp (revision 54915) +++ detail/value_semantic.hpp (working copy) @@ -8,6 +8,8 @@ #include <boost/throw_exception.hpp> +#include <boost/algorithm/string.hpp> + namespace boost { namespace program_options { extern BOOST_PROGRAM_OPTIONS_DECL std::string arg; @@ -142,11 +144,28 @@ /* We call validate so that if user provided a validator for class T, we use it even when parsing vector<T>. */ - boost::any a; - std::vector<std::basic_string<charT> > v; - v.push_back(s[i]); - validate(a, v, (T*)0, 0); - tv->push_back(boost::any_cast<T>(a)); + + vector<std::basic_string<charT>> value; + + // test if vector notation is used in input + if (*s[i].begin() == '(' && *s[i].rbegin() == ')') + { + // TODO: test if vector is expected by user, if not, no spl itting + // TODO: if vector expected, test if its vector of strings, that will need special treatment + split( value, s[i].substr(1, s[i].size()-2), is_any_of(" ") ); + } + else + value.push_back(s[i]); + + // validate option values + for (unsigned j = 0; j < value.size(); j++) + { + boost::any a; + std::vector<std::basic_string<charT> > v; + v.push_back(value[j]); + validate(a, v, (T*)0, 0); + tv->push_back(boost::any_cast<T>(a)); + } } catch(const bad_lexical_cast& /*e*/) { boost::throw_exception(invalid_option_value(s[i]));
Hi, I have made another change to the code, in config_file.cpp this time. I noticed that collect_unrecognized() didn't work on output of the config file parser as it doesnt set the original_tokens field. Its doens't error, just doesn't collect anything. I have thus written the parser such that the keyname is written to the original_tokens field if the keyname is unregistered, allowing collect_unrecognized to do its job as expected. diff below. Is this the right list for this kind of discussion by the way? Best, Diederick ------ Index: config_file.cpp =================================================================== --- config_file.cpp (revision 54915) +++ config_file.cpp (working copy) @@ -114,6 +114,15 @@ this->value().value.clear(); this->value().value.push_back(value); this->value().unregistered = !registered; + + // add name to original_tokens so we can run collect_unregi stered + // over po::parsed_options from po::parse_config_file too + this->value().original_tokens.clear(); + if (!registered) + { + this->value().original_tokens.push_back(name); + } + break; } else { On Tue, Jul 14, 2009 at 12:53 PM, Diederick C. Niehorster<dcnieho@gmail.com> wrote:
Hi Volodya,
Thank you for your reply. I have placed my code, improved, into the validate() function now (see below). Thats a good idea anyway, this way it works on input from any source. I am still stuck with the issue of how to test whether users specified a vector or not as return target in their options_description, that is: ("Tex.GaussSigma", po::value<float>(&fTexGaussSigma) , "Tex.GaussSigma") or ("Tex.Color" , po::value<vector<float>>(&vfTexColor), "Expt.Color")
Due to the very complicated flow (if you've just looked at the code for a few hours) I find it hard to figure out. I suppose i have to run typeid(*something*).name() to get the type of some variable and then check if its a vector or not. But what should be in the place of *something*?
As a general point, do you think this would be a good addition to the parser, that is, would you want to incorporate this kind of extention into the Boost release? It is a much easier way of specifying a vector, whether on the command line or from another source without having to resort to positional options, which are not a good solution for every source and situation. Compare (for command line) [--test "(1 2)"] with [--test 1 --test 2].
Lastly, in an old post (cant find it now, was 2005 or 2006) a user asked whether it was possible to do po::parse_config_file(InitFileName,poInit,true); instead of po::parse_config_file(ifs,poInit,true); as the documentation apparently reflected this at that point. You replied that you'd fix that at that time (I believe it was you, doesn't really matter), but when i just tried it didnt work. Have you decided against it?
Best, Diederick
----- Index: detail/value_semantic.hpp =================================================================== --- detail/value_semantic.hpp (revision 54915) +++ detail/value_semantic.hpp (working copy) @@ -8,6 +8,8 @@
#include <boost/throw_exception.hpp>
+#include <boost/algorithm/string.hpp> + namespace boost { namespace program_options {
extern BOOST_PROGRAM_OPTIONS_DECL std::string arg; @@ -142,11 +144,28 @@ /* We call validate so that if user provided a validator for class T, we use it even when parsing vector<T>. */ - boost::any a; - std::vector<std::basic_string<charT> > v; - v.push_back(s[i]); - validate(a, v, (T*)0, 0); - tv->push_back(boost::any_cast<T>(a)); + + vector<std::basic_string<charT>> value; + + // test if vector notation is used in input + if (*s[i].begin() == '(' && *s[i].rbegin() == ')') + { + // TODO: test if vector is expected by user, if not, no spl itting + // TODO: if vector expected, test if its vector of strings, that will need special treatment + split( value, s[i].substr(1, s[i].size()-2), is_any_of(" ") ); + } + else + value.push_back(s[i]); + + // validate option values + for (unsigned j = 0; j < value.size(); j++) + { + boost::any a; + std::vector<std::basic_string<charT> > v; + v.push_back(value[j]); + validate(a, v, (T*)0, 0); + tv->push_back(boost::any_cast<T>(a)); + } } catch(const bad_lexical_cast& /*e*/) { boost::throw_exception(invalid_option_value(s[i]));
Diederick C. Niehorster wrote:
Hi Volodya,
Thank you for your reply. I have placed my code, improved, into the validate() function now (see below). Thats a good idea anyway, this way it works on input from any source. I am still stuck with the issue of how to test whether users specified a vector or not as return target in their options_description, that is: ("Tex.GaussSigma", po::value<float>(&fTexGaussSigma) , "Tex.GaussSigma") or ("Tex.Color" , po::value<vector<float>>(&vfTexColor), "Expt.Color")
Due to the very complicated flow (if you've just looked at the code for a few hours) I find it hard to figure out. I suppose i have to run typeid(*something*).name() to get the type of some variable and then check if its a vector or not. But what should be in the place of *something*?
I think you are changing the validate overload that is for vectors, not? It won't ever be called for non-vectors.
As a general point, do you think this would be a good addition to the parser, that is, would you want to incorporate this kind of extention into the Boost release? It is a much easier way of specifying a vector, whether on the command line or from another source without having to resort to positional options, which are not a good solution for every source and situation. Compare (for command line) [--test "(1 2)"] with [--test 1 --test 2].
I am not yet sure this is good addition. In any case, it would be better to first allow to specify custom validate function for a given option.
Lastly, in an old post (cant find it now, was 2005 or 2006) a user asked whether it was possible to do po::parse_config_file(InitFileName,poInit,true); instead of po::parse_config_file(ifs,poInit,true); as the documentation apparently reflected this at that point. You replied that you'd fix that at that time (I believe it was you, doesn't really matter), but when i just tried it didnt work. Have you decided against it?
I honestly don't remember. If you care about this change, please file a bug at svn.boost.org, so that it's not forgotted for another 4 years. - Volodya
Hi Volodya,
Thank you for your reply. I have placed my code, improved, into the validate() function now (see below). Thats a good idea anyway, this way it works on input from any source. I am still stuck with the issue of how to test whether users specified a vector or not as return target in their options_description, that is: ("Tex.GaussSigma", po::value<float>(&fTexGaussSigma) , "Tex.GaussSigma") or ("Tex.Color" , po::value<vector<float>>(&vfTexColor), "Expt.Color")
Due to the very complicated flow (if you've just looked at the code for a few hours) I find it hard to figure out. I suppose i have to run typeid(*something*).name() to get the type of some variable and then check if its a vector or not. But what should be in the place of *something*?
I think you are changing the validate overload that is for vectors, not? It won't ever be called for non-vectors.
I indeed changed the validate() function that would only be called for vectors, good point! I was convinced by my memory when I wrote that email that I saw that function process all options, indeed it only processes the ones specified to be vector. Thanks!
As a general point, do you think this would be a good addition to the parser, that is, would you want to incorporate this kind of extention into the Boost release? It is a much easier way of specifying a vector, whether on the command line or from another source without having to resort to positional options, which are not a good solution for every source and situation. Compare (for command line) [--test "(1 2)"] with [--test 1 --test 2].
I am not yet sure this is good addition. In any case, it would be better to first allow to specify custom validate function for a given option.
I see that it is possible to define/override the validator for a user-defined data-type/class, this is not what you are suggesting, right? Instead, you are suggesting to make it possible to store a custom validator function in the option_description, which would then be called instead, right? Hmm, that would do the trick too, though that operates on a conceptually different level. I would then have to specify that custom validator for every option of type vector<T>. Instead, this is a suggestion for enhanced syntax for specifying vectors (that is, options that can occur multiple times) on the command line, in a config file or even in the environment variable (well, who'd do that?). It will not break any existing option input systems defined by users. IMHO, allowing this enhanced syntax is a simple, local change to the validate function for vectors that can be made quickly, whereas allowing to specify a custom validate function for a given option will be a much bigger hassle, though also a necessary thing (say you want only even ints as input for whatever reason, you'll need this). I'll do the coding (still need to take care of vectors of strings) and I'll put it in as a feature request/bug report so we can discuss this at the appropriate place.
Lastly, in an old post (cant find it now, was 2005 or 2006) a user asked whether it was possible to do po::parse_config_file(InitFileName,poInit,true); instead of po::parse_config_file(ifs,poInit,true); as the documentation apparently reflected this at that point. You replied that you'd fix that at that time (I believe it was you, doesn't really matter), but when i just tried it didnt work. Have you decided against it?
I honestly don't remember. If you care about this change, please file a bug at svn.boost.org, so that it's not forgotted for another 4 years.
Will do so, seems like a good addition, allowing less boilerplate in using config files.
- Volodya
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Diederick C. Niehorster wrote:
I see that it is possible to define/override the validator for a user-defined data-type/class, this is not what you are suggesting, right? Instead, you are suggesting to make it possible to store a custom validator function in the option_description, which would then be called instead, right? Hmm, that would do the trick too, though that operates on a conceptually different level. I would then have to specify that custom validator for every option of type vector<T>. Instead, this is a suggestion for enhanced syntax for specifying vectors (that is, options that can occur multiple times) on the command line, in a config file or even in the environment variable (well, who'd do that?). It will not break any existing option input systems defined by users. IMHO, allowing this enhanced syntax is a simple, local change to the validate function for vectors that can be made quickly, whereas allowing to specify a custom validate function for a given option will be a much bigger hassle, though also a necessary thing (say you want only even ints as input for whatever reason, you'll need this). I'll do the coding (still need to take care of vectors of strings) and I'll put it in as a feature request/bug report so we can discuss this at the appropriate place.
Thanks. Filing this at svn.boost.org is indeed good idea. No changes like this are likely to be in 1.40 (since the window has closed), and then I'll be able to ponder about this change. - Volodya
participants (2)
-
Diederick C. Niehorster -
Vladimir Prus