Re: [boost] [nowide] Easy Unicode For Windows: Request For Comments/Preliminary Review

Subject: Re: [boost] [nowide] Easy Unicode For Windows: Request For Comments/Preliminary Review
From: Artyom Beilis (artyomtnk_at_[hidden])
Date: 2012-05-29 10:25:45

Hello,

To make the purpose of Boost.Nowide more clear I'll add an example from the docs.


Let's write a simple program that confirms the C++2011 or C++2003 standard 
that counts a number of lines in the file:


    #include <fstream>
    #include <iostream>

    int main(int argc,char **argv)
    {
        if(argc!=2) {
            std::cerr << "Usage: file_name" << std::endl;
            return 1;
        }

        std::ifstream f(argv[1]);
        if(!f) {
            std::cerr << "Can't open a file " << argv[1] << std::endl;
            return 1;
        }
        int total_lines = 0;
        while(f) {
            if(f.get() == '\n')
                total_lines++;
        }
        f.close();
        std::cout << "File " << argv[1] << " has " << total_lines << " lines" << std::endl;
        return 0;
    }

Any Bugs?

This trivial program would not work on Windows if the file name is Unicode
file name, argv - does not hold Unicode string, std::ifstream can't open Uicode file name and
std::cout can't print Unicode characters to the console...

Boost.Nowide provides an alternative for common standard library function and suggest a general
pattern to handle Unicode strings in the cross platform program:

    #include <boost/nowide/fstream.hpp>
    #include <boost/nowide/iostream.hpp>
    #include <boost/nowide/args.hpp> 

    int main(int argc,char **argv)
    {
        //
        // Fix arguments - argv holds Unicode string (UTF-8)
        //
        boost::nowide::args a(argc,argv); 
        if(argc!=2) {
            boost::nowide::cerr << "Usage: file_name" << std::endl;
            return 1;
        }

        //
        // Fix fstream it can open a file using Unicode file name (UTF-8)
        //
        boost::nowide::ifstream f(argv[1]);
        if(!f) {
            //
            // cerr can print Unicode characters to console regardless console code page
            //
            boost::nowide::cerr << "Can't open a file " << argv[1] << std::endl;
            return 1;
        }
        int total_lines = 0;
        while(f) {
            if(f.get() == '\n')
                total_lines++;
        }
        f.close();
        //
        // cout can print Unicode characters to console regardless console code page
        //
        boost::nowide::cout << "File " << argv[1] << " has " << total_lines << " lines" << std::endl;
        return 0;
    }

This is the general approach, it also provides glue conversion functions to handle Unicode on API boundary 
level where needed


   #ifdef _WIN32
   bool copy_file(std::string const &src,std::string const &tgt)
   {
       return CopyFileW(boost::nowide::convert(src).c_str(),
                        boost::nowide::convert(tgt).c_str(),
                        TRUE);
   }
   #else
   bool copy_file(std::string const &src,std::string const &tgt)
   { 
      // POSIX implementation
   }
   #endif


Waiting for Comments:

Download:    http://cppcms.com/files/nowide/nowide.zip
Documents:   http://cppcms.com/files/nowide/html/


Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/



> Hello all Booster,
> 
> I wan to get comments on a library that I want to submit for a formal review.
> 
> The library provides an implementation of standard C and C++ library
> functions such that their inputs are UTF-8 aware on Windows without
> requiring using Wide API to make program work on Windows.
> 
> Library:    Boost.Nowide
> Download:    http://cppcms.com/files/nowide/nowide.zip
> Documents:   http://cppcms.com/files/nowide/html/
> Features:    http://cppcms.com/files/nowide/html/index.html#main_the_solution
> 
> 
> Tested On:
> 
> OS: Windows 7 32/64 bit, Linux
> Compilers: GCC-4.6, MSVC-10
> 
>