integrating NaCl support into boost
Hi boost community, some introdution: NaCl (Google NativeClient, https://developer.chrome.com/native-client) - open-source technology authored by Google Inc. providing way to run native code inside web-browser in portable and secure manner. NaCl provides posix-like environment, and supports newlib and glibc standard libraries. currently NaCl is supported by Google Chrome web-browser, and range of applications and software libraries are ported to NaCl (include zlib, quake, etc.). how about integrating NaCl support patches into the boost main-line (trunk/master), I am talking about these NaCl ports patches from Google: https://chromium.googlesource.com/external/naclports/+/master/ports/boost (I actually have a bit more correct patches that allow to compile boost 1.59 for NaCl platforms, while Google patches are still for old version 1.55). some details about patches and which boost areas are covered by patches: - boost/config - common setup for NaCl - boost/cstdint - small change to expose boost::uintptr_t/boost::intptr_t for NaCl, similarly to how it was done to other compilers - buitin.jam and gcc.jam - introduced new os-name for NaCl, in order to properly setup threading libraries (NaCl has pthread but no rt, just like BSD) - boost/system/errorcode - oddball version of strerror_r (just like cygwin), also ENOTRECOVERABLE & EOWNERDEAD errno codes are not supported by PNaCl - boost/log/light_rw_mutex - not supported on NaCl note that boost build is incomplete - certain libraries are not (yet) ported to NaCl, because of platform-specific stuff like file/network I/O : - context - coroutine - coruutine2 - log - filesystem - test - timer - wave not sure if it's OK for start, or all boost libraries should be ported together in order to process further. I would be grateful to hear suggestion about how to process with it, e.g. which actions should I take to make it really happen. I personally suppose making boost to officially support more platforms will make boost even more popular and allow more developers to use it and port their applications to other platforms. I look forward to hearing from you soon. yours sincerely, Konstantin
On 16/09/2015 15:41, Konstantin Ivlev wrote:
Hi boost community,
some introdution: NaCl (Google NativeClient, https://developer.chrome.com/native-client) - open-source technology authored by Google Inc. providing way to run native code inside web-browser in portable and secure manner. NaCl provides posix-like environment, and supports newlib and glibc standard libraries. currently NaCl is supported by Google Chrome web-browser, and range of applications and software libraries are ported to NaCl (include zlib, quake, etc.).
how about integrating NaCl support patches into the boost main-line (trunk/master), I am talking about these NaCl ports patches from Google:
Is there an easy way to have a working NaCl environment to build and execute boost tests? That would certainly help a lot. Ion
On 16/09/2015 14:41, Konstantin Ivlev wrote:
Hi boost community,
some introdution: NaCl (Google NativeClient, https://developer.chrome.com/native-client) - open-source technology authored by Google Inc. providing way to run native code inside web-browser in portable and secure manner. NaCl provides posix-like environment, and supports newlib and glibc standard libraries. currently NaCl is supported by Google Chrome web-browser, and range of applications and software libraries are ported to NaCl (include zlib, quake, etc.).
how about integrating NaCl support patches into the boost main-line (trunk/master), I am talking about these NaCl ports patches from Google:
https://chromium.googlesource.com/external/naclports/+/master/ports/boost
(I actually have a bit more correct patches that allow to compile boost 1.59 for NaCl platforms, while Google patches are still for old version 1.55).
some details about patches and which boost areas are covered by patches:
- boost/config - common setup for NaCl - boost/cstdint - small change to expose boost::uintptr_t/boost::intptr_t for NaCl, similarly to how it was done to other compilers - buitin.jam and gcc.jam - introduced new os-name for NaCl, in order to properly setup threading libraries (NaCl has pthread but no rt, just like BSD) - boost/system/errorcode - oddball version of strerror_r (just like cygwin), also ENOTRECOVERABLE & EOWNERDEAD errno codes are not supported by PNaCl - boost/log/light_rw_mutex - not supported on NaCl
note that boost build is incomplete - certain libraries are not (yet) ported to NaCl, because of platform-specific stuff like file/network I/O :
- context - coroutine - coruutine2 - log - filesystem - test - timer - wave
not sure if it's OK for start, or all boost libraries should be ported together in order to process further.
I would be grateful to hear suggestion about how to process with it, e.g. which actions should I take to make it really happen. Sounds like a reasonable idea, only issue I see (and this is a general one rather than NaCl specific) is that there seems to be an explosion of platforms lately, and it seems unlikely that we can keep up to date with them all.
But leaving that aside I suggest you start with PR's for Config and Build - which would enable some level of testing to proceed - and then proceed with patches for other libraries from there. One wrinkle - I notice that Boost.Test is listed as unsupported, but without that you won't actually be able to run meaningful tests - how much work would it be to get that supported? John.
I personally suppose making boost to officially support more platforms will make boost even more popular and allow more developers to use it and port their applications to other platforms.
I look forward to hearing from you soon.
yours sincerely, Konstantin
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
thank you guys for your answers. > But leaving that aside I suggest you start with PR's for Config and Build - which would enable some level of testing to proceed - and then proceed with patches for other libraries from there. okay, I will start with this. so do I right understand that general strategy is to devide large patch into several small patches, each one for corresponding boost sub-module, and get them reviewed/approved/marged separately? > One wrinkle - I notice that Boost.Test is listed as unsupported, but without that you won't actually be able to run meaningful tests - how much work would it be to get that supported? I suppose this shouldn't be a big deal getting Boost.Test built for NaCl, probably 1 or 2 days. currently I have 3 kinds of compiler errors for Boost.Test: [exec] libs/timer/src/cpu_timer.cpp:103: error: ‘_SC_CLK_TCK’ was not declared in this scope NaCl defines _SC_CLK_TCK in its sys/unistd.h, but boost doesn't see it for some reason. [exec] ./boost/test/execution_monitor.hpp:492: error: ‘FE_DIVBYZERO’ was not declared in this scope [exec] ./boost/test/execution_monitor.hpp:493: error: ‘FE_INEXACT’ was not declared in this scope and many similar errors for FE_* constants (I see them as part of C++11 according to www.cplusplus.com/reference/cfenv/FE_DIVBYZERO/). for some reason at least one NaCl tool-chain (newlib) doesn't define mentioned constants. I need to investigate more purpose of that constants and how to properly deal with them in NaCl. [exec] ./boost/test/impl/debug.ipp:897: error: ‘::select’ has not been declared that's one weird thing in NaCl - it doesn't define select in its headers (at least for newlib), but it's available in nacl_io static library like most BSD sockets functions (altought other functions like recv/send/etc are defined in headers). in my code and multiple other ports (e.g. GLib) I've solved that by adding forward declaration of select prototype specially for NaCl. that's all kind of errors with Boost.Test. other libraries have their own issues. please also note that NaCl provides multiple tool-chains: - nacl arm newlib - nacl x86 newlib (can build x86 and x64 binaries) - nacl x86 glibc (can build x86 and x64 binaries) - pnacl so technically there are at least 6 different build configurations for NaCl, each of them has its own headers with different portability issues and limitations. I am currently building ofr nacl x86 newlib and verifying boost patches on it. however, I can also ensure patches works for the rest configurations (I think it will make sense for other people). > Is there an easy way to have a working NaCl environment to build and execute boost tests? That would certainly help a lot. that depends on which way is most comfortable for you? for example I can suggest the following: - I can make Virtual Box image with NaCl environment build set up and share - I also can prepare docker container - I can write puppet module which will install all necessary stuff - I can share my own build environment with you and boost community and provide SSH access to it - if you prefer to installing environment on your own, you can just follow instructions on https://developer.chrome.com/native-client/sdk/download let me know which way is the most comfortable for you and I will take necessary actions to prepare build environment. 2015-09-16 22:48 GMT+06:00 Ion Gaztañaga <igaztanaga@gmail.com>: > On 16/09/2015 15:41, Konstantin Ivlev wrote: > >> Hi boost community, >> >> some introdution: >> NaCl (Google NativeClient, https://developer.chrome.com/native-client) - >> open-source technology authored by Google Inc. providing way to run native >> code inside web-browser in portable and secure manner. NaCl provides >> posix-like environment, and supports newlib and glibc standard libraries. >> currently NaCl is supported by Google Chrome web-browser, and range of >> applications and software libraries are ported to NaCl (include zlib, >> quake, etc.). >> >> how about integrating NaCl support patches into the boost main-line >> (trunk/master), I am talking about these NaCl ports patches from Google: >> > > Is there an easy way to have a working NaCl environment to build and > execute boost tests? That would certainly help a lot. > > Ion > > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost >
On 9/16/2015 11:17 PM, Konstantin Ivlev wrote: > thank you guys for your answers. > >> But leaving that aside I suggest you start with PR's for Config and Build > - which would enable some level of testing to proceed - and then proceed > with patches for other libraries from there. > > okay, I will start with this. so do I right understand that general > strategy is to devide large patch into several small patches, each one for > corresponding boost sub-module, and get them reviewed/approved/marged > separately? Yes. The best way is a pull request against the individual Boost library/tool ( https://svn.boost.org/trac/boost/wiki/StartModPatchAndPullReq ). A pull request should always be made against the 'develop' branch of a Boost library/tool. > >> One wrinkle - I notice that Boost.Test is listed as unsupported, but > without that you won't actually be able to run meaningful tests - how much > work would it be to get that supported? > > I suppose this shouldn't be a big deal getting Boost.Test built for NaCl, > probably 1 or 2 days. currently I have 3 kinds of compiler errors for > Boost.Test: > [exec] libs/timer/src/cpu_timer.cpp:103: error: ‘_SC_CLK_TCK’ was not > declared in this scope > NaCl defines _SC_CLK_TCK in its sys/unistd.h, but boost doesn't see it for > some reason. > [exec] ./boost/test/execution_monitor.hpp:492: error: ‘FE_DIVBYZERO’ was > not declared in this scope > [exec] ./boost/test/execution_monitor.hpp:493: error: ‘FE_INEXACT’ was not > declared in this scope > and many similar errors for FE_* constants (I see them as part of C++11 > according to www.cplusplus.com/reference/cfenv/FE_DIVBYZERO/). > for some reason at least one NaCl tool-chain (newlib) doesn't define > mentioned constants. I need to investigate more purpose of that constants > and how to properly deal with them in NaCl. > [exec] ./boost/test/impl/debug.ipp:897: error: ‘::select’ has not been > declared > that's one weird thing in NaCl - it doesn't define select in its headers > (at least for newlib), but it's available in nacl_io static library like > most BSD sockets functions (altought other functions like recv/send/etc are > defined in headers). in my code and multiple other ports (e.g. GLib) I've > solved that by adding forward declaration of select prototype specially for > NaCl. > that's all kind of errors with Boost.Test. other libraries have their own > issues. > > please also note that NaCl provides multiple tool-chains: > - nacl arm newlib > - nacl x86 newlib (can build x86 and x64 binaries) > - nacl x86 glibc (can build x86 and x64 binaries) > - pnacl > > so technically there are at least 6 different build configurations for > NaCl, each of them has its own headers with different portability issues > and limitations. I am currently building ofr nacl x86 newlib and verifying > boost patches on it. however, I can also ensure patches works for the rest > configurations (I think it will make sense for other people). > Typically in order for a library developer to test nacl he would need a Boost build toolset which he can use so that his Boost build command line would look like: b2 toolset=some_nacl_toolset etc. >> Is there an easy way to have a working NaCl environment to build and > execute boost tests? That would certainly help a lot. > > that depends on which way is most comfortable for you? for example I can > suggest the following: > - I can make Virtual Box image with NaCl environment build set up and share > - I also can prepare docker container > - I can write puppet module which will install all necessary stuff > - I can share my own build environment with you and boost community and > provide SSH access to it > - if you prefer to installing environment on your own, you can just follow > instructions on https://developer.chrome.com/native-client/sdk/download > > let me know which way is the most comfortable for you and I will take > necessary actions to prepare build environment. What would also be helpful is if someone from your group could contribute regression tests for nacl toolsets once everything else has been setup for Boost developers. > > > > 2015-09-16 22:48 GMT+06:00 Ion Gaztañaga <igaztanaga@gmail.com>: > >> On 16/09/2015 15:41, Konstantin Ivlev wrote: >> >>> Hi boost community, >>> >>> some introdution: >>> NaCl (Google NativeClient, https://developer.chrome.com/native-client) - >>> open-source technology authored by Google Inc. providing way to run native >>> code inside web-browser in portable and secure manner. NaCl provides >>> posix-like environment, and supports newlib and glibc standard libraries. >>> currently NaCl is supported by Google Chrome web-browser, and range of >>> applications and software libraries are ported to NaCl (include zlib, >>> quake, etc.). >>> >>> how about integrating NaCl support patches into the boost main-line >>> (trunk/master), I am talking about these NaCl ports patches from Google: >>> >> >> Is there an easy way to have a working NaCl environment to build and >> execute boost tests? That would certainly help a lot. >> >> Ion >> >> >> _______________________________________________ >> Unsubscribe & other changes: >> http://lists.boost.org/mailman/listinfo.cgi/boost >> > > _______________________________________________ > Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost >
On 17/09/2015 5:17, Konstantin Ivlev wrote:
that depends on which way is most comfortable for you? for example I can suggest the following: - I can make Virtual Box image with NaCl environment build set up and share - I also can prepare docker container - I can write puppet module which will install all necessary stuff - I can share my own build environment with you and boost community and provide SSH access to it - if you prefer to installing environment on your own, you can just follow instructions on https://developer.chrome.com/native-client/sdk/download
let me know which way is the most comfortable for you and I will take necessary actions to prepare build environment.
Thanks for the information. The first step would be to analyze how NaCl should be defined inside the Boost Build system. As a new toolset, as a gcc toolset plus a different OS, etc. and make Boost.Build work just pointing to the local NCL SDK installation. It's not trivial to answer. That way we could run tests locally and also run automated regression/reporting tests. Best, Ion
participants (4)
-
Edward Diener -
Ion Gaztañaga -
John Maddock -
Konstantin Ivlev