August 2015 - Boost - lists.boost.org

Re: [boost] [asio-users] [http] Formal review of Boost.Http
by Niall Douglas 16 Aug '15

16 Aug '15

On 7 Aug 2015 at 19:29, Vinícius dos Santos Oliveira wrote: > > I do not believe Http is ready for peer review for these reasons, and > > I personally would urge you withdraw it until you have fixed these > > issues and then come back to us. If you do not withdraw it, I will > > vote for rejection. > > Can I persuade you to review the Boost.Http design? I'd reject myself if > you find any fundamental flaw on the design and even if you're going to > reject the library because the lack of tooling (not directly related to > design or documentation, which are the most important items on any > library), it'd be useful to know what needs to be addressed on the design > before I submit again (in the case the submission is rejected). I think my earlier email was poorly phrased now I've slept on it. I made my objections seem like they were about portability, when they were really about design and intent of Http. What I should have said was this: Standalone ASIO is the reference Networking TS implementation for standardisation. Http is currently a set of extensions to Boost.ASIO specifically when it *should* be a set of extensions to the reference Networking TS implementation for C++ standardisation. This is why your current design and architecture is flawed, but luckily there isn't too much work to fix this by finishing the port of Http to the Networking TS. The Internet of Things tends to involve lots of small somewhat underpowered internet devices where C++ is likely going to play a much more important role than historically in embedded systems. The chances are that any Networking TS in C++ will be very extensively used in such IoT applications. If there were also a high quality set of extensions adding HTTP support to the Networking TS, I think your chances are good that your Http library would be seriously considered by WG21 for standardisation. *This* is why I very strongly believe that Http needs to cleave itself tightly to the Networking TS reference implementation, and not to Boost.ASIO. Moreover, if you make Http an extension of the Networking TS, you get Boost support with that for free - Http becomes dual use. > About C++03 support, it's not required for Boost acceptance, so I can add > later. I want to increase tests coverage a lot before porting code to > C++03, so I don't introduce accidental bugs in the process. As a strong advocate for C++ 11 over 03 I can understand. However, IoT developers tend to be stuck on some pretty ancient toolsets for longer than others. 03 support I think would greatly increase your potential userbase. > In the case the abstractions I need enter on the C++ standard, maybe your > APIbind will help me. I actually would advise you to choose ASIO's STL abstraction machinery over my own alternative APIBind in this instance. > But I just don't find any reference to BOOST_ASIO_HAS_STD_SYSTEM_ERROR > outside of config.hpp. Even if it was used, it is undocumented, which means > it is an implementation detail that won't be guaranteed to be there forever > (accidental behaviour versus a documented guarantee). I'm pretty sure that implementation detail won't change in a hurry. STL abstraction machinery changes as quickly as a STL does i.e. every decade or so. > > https://svn.boost.org/trac/boost/wiki/BestPracticeHandbook#a7.SAFETY:S > > tronglyconsideranightlyorweeklyinputfuzzautomatedtestifyourlibraryisab > > letoacceptuntrustedinput > > That's why this first implementation uses a proven-parser that is very well > tested. The buck always stops with the top most layer, not internally used third party libraries. You NEED to fuzz test untrusted inputs when parsing HTTP. For all you know, your STL implementation could be the thing with the overflow bug, or the way in which you use it. > > * I don't understand why you cannot issue more than one async read at > > a time nor async write at a time. In something like pipelined HTTP > > that seems like a very common pattern. I smell a potential design > > flaw, and while the docs mention a "queue_socket" I can't see such a > > thing in the reference section. > > > > Too much synchronization and buffering. > > The pipeline has the following requests: > > A => B => C > > If I were to allow replies to request B, the design would be more complex, > increasing the difficult to implement alternative backends and everyone's > life. Also, the reply to request B cannot actually be sent while the reply > to request A is on hold. This means that the whole response to request B > would need buffering and if response B is a video live stream, this means > an implicit (the user cannot know) crash after an unfortunate memory > exhaustion. I couldn't claim a **lightweight** server if this excessive > buffering was to occur. I couldn't claim the project is appropriate for > embedded devices if I had chosen the buffering approach. It's a trade-off. > I should write about it in the "design choice" of the documentation. > > HTTP/2.0 does have proper multiplexing and it can be more useful to respond > several requests without "head of line blocking". Ok, let's accept that HTTP before 2.0 can't multiplex (not my experience personally, but I agree it's got big gotchas before 2.0). You need a user facing API for Http which lets user code multiplex where that is available, and not multiplex when that is not available. In other words, identical code written against Http must automatically be optimal in either connection case. You're probably going to ask how I might implement that right? I think my first guess is you need two completion handler layers and therefore two completion dispatcher reactors: the bottom layer is the same as ASIO's and uses the ASIO reactor, whilst the top layer which users of Http sees is a reactor operated by Http and is disconnected, though operated by, the ASIO reactor. > A queue socket is a concept, I don't provide one. I can look into how make > the documentation less confusing by adding a page exclusively dedicated to > explain the problem it solves. Would that work for you? It could be this queue socket concept of yours is exactly what I just proposed as two completion handler and reactor layers. In either case, I don't think queue sockets should be a concept, I think they need to be the core of your library's user facing API. If power users wish to bypass that user facing API layer for less hand holding and less latency, that should also be possible. But I think HTTP multiplexing should be assumed to be the default starting point for Http applications. Once HTTP 2.0 is more common, it will seem madness that Http's user facing API is _not_ 100% multiplexing. > * Your tutorial examples use "using namespace std;" at global level. > > That's a showstopper of bad practice, and you need to purge all of > > those. Same for "using namespace boost;" at global level. > > > > The tutorials should be as small as possible to not scary the users. > > But I can put a comment "you should avoid using namespace". Does it help? using namespace is fine at local scope. It's not fine at global scope. Replacing all global using namespace with local using namespace is all you need to do. > * Your reference section is one very long thin list. You might want > > to use more horizontal space. > > > > I'd be happy already if I knew hot to remove the TOC from the reference > page. It'd be much more clean already. Personally in AFIO I hack the CSS to make the list into columns. ASIO employs a table. There are many solutions. > * I personally found the design rationale not useful. I specifically > > want to know: > > > > 1. Why did Http choose ASIO as my base over leading competitor X > > and Y? > > 2. What sacrifices occurred due to that choice? i.e. things I'd > > love weren't the case. > > > > I assume I don't need to reply these questions during the review, given the > reviewers may already be experienced on the answers. Well ... I'm not sure that's actually the case. It's not we don't have opinions on the answers, it's whether our answers are different to your answers. You'll see this during the AFIO review later this month - I have some really weird opinions no one else shares like choosing my own custom lightweight monadic future-promise implementation over async_result. That will be the source of much criticism I'm sure. > 3. What alternatives may be better suited for the user examining > > Http for suitability and why? i.e. make your documentation useful to > > people with a problem to solve, not just a howto manual. > > > > I don't quite understand this point. What I'm really asking is for details of when before starting Http you reviewed the options available and the designs they used, and exactly why you chose the design choices in Http you did with reference and comparision to prior art. Like a literature review, but for programming libraries. Does this make sense? It's partially for us to help us understand your choices, but also to help those examining Http to see if it's useful to them to figure out the best solution to their problem (which may be to use an alternative to Http). Documentation which is useful over documentation which is reference. > * Benchmarks, especially latency ones which are important for many > > uses of HTTP are missing. I'd particularly like to know confidence > > intervals on the statistical distribution so I know worst case > > outcomes. > > > > These values will change when I replace the parser used now. Also, they can > change based on the buffer sizes and parser options, which the user is/will > be able to change. > > Nevertheless, I do want to do some serious benchmarking, specially because > it'll give me a reference point when I write my own parser and want to > compare performance speedup. I can do some poor benchmark this weekend and > show the results. That would be interesting, but don't go crazy on it. Also, please do enable the undefined behaviour sanitiser permanently including in release mode and with full stack smashing protection before running benchmarks. Anything parsing malicious input should have all those turned on where available. > I hope all the above isn't too disheartening Vinícius. > > Not at all. I think you're the second person to dedicate more time > reviewing my work and this isn't disheartening at all. Just that your > feedback is usually around tooling and not the design itself (everybody has > its preferences). That's because I like the design a lot and have little bad to say about it. I've also been subscribed to your github since the beginning, and I have been reviewing your code on and off as you wrote it. You're a talented programmer. > > Jamfile for the main library build, > > Consider this done. I won't integrate into Boost without removing CMake > first. This was already partially stated in the Bjorn's announcement (but > somehow implicit, to be fair). I have no problem at all with dual build systems if you're happy keeping both maintained. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

3 6

Re: [boost] [asio-users] [http] Formal review of Boost.Http
by Niall Douglas 15 Aug '15

15 Aug '15

On 13 Aug 2015 at 8:58, Vinícius dos Santos Oliveira wrote: > First of all, sorry to all members of the list about my unavailability. I > was planing to write an experimental HTTP 2.0 backend, so I could give more > confidence about how much I believe this Boost.Http core I present for > review is the right abstraction. > > Anyway, looks like I took the wrong approach. I should have answered the > easiest questions first and implement the HTTP 2.0 backend (experimental, > using existing-library and not Boost-quality) later. That's a wise move Vinícius I think. > > Where I'm really at is I think if Http is accepted you're going to > > either have to ditch it and reimplement atop the Networking TS as > > Chris folds the substantial changes WG21 will force onto ASIO into > > Boost.ASIO, or end up refactoring Http to cleave more closely to the > > Networking TS anyway. > > I'm okay with that. You may be, but I would worry about your users. Boost users expect stability and permanence. > > I think for the majority of HTTP users they just want it to work > > without surprises to a high default performance level. If you look at > > the history of the HTTP library support in Python you'll see what I > > mean - firstly, it's surprisingly easy to get a HTTP library API > > design wrong, even in a v2 refactor. And secondly that people need > > both a stupid-simple API and a more bare metal API *simultaneously* > > with HTTP, and therein lies the design gotcha. > > > > If you want a high level API, you're going to use coroutines. There is > nothing in the wild so readable as coroutines. Coroutines are **the** > solution to solve spaghetti code in asynchronous abstractions. Lambdas and > futures will never be as readable as coroutines. Anyway... God no. I would see a top level API as being synchronous, so something like python's urllib3: import requests r = requests.get('http://www.boost.org') print(r.json()) It may coroutinise but still appear to be synchronous, but that's a separate moving ball and not one Http can tackle alone. > The more I think, the more I believe how much such "middle-level" API is > underestimated. > [snip] > It's not really like Boost.Http is too low level. It's more like only the > less polemic building blocks are available now. You'll see the same main > players in a higher-level abstraction and the main difference is most > likely that you won't manage the main players yourself. Think of it as a Boost user. The average Boost user is going to see Boost.Http and think to themselves, "oh goody I can now write: import requests r = requests.get('http://www.boost.org') print(r.json()) ... in C++ and yay I've just chopped 10,000 lines of buggy hack code from my codebase." And that's exactly what I would expect too from a library called Boost.Http. What you're providing isn't that though: it's a set of low level HTTP related extensions to Boost.ASIO. From my review of your code and docs, and what you've told me, I think the ideal location for your library is within ASIO under the asio::http namespace. However, assuming Chris isn't keen to allow that as a git submodule within ASIO (he might be, I don't know), I think as a very minimum you need a name for your library which says it is a set of low level HTTP related primitives for Boost.ASIO. That name is not Boost.Http for sure. A later library you might submit could be called Boost.Http, but it would provide a daft easy API like the above for Python's urllib3. In fact, if you can achieve a more appropriate name choice, I'll change my vote from rejection to a conditional acceptance as you've done a great job in persuading me from where I was. You've impressed with me your awareness of the tradeoffs involved in a moving API environment, and though I severely personally disagree with your choice of Boost.ASIO over ASIO, I can also see that targeting a more stable API in the form of Boost.ASIO for now makes sense. How does that sound? Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

2 1

Making the Windows binaries 'official'
by Tom Kent 15 Aug '15

15 Aug '15

<history> After the boostpro.com organization dissolved, there was never an official place to go to get pre-built binaries for the releases. Some random components were made available on the sourceforge download page and I had been hosting them on my own website (http://boost.teeks99.com) since I started building them around the release of 1.37. A couple years ago, before the release of 1.54, I asked the community if they would like my builds to be hosted on the sourceforge site, and the answer was positive: http://lists.boost.org/Archives/boost/2013/04/202460.php Since the 1.54 release, I've generally been uploading the windows binaries based on the RCs and enabling the downloads just a few minutes after the release notice goes out. </history> I would now like to propose that we make these libraries an official part of each Boost release. I don't think this entails much, just the following: 1) Have the release team coordinate with me to ensure that the binaries are ready before announcing the release. 2) Include a link to them in the release announcement. 3) Link to them from the website. It would also be good to get others who can run the scripts to perform the build, if I am someday unable to do this. The biggest hurdle here is having a (virtual) machine with all six versions of visual studio on it. Is this something the community would be in favor of? I know that many of the people on these lists don't use windows and if they do are generally capable of building the windows binaries themselves, however there is a large group out there that prefers to just get the installer for them (like they do for other projects like Qt, wxWidgets, etc). Thoughts? Tom Kent

2 2

[job] exec job-graphs in a NUMA env (was a thread pool that allows recursive calls)
by Oliver Kowalke 15 Aug '15

15 Aug '15

This library is related to the posting 'a thread pool that allows recursive calls' (http://lists.boost.org/Archives/boost/2015/06/223426.php) e.g. the library solves the many-dependent-jobs problem. boost.job is a derivative work of boost.task (which was published in 2008, the code is refactored and now split into boost.context, boost.coroutine2/boost.fiber and boost.job). boost.job provides a framework for asynchronous execution of job-graphs (a job is a small unit of code that can be executed independently and parallel, e.g. it's a fain-grained work item) of arbitrary dependency, e.g. solving the many-dependent-jobs problem (M: worker threads, N: jobs waiting for other jobs; M << N). Each (logical) processor of a dedicated processor set (configurable) gets a worker-thread assigned. Each worker-thread runs a fiber-pool (launch policy is customizable). A submitted job is always executed by a worker-fiber. While using boost.fiber jobs are scheduled cooperatively so that a job can yield to other jobs during its execution. Jobs can be synchronized via primitives like mutex's, conditions, barriers and channels (message exchange) from boost.fiber without blocking the worker-thread. Fibers provide a fast context switch. boost.job supports to inspect the NUMA topology (e.g. which logical processors are online, and which processors share L1/L2/L3 cache). This information can be used to select a appropriate processor for a job. For a fast access to job related data, boost.job provides functions and classes to allocate memory on NUMA nodes (processors of a NUMA node share cache and have a faster access to its associated memory bank than processors of other nodes.). source: https://github.com/olk/boost-job documentation: http://olk.github.io/libs/job/doc/html/

2 2

Re: [boost] Build Error VS14 + Intel 15.0 (Update 4, the latest)
by degski 15 Aug '15

15 Aug '15

Changing intel-win.jam, line 476 (boost 1.59.0 rc2) from: .iclvars-15.0-supported-vcs = "12.0 11.0 10.0 9.0" ; to: .iclvars-15.0-supported-vcs = "14.0 12.0 11.0 10.0 9.0" ; solves the problem.

2 1

Master branch is open for changes
by Marshall Clow 14 Aug '15

14 Aug '15

The release has been tagged, and library maintainers are free to merge to release again. Thanks to everyone for their cooperation. -- Marshall

1 0

Boost 1.59.0 has been released
by Marshall Clow 14 Aug '15

14 Aug '15

Release 1.59.0 of the Boost C++ Libraries is now available. These open-source libraries work well with the C++ Standard Library, and are usable across a broad spectrum of applications. The Boost license encourages both commercial and non-commercial use. This release contains two new libraries and numerous enhancements and bug fixes for existing libraries. New Libraries * Convert: An extendible and configurable type-conversion framework, from Vladimir Batov. * Coroutine2: from Oliver Kowalke For details, including download links, see http://www.boost.org/users/news/version_1.59.0 You can also download directly from SourceForge: http://sourceforge.net/projects/boost/files/boost/1.59.0/ To install this release on your system, see http://www.boost.org/doc/libs/release/more/getting_started/index.html Thanks, --The Boost release team Vladimir Prus Rene Rivera Marshall Clow Eric Niebler Daniel James Beman Dawes

5 4

[filesystem] proposal: treat reparse files as regular files
by Paul Harris 14 Aug '15

14 Aug '15

Hi all, -- Proposal -- tl;dr : I propose that we treat all non-symlink "reparse_files" as "regular_files". If the boost library user wants to do something special with these plain reparse files, they should use alternative means. But typically they are supposed to be treated as regular files. This means we could drop the "reparse_file" enum, or continue to use it for a special-case whats_my_real_status() function. --- Motivation --- Windows Server 2012 uses reparse points to implement deduplification. Those files should be treated as regular files in all circumstances. Currently, they are not classed as "regular" files, so fs::copy() will skip those files, and library-user code written to list files based on official examples will ignore all dedup'd files. This is causing serious and latent problems at the user end, because deduping only happens occasionally after X days, and users cannot easily check if a file is dedup'd (they look just like regular files). --- Real life example --- Another example of reparse use is the "Symantec Enterprise Vault" (version 10), which I found running on one site. It replaces files on the server with reparse-point files. FSUTIL REPARSEPOINT QUERY filename.txt shows the contents of the reparse buffer, which is a URL to an internal HTTP server. The url points to a .asp link with a bunch of codes and dates to identify the file in the server. Copy-pasting that URL into a webbrowser allows you to directly download the file via the webbrowser, which is pretty neat I suppose. In this case, the reparsed-files in Windows Explorer all have grey X crosses on their file icon. If you "type" them (via cmd) or open them, the icon loses the grey cross and the file is no longer a reparse point file. My software refused to read the files because they were "not regular files". Once I adjusted the boost code (described below), my software saw them as regular and opened the files. The file icons lost the grey cross. SO it seems that the file server automatically downloads and replaces the files with the stored content on demand, and the file reading client program should really just treat these files as normal files. --- Short logic --- reparse files (that are not symlinks) should almost always be treated as plain files. They are a mechanism for MS file servers to store files in clever ways, but the client should not care and just read/write them as if they were normal files. This is different to all the other "other" files which can't be treated like normal files: block, character, fifo, socket, unknown So, reparse files should not be grouped with the "other" file types. They are also NOT symlinks, and should not be treated as symlinks (which would require special decisions for copying, or querying the status, or checking if the target still exists). --- What are reparse files --- I did some reading, if I understand correctly: Reparse points give drivers (on the server) a chance to get data through some other specialised means (eg query from a cluster store). They are processed by the server, not the client, so clients should treat reparse data as opaque data. EXCEPT for symlink reparse files. https://msdn.microsoft.com/en-us/library/dd541667.aspx quote:"The following reparse tags, with the exception of IO_REPARSE_TAG_SYMLINK, are processed on the server and are not processed by a client after transmission over the wire. Clients should treat associated reparse data as opaque data." It seems like the rest of the tags are used for connecting files to other types of storage (eg long term storage, cluster storage). Clients may need to do something special with SOME reparse point files, IF the client cares about how long the file read may take. https://msdn.microsoft.com/en-us/library/windows/desktop/aa365505(v=vs.85).… quote: "Most applications should take special actions for files that have been moved to long-term storage, if only to notify the user that it may take a while to retrieve the file." --- Changes required --- Option 1: change is_regular_file() to return true where type==reparse_file I don't like this option, as library-users could be checking the type directly instead of using is_regular_file(). Option 2: These functions return reparse_file: fs::file_type query_file_type(const path& p, error_code* ec) file_status status(const path& p, error_code* ec) file_status symlink_status(const path& p, error_code* ec) They should instead return regular_file instead. --- How to test with dedup files --- Creating dedup'd files is a feature only available on Windows Server 2012, I believe, although Windows XP/Vista/7/8/10 clients all can read dedup files. Here is how I created a windows server to test with (for free!) on a demo Azure cloud server. I have one working, so if anyone would like to use it for their testing, let me know. Step one: follow this blog article: http://blogs.technet.com/b/tommypatterson/p/azureservertrial.aspx once the machine was "running" I clicked Connect at the bottom. That gave me an .rdp file which in theory I could use with rdesktop, but it uses a DNS name that was only just created, so that didn't work. When you click the name of the server in the list, it shows the public IP on the right.. and the port then you can do this $ rdesktop that.ip.addr:port But only if you have the latest rdesktop AND you have set up kerberos something-something. Instead I found a windows computer and used remote desktop from there. --- Once inside, in the "Server Manager --> Dashboard" window on the screen, click "Add Roles" then go next next until "Server Roles" expand "File and Storage services" , "File and iSCSI" , and tick "Data Deduplication" Then next next etc and Install. Wait a bit... and its done. http://www.techrepublic.com/blog/data-center/configuring-windows-server-8-d… --- Continuing on that webpage... Time to enable dedup. There is a temp disk D: so lets enable there. Method 1... I did this and then went to method 2... Start PowerShell, type: "Enable-DedupVolume D:" Method 2... in that same Dashboard, hit the 4th button (File and Storage Services) Then Volumes --> Disks click Volume 1 at the top, and then right click D: at the bottom --> Configure Dedup. To try and accelerate this puppy, I set the "age to dedup" to 0 days. http://www.techrepublic.com/blog/data-center/windows-server-2012-deduplicat… --- Time to make something to dedup. We'll just duplicate the warning.txt file that exists on D: In powershell: PS> D: PS> $file = Get-Content DATALOSS_WARNING_README.txt Then, do these 2 commands a bunch of times until "big.txt" gets to say 6MB PS> Add-Content big.txt $file PS> $file = Get-Content big.txt Then use windows explorer (or other) to make a dozen copies of big.txt Copy c:\windows\explorer.exe to D: to give it something to dedup Go to D: and then copy-paste explorer.exe a dozen times. In PowerShell, type: PS> Update-DedupStatus -Volume D: PS> Start-DedupStatus -Type Optimization -Volume D: and then wait for it to finish. you can track its progress with: PS> Get-DedupJob PS> Get-DedupStatus -Volume D: --- So, once its deduped, you check. PS> FSUTIL REPARSEPOINT QUERY big.txt you should see that its a reparse point with that 0x800etc0013 code. Copy-paste big.txt to big2.txt and check it with the query, and it should tell you big2 is NOT a reparse point. NOW you have some files to test the boost library... You can't zip them up (they lose the dedup tag), you have to run boost binaries ON the computer in the sky. --- Finish --- Thanks for reading, Paul

5 30

[COre] Pull request for enable_if_type
by Joel FALCOU 13 Aug '15

13 Aug '15

Hi, I just refurbished and pushed an old patch adding the enable_if_type SFINAE utility to core. https://github.com/boostorg/core/pull/15 I tried to keep it simple and make the doc proper. This utility has been laying around various boost library and was "reinvetned" in bosot.SIMD. I think it's worth being in core so code duplication is kept to a minimum. Comments welcome

2 3

[spirit] 1.59.0 rc3: compiler warning in real_impl.hpp
by Tobias Loew 13 Aug '15

13 Aug '15

Hi, with MSVC 11, 32-bit build I get a compiler warning in boost\spirit\home\qi\numeric\detail\real_impl.hpp : 287 warning C4244: '=' : conversion from 'unsigned __int64' to 'double', possible loss of data since I usually comile with warnings-as-errors this broke my build. Tobias -- View this message in context: http://boost.2283326.n4.nabble.com/spirit-1-59-0-rc3-compiler-warning-in-re… Sent from the Boost - Dev mailing list archive at Nabble.com.

2 3

Any interest in a M:N green threading library?
by Xu Chen 13 Aug '15

13 Aug '15

Just cleanup some codes I wrote before, there is a M:N green threading library built on top of Boost.Thread and Boost.Coroutine, conceptually it works like goroutines in Go and (once existed) green threads in Rust. With some refinements, it now supports all C++11/14 threading standard, along with some extensions exist in Boost.Thread such as thread_group and chainable futures. It can be integrated with Boost.ASIO, all ASIO async functions can be used as sync ones in green threads, similar to Boost.ASIO/Boost.Coroutine integration. Also it has some facilities to communicate with foreign threads, can by used to integrate with external libs such as database drivers. The code can be found at https://github.com/windoze/boost.green_thread I don't know much about boostbook tools so the documents are kind of crippled, but I'll improve it if needed. Regards, Chen Xu

4 5

Re: [boost] [build] Problems with VC14 and MSPDB140.DLL (was: [testing][teeks] MSVC-14 errors about MSPDB140.DLL)
by Adam Merz 12 Aug '15

12 Aug '15

Andrey Semashev <andrey.semashev <at> gmail.com> writes: > > I created a Connect ticket about it: > > https://connect.microsoft.com/VisualStudio/feedback/details/1651822/incorr ect-mspdb140-dll-version-picked-in-x86-x64-cross-tools-environment > This ticket appears to be set to private; would you mind editing it to public visibility so that we may upvote it? Regards,

5 5

1.59.0 rc3 is now available
by Marshall Clow 12 Aug '15

12 Aug '15

New Release candidate files for 1.59.0 are available at: http://boost.cowic.de/rc/ RC3 should fix the problem with VC14 and MSPDB140.DLL. As always, the release managers would appreciate it if you download the candidate of your choice and give building it a try. Please report both success and failure, and anything else that is noteworthy. This helps ensure the candidates build OK before we push them out to SourceForge. The files (and associated md5s) are: MD5 (boost_1_59_0_rc3.7z) = 0a2e512844f3e30a6240f8139ee983f3 MD5 (boost_1_59_0_rc3.tar.bz2) = 6aa9a5c6a4ca1016edd0ed1178e3cb87 MD5 (boost_1_59_0_rc3.tar.gz) = 51528a0e3b33d9e10aaa311d9eb451e3 MD5 (boost_1_59_0_rc3.zip) = 08d29a2d85db3ebc8c6fdfa3a1f2b83c Thanks! -- The release managers P.S. Release notes are here: http://www.boost.org/users/history/version_1_59_0.html

3 3

[context] msvc /SAFESEH
by Jeff Flinn 12 Aug '15

12 Aug '15

I need to build my application with /SAFESEH which uses boost coroutine which uses context. I'm getting the following link errors: 1>libboost_context-vc100-mt-gd-1_53.lib(jump_i386_ms_pe_masm.obj) : error LNK2026: module unsafe for SAFESEH image. 1>libboost_context-vc100-mt-gd-1_53.lib(make_i386_ms_pe_masm.obj) : error LNK2026: module unsafe for SAFESEH image. I've built boost static libs with the same MSVC vesion and options (sans /SAFESEH as it's not a compile/lib option) Compiler/linker versions: Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86 Microsoft (R) Incremental Linker Version 10.00.40219.01 Has anyone run into this and resolved the error? Any insight is appreciated. Thanks, Jeff

3 9

[build] Overriding warning level for clang
by Edward Diener 11 Aug '15

11 Aug '15

In my user-config.jam for clang I have added: <warnings>on to the configuration section of the 'using clang' toolset. Nonetheless I still see '-Wall' on the command line when I compile. My other <cxxflags> settings which are part of the toolset are reflected by the command line. Even if I change to <warnings>off in the toolset I still see '-Wall' on the command line. Why is this not working and is there any way I can change the warning level for clang so that it is not 'Wall' on the command line ? The clang compiler is wonderful but with 'Wall' it spews out a boatload of information I normally do not need.

1 0

[build] Problems with VC14 and MSPDB140.DLL (was: [testing][teeks] MSVC-14 errors about MSPDB140.DLL)
by Andrey Semashev 11 Aug '15

11 Aug '15

On Sat, May 23, 2015 at 4:40 PM, Hartmut Kaiser <hartmut.kaiser(a)gmail.com> wrote: > >> All tests for Boost.Log (and I suspect any other built libraries) on MSVC- >> 14 >> are failing because of this error: >> >> LINK : fatal error LNK1101: incorrect MSPDB140.DLL version; recheck >> installation of this product > > My workaround was to edit the 'Path' env variable in one of the temp files > generated by bjam (b2_msvc_14.0_vcvarsall_amd64.cmd in your temp directory) > not to point to the x64 cross compiler, but to the native one. For me this > involved changing > > C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64_x86 > > To > > C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64 I encountered the same problem with VC14 RTM on my VM. This was a clean install of VC14 RTM, there were no CTPs or RCs installed on this machine. I removed the b2_msvc_14.0_vcvarsall*.cmd scripts and ran b2 with address-model=64 to build my tests so that these scripts are re-created from scratch, so it looks like there is a problem with how these scripts are generated by b2. I found 3 instances of mspdb140.dll on my system in these locations: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64 The last two are for 32 and 64-bit native compilers. The first one, I assume, is for use by IDE. The problem is that the first path is listed in b2_msvc_14.0_vcvarsall_x86_amd64.cmd in the "SET Path=..." line prior to the compiler-specific paths. Removing it from the Path solved the problem. Could Boost.Build be adjusted so that the cmd files are generated correctly in the first place?

9 25

[range] index adaptor with size_t as index_type
by Tobias Loew 10 Aug '15

10 Aug '15

>From version 1.55 to 1.56 the index_type of the index of the range index-adaptor was changed from size_t to ptr_diff which e.g. allows for negativ start values. But this has the side-effect that comparison with other size_t-values (e.g. a call to vector's size()) results in a compiler warning for comparing signed and unsigned values (on MSVC 11: warning C4018: '<' : signed/unsigned mismatch). I'm proposing a specialized version of the index adaptor, where the index_type is size_t and the start index is 0 (zero) and explicit specification of the start-value is not possible. A possible name could be "uindex". comments / critics / pros / cons are welcome Tobias -- View this message in context: http://boost.2283326.n4.nabble.com/range-index-adaptor-with-size-t-as-index… Sent from the Boost - Dev mailing list archive at Nabble.com.

1 0

[python] Help with boost python build
by Andy Thomason 10 Aug '15

10 Aug '15

Hi All, I have had highly variable results with Boost.Python mostly due to linker errors or failures of Python to find DLLs. Python is not great at reporting why it fails, for example not naming failures to load dependent DLLs on Windows which does not make this a joyous experience. I have never managed to get any Boost.Python example - mine or otherwise - working on Windows and have had limited success on Linux. It is little surprise that all the examples on Stackoverflow avoid Bjam for builds. For my genetics library, I have an example that has a single source file and the Jamfile reads thus: python-extension genetics : bindings.cpp : <toolset>gcc:<cxxflags>-std=c++11 ; with: using python : 2.7 ; in ~/boost-config.jam ie. as minimal as possible. This builds well on linux (I am adding a library to the head of modular boost). When I run: python -c "import genetics" with appropriate PYTHONPATH I get a run-time symbol search error: ImportError: <...>/genetics.so: undefined symbol: _ZNK5boost6python7objects21py_function_impl_base9max_arityEv My guess would be that rather than this one symbol, Python is failing to load the shared lib: ~/modular-boost/bin.v2/libs/python/build/gcc-4.8/debug/libboost_python.so.1.59.0 As a postscript, It would be lovely if there were no static or dynamic libraries to link with when compiling with Boost.Python and a unity build would be a nice option as this would bypass linker and runtime related errors. It would probably compile faster in my long experience with such projects. Any help would be welcome. Andy. --- This email has been checked for viruses by Avast antivirus software. http://www.avast.com

1 0

[regression tests] Adding regression tests for mingw(-64)/gcc and clang on Windows
by Edward Diener 10 Aug '15

10 Aug '15

I don't know if I am supposed to ask questions about running regression tests in this mailing list or in the Boost testing mailing list, but I will try here first. The Boost testing mailing list seems to be for actual test result problems and the like. For a long time now we have had no regression tests for mingw/gcc or clang targeting gcc on Windows. The only regression tests for Windows are for various releases of VC++. I have limited resources but I have a laptop usually sitting idle, except for occasional travel where I take it with me, and I am set up to run tests on that machine for mingw/gcc and clang on Windows. However mingw/gcc and clang have limitations when running tests in that, both relying on various mingw/gcc implementations, the path to the implementation must be added at the beginning of the PATH before running any of the tests. I have already argued with the mingw-64 people about this necessity but they are adamant that this should be the case even just to compile/link properly ( and have setup their directory structure to enforce it ), and I don't want to waste my breath further with them. Currently I have a batch file that does that for me when running tests for aany particular library, but I don't know how that can be achieved when running the regression tests unless possibly I run regression tests for only a single toolset at a time, and manipulate the PATH accodingly. But that is really time consuming since evidently all of Boost must be downloaded to the run directory each time. Is there an easy solution to this or do I have to look at run.py and try modify it accordingly ( I know Python well ) ? Also I notice on the Boost web site regression test page ( http://www.boost.org/development/running_regression_tests.html ) that the options of the regression tests are somewhat documented but none of the commands are documented. Is there further documentation about the regression test commands anywhere ?

1 0

Build Error VS14 + Intel 15.0 (Update 4, the latest)
by degski 10 Aug '15

10 Aug '15

I have encountered the following error when building boost 1.58 (and 1.59-b1): C:/boost_1_58_0/tools/build/src/tools\intel-win.jam:194: in configure-really from module intel-win error: msvc 14 not supported for intel toolset version 15.0 I'm using the following line to build boost: bjam -a -d+2 --toolset=intel-15.0-vc14 "-sINTEL_PATH=C:\Program Files (x86)\Intel\Composer XE 2015\bin\intel64" cxxflags="/O2 /Quse-intel-optimized-headers /Qstd=c++14 /Qipo /Zc:forScope /Oi /MT /EHsc /Qparallel" "-sINTEL_BASE_MSVC_TOOLSET=vc-14_0" "-sINTEL_VERSION=15" "-sZLIB_SOURCE=%HOME%\zlib-1.2.8" "-sBZIP2_SOURCE=%HOME%\bzip2-1.0.6" variant=release instruction-set=nehalem threading=multi link=static runtime-link=static --stagedir=stage64_static address-model=64 architecture=x86 --without-python --without-mpi --disable-filesystem2 stage > icc_log_stage.log The above worked fine with VS11 and Intel 15.0. The combo (VS14/ICC15) is obviously supported by (MS/Intel), and works fine in the IDE (Community Edition). I confirm the problem with bootstrap.bat, this has been around for a long time, and always had to build the 32bit version first. I ran into it as I have decided to stop building 32-bit. The obvious work-around is to build bjam seperately, not a killer for me, but might be for some. The main issue I report, is though and it would be a shame if the 1.59 release would not allow for that combo, as finally MS have released a free, non-crippled, release of their IDE with (finally, in 2015) full support for C++11... I'm happy to test some things out if necessary, I'm not capable of adressing the issue though...

1 0

Re: [boost] [asio-users] [http] Formal review of Boost.Http
by Lee Clagett 09 Aug '15

09 Aug '15

On Sat, Aug 8, 2015 at 5:16 AM, Vinícius dos Santos Oliveira < vinipsmaker(a)users.sf.net> wrote: > 2015-08-08 0:10 GMT-03:00 Lee Clagett <forum(a)leeclagett.com>: > >> FWIW it should be possible to do client and server side pipelining with >> this design right now. The reads and writes of the HTTP Socket concept >> should be (and currently are by `basic_socket`) handled independently. >> Since the notification for a write completion is based on the lower layer >> write completion, and NOT whether a HTTP response was received, multiple >> (pipelined) requests should be possible by a client. Similarly, this design >> should allow servers to handle pipelined requests in parrallel. Providing a >> framework for client or server pipelining will require more work, and >> should probably not be a part of the Socket concept. In fact - >> > > The current design **does** support pipelining. Some HTTP clients disable > pipelining because some buggy servers will get confuse. This won't happen > with Boost.Http (there are tests to ensure pipelning is working). > > What the current design does **not** support is handling pipelined > requests concurrently. When you starting read a request, the write_state[1] > will change to finished until the fully request is received, so you can't > get multiple request while you don't issue the replies. This behaviour is > explained in the ServerSocket concept[2]. > > There are just too many ways to allow HTTP concurrent pipeline that would > be transparent for the user, but all of them are heavier. If I was to allow > handling multiple concurrent HTTP pipelined requests, I'd expose user > cooperation, so the abstraction doesn't become so heavy. The user would > need to be aware if the reply for some request can already be delivered or > not. Of course this means the socket will need to remember order and attach > some id to the request messages. It can be problematic in alternative > backends because the id could be of a different type and makes it difficult > to use the same message type with different HTTP backends. What do you > think? The design would just be more complex for not much gain. HTTP/2.0 > allow real multiplexing and doesn't show this problem. > I agree that providing any pipelining framework in the library is probably unnecessary right now. However, the design should not inhibit such functionality from being added (which I do not think is a problem with this design). > I think the ServerSocket concept should be removed entirely. The >> `http::basic_socket<...>` models both concepts, so theres lots of server >> specific code in it, which seems confusing to me (is this really a "basic" >> socket?). >> > > basic_socket uses basic_ prefix just like basic_string. > I was asking whether basic_socket should NOT model the http::ServerSocket concept. I understood the naming convention, and the decision to provide the implementation as a configurable template. The problem is the lack of composability. If a different http::Socket concept is desired (someone not using ASIO, etc.), then the http::ServerSocket concept has to be re-implemented also since there is no other implementation. The obvious way is to achieve composability is to use inheritance with: `http::basic_server_socket<Socket>` where `Socket` is a http::Socket concept. Inheritance in this situation has its drawbacks, for sure (a virtual destructor should be considered). > I think standalone functions for typical client and server operations can >> be implemented as "composed" operations similar to `asio::async_write`. The >> documentation will have to be explicit on what the composed function is >> doing so that users know whether the Socket concept has outstanding reads >> or writes (and therefore cannot do certain operations until the handler is >> invoked). In fact, if there is a server operation that _cannot_ be done >> with the Socket concept, then the concept probably needs to be re-designed. >> > > About "users know whether..." seems like a bad idea if I want to deliver > multiple backends. Some implementation details should just be abstracted > away. > > http::SocketServer implementations manipulate the state of the http::Socket (and indirectly the asio::tcp::socket), making some actions unavailable after invoking those functions. For example, after invoking `async_write_response`, no writes on the http::Socket/http::ServerSocket or asio::ip::tcp::socket can occur until the callback is invoked because it calls asio::write on the asio::ip::tcp::socket directly. So unless I am incorrect, it is important for the users to know whats being manipulated (the effects). I don't understand what you mean by "if there is a server operation that > cannot be done with the Socket concept [...]". Boost.Http has two concepts, > ServerSocket and Socket. Socket concept will be useful to client-side HTTP > and that's the reason why there are two concepts. > > The http::ServerSocket concept requires a http::Socket to write its messages, but does the http::ServerSocket concept need to be a refinement of the http::Socket concept? This implies a strong relationship. The current specification looks like a http::ServerTraits concept - its specifying how a http::Socket is being used in situations specific to servers. There doesn't appear to be any additional state tracking needed for the http::ServerSocket functions, which is why I suggested standalone functions (I didn't see the FileServer section). In fact implementation can be inverted; make all of the current implementations for http::ServerSocket standalone functions, and then have the default http::ServerTraits implementation call the standalone versions of the functions. The http::ServerTraits should then take a http::Socket as an argument for each of the functions. This complete de-coupling would allow someone to independently change the behavior of any related server functions, even while using the same http::Socket object. [1] https://boostgsoc14.github.io/boost.http/reference/write_state.html > [2] "The ServerSocket object MUST prevent the user from issuing new > replies while [...]", > https://boostgsoc14.github.io/boost.http/reference/server_socket_concept.ht… > > Lee

2 3

[Utility] intuitive scoping with BOOST_WITH
by Marcel Ebmer 08 Aug '15

08 Aug '15

Hi Everyone, What if you could do std::mutex the_mutex; using Lock = std::unique_lock<std::mutex>; void f() { with(Lock{the_mutex}) { do_something(); do_other_thing(); } } *...or...* struct Pushed_matrix { Pushed_matrix() { glPushMatrix(); } ~Pushed_matrix() { glPopMatrix(); } }; void f() { with(Pushed_matrix{}) draw_something(); } At C++Now (2015), I presented this idea of mine and a macro implementation (BOOST_WITH) in a lightning talk. Since reception was good and more serious than the talk itself, I continued to work on it. I would appreciate informal review and opinions: https://github.com/maysl/BOOST_WITH Marcel

3 3

1.59.0 rc2 is now available
by Marshall Clow 08 Aug '15

08 Aug '15

Release candidate files for 1.59.0 are available at: http://boost.cowic.de/rc/ (rc1 failed a smoke test) As always, the release managers would appreciate it if you download the candidate of your choice and give building it a try. Please report both success and failure, and anything else that is noteworthy. This helps ensure the candidates build OK before we push them out to SourceForge. The files (and associated md5s) are: MD5 (boost_1_59_0_rc2.7z) = 1854999b8d32ba89acbf4384fe55e0ab MD5 (boost_1_59_0_rc2.tar.bz2) = 1783f07dd4736ffd9fc97874ce4ef4bf MD5 (boost_1_59_0_rc2.tar.gz) = 47e6db09fd7e1265b9b7046f654e9a72 MD5 (boost_1_59_0_rc2.zip) = 9061c5bbc3399f3f220036e896c202aa Thanks! -- The release managers P.S. Release notes are here: http://www.boost.org/users/history/version_1_59_0.html

3 2

[1.59] Still unmerged changes in the super-project
by John Maddock 08 Aug '15

08 Aug '15

There are a number of unmerged changes in develop of the super-project that look like they should really be in release, in particular: * Jamroot, contains amongst other things a very old fix for Oracle C++ support, without which building with Oracle is more or less broken. In fact unmerged commits on this file go back to....wait for it... 2011 ! * bootstrap.bat has fixes for some rather embarrassing typos. * bootstrap.sh has lots of fixes, also going back to 2011. Those were the "highlights" that stood out to me, the full list with unmerged changes is: .gitmodules | 613 ++++++++++++++++++------------------ .travis.yml | 6 +- Jamroot | 32 +- boostcpp.jam | 47 +-- boostcpp.py | 459 --------------------------- bootstrap.bat | 4 +- bootstrap.sh | 24 +- doc/Jamfile.v2 | 2 - doc/src/boost.xml | 2 - libs/libraries.htm | 10 +- libs/maintainers.txt | 5 +- status/Jamfile.v2 | 4 - status/explicit-failures-markup.xml | 41 +-- tools/Jamfile.v2 | 2 +- 14 files changed, 336 insertions(+), 915 deletions Best, John.

3 2

[iostreams] Ticket #5237 - Chance to get fix applied into Boost 1.59?
by Deniz Bahadir 08 Aug '15

08 Aug '15

Hi everyone, in our application we use Boost.Iostreams with GZip functionality to write some data into gzipped output-files. Eventually we realized, that we would get corrupted output-files if no data was available that could be written to the output-file By looking for a patch we stumbled across an issue-report [1] which seems to be open for more than 4 years, although a simple fix was already provided in its comments 3,5 years ago. My question is: Can somebody apply that fix to the development branch and could it still make it into the upcoming Boost 1.59 release? Thanks, Deniz [1] https://svn.boost.org/trac/boost/ticket/5237

4 4