Re: Changing virtual memory amount in Appveyor and Travis CI tests

From: Edward Diener (eldiener_at_[hidden])
Date: 2020-09-21 04:20:40

On 9/20/2020 11:59 PM, Gavin Lambert via Boost wrote:
> On 21/09/2020 14:14, Edward Diener wrote:
>> Is it possible through .yml file for Appveyor and Travis CI to change 
>> the amount of virtual memory when running a CI test. I am running CI 
>> tests for Boost PP which heavily use the preprocessor, and some of the 
>> extended tests I am running fail on both Appveyor and Travis CIÂ  with 
>> out of memory conditions while on my own local computer, with about 32 
>> GB of ram, the tests succeed. I think in the real world most users now 
>> have at least 32 GB of ram or more, while these virtual environments 
>> in Appveyor and/or Travis CI probably run their tests with much less 
>> overall virtual memory than an average user would have.
> 
> According to their websites, Appveyor VMs have a limit of around 6 GB 
> and Travis VMs have a limit around 8 GB for Windows and 4 GB for 
> Linux/MacOS.  Neither appear to be configurable in any way (but I am not 
> an expert), which is understandable.
> 
> I realise you're probably deliberately testing edge cases with 
> pathologic memory usage, but normally there'd be something seriously 
> wrong if a build comes remotely close to using that much.

I agree, which makes me think that the compilers/preprocessors involved 
limit the amount of memory why they allow themselves to use. But these 
"out of memory" type of conditions are occurring in gcc, clang, and 
vc++, so all three are limiting preprocessor memory in some way. Oh 
well, I do warn the end-user of Boost PP about pushing some of these 
limitations, but it is disappointing that the preprocessors can only 
reflect dubious "out of memory" type conditions rather than error out on 
what situations are really causing the preprocessors to fail.

> 
> While yes, these days usually developers do have 32 GB RAM or more, the 
> "popular" RAM size is still 16 GB or even less, and a lot of this space 
> would be intended for parallel compiles or VMs and any single compile 
> should assume it must use a lot less. 

Even if 16 GB, these CIs limiting memory to 4 GB or 6 GB seem a 
disservice to me.