Re: [boost] [conversion] Isolating the phantom file changes problem

Subject: Re: [boost] [conversion] Isolating the phantom file changes problem
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2013-11-29 12:26:56

On 29 Nov 2013 at 8:14, Beman Dawes wrote:

> > When we talk about renormalisation, we're talking about the procedure
> > described in the gitattributes man page.

Unfortunately that procedure would corrupt many files within Boost.

> +1
> 
> There is also a nice discussion at
> https://help.github.com/articles/dealing-with-line-endings#re-normalizing-a-repository
> 
> I've forked the boost super repo and am testing the procedure now.

Off the top of my head, you'll need to watch for the following (this 
list is incomplete):

* Files with text file extensions not in ASCII or UTF-8. If you use 
simple EOL renormalisation with UTF-16 text for example, you'll 
corrupt that text.

* Text files with intentionally mixed EOLs. You'll need to change 
their extension to not .txt (best), or add special exceptions to 
.gitattributes (brittle, I wouldn't recommend this option).

* Scan the first 8Kb of every file with an extension not marked as 
text nor binary in .gitattributes for zeros. If you don't find a 
zero, git will assume it is text and EOL normalise it. Unfortunately 
some binary file types such as PDF don't have zeros in their first 
8Kb, so that would be very bad.

We never dealt with these issues during conversion, and there are 
probably more we don't know about yet. This is why I said Boost is 
not ready to do the transition - plus too few want to do the manual 
labour involved in achieving a "perfect" conversion and just want 
"someone else" to do the tedious work for them.

Niall

-- 
Currently unemployed and looking for work.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/