Re: [boost] [GSoC] [Boost.Hana] Formal review request

Subject: Re: [boost] [GSoC] [Boost.Hana] Formal review request
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2014-08-03 19:37:29

On 3 Aug 2014 at 12:58, Rob Stewart wrote:

> >> > Oh I know programmers similar to how you visualise code in your
> >head
> >> > would agree with you absolutely. But you must remember programmers
> >> > like me don't see C++ as really in fact having types nor classes
> >nor
> >> > concepts - I just see methods of programming the compiler to output
> >> > patterns of assembler code, and I think primarily in terms of
> >chains
> >> > of assembler instructions.
> 
> I have to assume you're using hyperbole to make your point.

I did make use of argumentum ad absurdum as otherwise no one would 
notice what I just said. People already filter out 75% of what you 
write, forcing you to repeat everything you say thrice. As I earn 
hourly, I can't afford that, writing email literally costs me money.  

> C++ has types and it has classes, so I can't believe you perceive it
> otherwise. OTOH, I can imagine that you see those things merely as levers
> and knobs to produce the desired machine code. 

You actually took the words out of my mouth - yes, levers and knobs 
is a good example, or a superior assembler macro implementation is a 
better one.

> >> > [snip]
> >> > It's the essential difference between language-focused coders and
> >> > err ... mongrel coders? I have to admit I'm not sure what to call
> >myself
> >> > really.
>
> You use abstractions whenever you discuss real world objects. In fact,
> you use them in any natural language conversation. I'm sure you use them
> in programming conversations, too. Consequently, I think you're
> overstating your case. 

I clarified what I meant in a subsequent email, but I was trying to 
explain that there are two big classes of programmer, one who thinks 
in terms of language and symbols and grammar, and the other who 
thinks in terms of stretchy sheets. That's also an argumentum ad 
absurdum, there are certainly more than just two classes, but 
hopefully you get what I mean.

> >> > Either way, I see ConceptCheck as a half baked feature
> >giving
> >> > me nothing useful but bloat and complexity and significantly adding
> >> > mess to documentation and steepening my learning curve.
> 
> Concepts are so much noise in introductory documentation. However, for
> detailed and reference docs, they are indispensable to avoid repetition
> and to reveal similarities and differences among the types on offer by a
> library. I wonder if your complaint is more about their use on specific
> cases than about them on general. 

You make a *very* valid point here. My biggest complaint is how they 
are used as an excuse for bad documentation, you are absolutely right 
on the button on that. My second, and much more minor complaint is 
they can make code more brittle than it should be because I don't 
think class inheritance nor SFINAE is the right way to implement 
policy/trait/concepts when improved language support is the only 
right way there, but that doesn't bother me anything like as much 
because I can simply go use an alternative library.

> >Practical programming isn't the right term ... I'm going to borrow a 
> >term Artur Laksberg used during my interview at Microsoft, let's call 
> >it "data flow centric programming". At scale, those patterns of 
> >assembler outputs start to look like stretchy sheets, and they come 
> >together to form a sort of hills and valleys. You could say that each 
> >CPU core is rather like a river trying to follow a path of least 
> >resistance amongst that topology, and our job as programmer is to 
> >apply transformations to the topology to yield new flows of river. It
> >is rare that applying any transformation isn't a zero-sum outcome, 
> >and that is where skill and experience comes in.
> >
> >When visualised in that way, there is no essential difference between 
> >programming languages, or operating systems, or the humans working on 
> >those systems. They're all programmed the same way.
> >
> >Advantages include agnosticism towards languages, OSs, idelogies ... 
> >disadvantages include inability to relate, communicate or otherwise 
> >transmit meaning to other engineers, except those of a similar bent. 
> 
> Your view of programming languages and libraries as means to an end
> doesn't mean you don't have to know the languages and libraries you
> happen to use. Documentation use natural and programming languages to
> convert information. Human communication uses abstractions for
> efficiency. Good abstractions are helpful, even to someone like you,
> aren't they? 

Of course. I just used a ton of similes to explain myself above which 
are the essence of abstraction. Abstraction basically *is* computer 
programming, perhaps even more so than maths.

What I was trying to convey was that some programmers get upset by 
the ugliness of breaking the purity of the type system, and it 
innately bothers them. I, on the other hand, have no problem with the 
monstrosity at 
https://github.com/ned14/nedtries/blob/master/nedtrie.h#L1853 which I 
would imagine would appall any right minded C++ person here (for 
those curious, it's a generic STL container wrapper which subverts 
that container to become a bitwise trie indexed container. The code 
is spectacularly evil, and includes casting references through 
intermediate placeholder types and back into their originals and lots 
of other similar fun. Note that I explicitly warn people in the 
documentation not to use it).

This is the difference I mean. Language/symbols/grammar programmers 
get upset at people using reinterpret_cast *at* *all* because it's 
seen like goto, a taboo. Me well I don't care about that, or goto 
either which I do use from time to time (it's sometimes the best 
solution available). What gets compilers to jump right in a 
maintainable way is all that matters to me (note I don't view the 
nedtries monstrosity as being maintainable, it was supposed to be a 
one-shot effort, I was quite mistaken on the popularity of that code 
sadly).

> >Most engineers accept the difference as a good thing adding an 
> >uncertain value, but the occasional one feels it as a vital threat 
> >that must be destroyed at all costs, which is unfortunate. That can 
> >introduce politics into engineering, which is usually a bad thing.
> 
> Differences are often beneficial. They can also be a source of friction.
> However, to my knowledge, I've never before encountered anyone with your
> view of programming in over 30 years of programming who wasn't using
> assembler. 

Well ... I still think of myself as primarily an assembler 
programmer, but I learned to not say so on my resume quite early on. 
I learned to program in 6502, moved into ARM, then adopted C as a 
more powerful macro programmer with the added benefit of being 
portable. That naturally evolved into C++ as an even more powerful 
macro engine and metaprogramming as even more powerful again, and I 
still debug mostly using the disassembly view which is why I still 
like Visual Studio so much (it has a much better disassembly based 
debugger than Eclipse). That way of seeing things is of course how a 
compiler writer sees things, and I found porting clang to QNX in my 
spare time whilst at BlackBerry effortless which some others thought 
too good to be possible or feasible or indeed wise.

This doesn't mean I can't write in other languages, which I'd call 
"interpreter" and includes BBC Basic, Python, Javascript, XSLT etc. 
They all have interpreter dynamics i.e. avoid creating and destroying 
things and you've probably got high performance. In 2011 as part of 
creating a cloudy Web 2.0 startup I wrote the main product in a 
combination of Python, C#, Javascript and Java, all of which was 
surprisingly quick and was far above the performance needed. If 
anything, in hindsight, I should have taken more shortcuts actually.

> >You're making the enormous assumption that all worthwhile programmers 
> >think like you and have the same value sets you do e.g. that domains 
> >have any relevance to parameter types at all, or that outstanding 
> >quality code cannot be written without a language-based approach.
> 
> I suspect, like me, he's never encountered someone like you. While the
> specific does not the general make, it certainly informs one's views. 

Many read my repeated posts here over these past eighteen months and 
have asked me personally why I bother. Your reply is why I do. It is 
still worth writing back here. Also, I would not have my current 
employment if it were not for those who read this mailing list and 
thought something of my posts, and I am extremely grateful for that.

> I'm not sure where I fit in your taxonomy, but I dislike
> Boost.Filesystem's documentation because Beman wrote it as if it was to
> be submitted for inclusion in the Standard, while providing little other
> information. I read the Standards, and I can extract beneficial
> information from them without a lot of difficulty, but it isn't helpful
> when trying to learn about something for the first time. 

I do see what you mean. However Filesystem is a very thin abstraction 
over POSIX. It contains few surprises, apart from the occasional bug. 
That significantly reduces my demand of its documentation - almost 
all of it can suffice with a single line and a few usage examples.

Where Filesystem seriously falls down in my opinion is its lack of 
API guarantees. For example, a big bugbear of mine is what are its 
guarantees of metadata consistency on a filesystem experiencing rapid 
change? If I read metadata about a file, and during the (let's say) 
five API calls used to read metadata that metadata changes, you end 
up with inconsistent metadata.

I want the relevant APIs to state how consistent they are on various 
operating systems. For example, POSIX lets us have stat which is 
atomic and all major POSIX implementation also provide a limited 
atomic directory enumeration API. Win32 provides lots of itty bitty 
calls which are not atomic, but going direct to the NT kernel gives 
you not just stat but a whole load of extra lovely stuff like you can 
atomically fetch the stat for all files matching a glob pattern in a 
directory.

I have tried to raise this with Beman before, but seen nothing about 
it. This sort of stuff affects how useful Filesystem is in the real 
world. Without guarantees Filesystem is not useful for database code 
etc.

> > I also want to see an essay at the beginning on the 
> >philosophy of the design (what I normally call a design rationale), 
> >because that will tell me how it all is intended to hang together. 
> 
> As do I.
> 
> >After all, as a result of the lack of direct language support there 
> >are many Concept check designs, each incommensurate with the others, 
> >hell even the STL itself has at least two if not three separate ways 
> >of Concept checking, none of which are fully baked either.
> 
> Flawed concept checking does not negate the value of concepts for the
> documentation. It just makes them less helpful at compile time. 

Ok, if you can deliver me documentation where the concept 
documentation appears at every point of use in a non-obtrusive way 
(let's say a wee '+' you can expand or hover over or something), I'll 
retract any objection I have to the use of concept checks in other 
people's code. You may need to give me a while to come around with my 
own code, but I think you're right earlier on, my single biggest 
objection is the laziness it introduces into documentation.

> >In truth, despite your assertion of me being in a minority, I suspect 
> >my sort of programmer is probably the majority in C++ if not in 
> >Boost.
> 
> Setting aside your view of languages as the means to the end of bits
> flowing over a landscape, I agree. 

FYI Artur Laksberg in Microsoft has a similar view to my own on flows 
of data. We talked about our similarity of vision at the pub during 
my interview there :)

> > If you want a pure or elegant type system you wouldn't choose 
> >C++ for example - in fact, if you cross off the list of reasons, the 
> >only reasons that *anyone* would intentionally choose C++ is down to 
> >(i) potential bare metal performance and (ii) it is also C which is 
> >useful for low level work and (iii) the tools and ecosystem are 
> >mature. That ought to bias against languagey type programmers, though 
>
> That's a misguided view. I choose C++ because it's multiparadigm. It
> provides low level tools, OO, meta-programming, etc. 

I can agree with that statement too.

> >for obvious reasons the best of the field whom I would assume would 
> >like to congregate here or on ISO are much more likely to be 
> >languagey type programmers as they are interested in C++ as a 
> >language.
> 
> I'm interested in improving my and other's ability to use C++ effectively.

And fittingly, we end on absolute agreement with one another. It's 
one of the reasons I volunteered to act as Boost GSoC admin, to help 
in getting more young blood in here which is sorely needed.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/