Re: [Boost-users] Could searching and indexing Boost docs might work better?

Subject: Re: [Boost-users] Could searching and indexing Boost docs might work better?
From: Daniel James (dnljms_at_[hidden])
Date: 2010-09-19 14:58:30

On 19 September 2010 15:20, David Abrahams <dave_at_[hidden]> wrote:
>
> Does anyone on this list have (or have a friend with) SEO expertise?  They would know how to address this.  Otherwise, Paul, maybe you could try to ask Google themselves?

The problem is that bots are currently blocked from accessing the
documentation pages because they are considered too expensive. I've
reduced the cost of serving the documentation pages, but apparently
that's not good enough because the regression test results are still
served from zipfiles. Someone needs to sort that out with whoever runs
the regression testing system.

All they need to do is set up their script to unzip it somewhere on
the server after uploading, or possibly just upload the pages without
zipping them (maybe not a good idea - there are a lot of pages), and
then let me know where they are on the server.

But it might be the case that the cost of regularly unzipping/rsyncing
the files would be greater than the cost of serving them from a
zipfile. I suspect that a tiny minority of pages are accessed between
uploads (ignoring bots, there's no need to remove the block on the
regression results).

Daniel