Boost logo

Boost-Commit :

Date view Thread view Subject view Author view

Subject: [Boost-commit] svn:boost r73077 - in website/public_html/beta: development site-tools site-tools/boost_site
From: dnljms_at_[hidden]
Date: 2011-07-13 18:16:37

Author: danieljames
Date: 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
New Revision: 73077
URL: http://svn.boost.org/trac/boost/changeset/73077

Log:
Website: Separate scripts for separate commands.

The single script with sub-commands works better when the script is
installed and on the path. Here the different commands are grouped
together since they're all in the same place.

Need to to handle the settings better.
Added:
   website/public_html/beta/site-tools/boost_site/settings.py (contents, props changed)
   website/public_html/beta/site-tools/boost_site/site_tools.py (contents, props changed)
      - copied, changed from r73076, /website/public_html/beta/site-tools/site-tools.py
   website/public_html/beta/site-tools/refresh.py (contents, props changed)
   website/public_html/beta/site-tools/update.py (contents, props changed)
Removed:
   website/public_html/beta/site-tools/site-tools.py
Text files modified:
   website/public_html/beta/development/website_updating.html | 7 +-
   website/public_html/beta/site-tools/README.txt | 25 ++++-------
   website/public_html/beta/site-tools/boost_site/site_tools.py | 86 +++------------------------------------
   3 files changed, 19 insertions(+), 99 deletions(-)

Modified: website/public_html/beta/development/website_updating.html
==============================================================================
--- website/public_html/beta/development/website_updating.html (original)
+++ website/public_html/beta/development/website_updating.html 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
@@ -324,10 +324,9 @@
                 <li>Create a new file in the <tt>/feed/news</tt> directory,
                 say <tt>/feed/news/gui_review_ends.qbk</tt>.</li>
 
- <li>In a shell, go to the <tt>/site-tools</tt> directory, and
- run <tt>python site-tools.py update</tt>. This will generate
- the page for the new news item, regenerate the
- <tt>/feed/news.rss</tt> file to include it, update pages
+ <li>In a shell, go to the <tt>/site-tools/update.py</tt>.
+ This will generate the page for the new news item, regenerate
+ the <tt>/feed/news.rss</tt> file to include it, update pages
                 which link to the new item and update the file
                 <tt>/site-tools/state/feed-pages.txt</tt> which tracks the
                 current state of generated files.</li>

Modified: website/public_html/beta/site-tools/README.txt
==============================================================================
--- website/public_html/beta/site-tools/README.txt (original)
+++ website/public_html/beta/site-tools/README.txt 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
@@ -2,26 +2,19 @@
 =======================
 
 This site contains several scripts for maintaining the Boost website.
-These are:
+Most users will just call `update.py` after updating any quickbook files.
 
-site-tools.py
--------------
+update.py
+---------
 
-A script to generate several of the pages on the Boost website,
-including the release notes, the home page and the downlaod page,
-as well as some other data for the site.
+Update the html pages and rss feeds for new or updated quickbook files.
 
-Usage: python site-tools.py [command]
+refresh.py
+----------
 
-Commands:
-
-update Update the html pages and rss feeds for new or updated
- quickbook files.
-
-refresh Reconvert all the quickbook files and regenerate the html
- pages. Does not update the rss feeds or add new pages.
- Useful for when quickbook, the scripts or the templates have
- been updated.
+Reconvert all the quickbook files and regenerate the html pages. Does
+not update the rss feeds or add new pages. Useful for when quickbook,
+the scripts or the templates have been updated.
 
 update-doc-list.php
 -------------------

Added: website/public_html/beta/site-tools/boost_site/settings.py
==============================================================================
--- (empty file)
+++ website/public_html/beta/site-tools/boost_site/settings.py 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
@@ -0,0 +1,45 @@
+# Copyright 2011 Daniel James
+# Distributed under the Boost Software License, Version 1.0.
+# (See accompanying file LICENSE_1_0.txt or http://www.boost.org/LICENSE_1_0.txt)
+
+settings = {
+ 'downloads' : [
+ 'feed/history/boost_1_47_0.qbk'
+ ],
+ 'pages': {
+ 'users/history/': {
+ 'src_files' : ['feed/history/*.qbk'],
+ 'template' : 'site-tools/templates/entry-template.html'
+ },
+ 'users/news/': {
+ 'src_files' : ['feed/news/*.qbk'],
+ 'template' : 'site-tools/templates/entry-template.html'
+ },
+ 'users/download/': {
+ 'src_files' : ['feed/downloads/*.qbk'],
+ 'template' : 'site-tools/templates/entry-template.html'
+ }
+ },
+ 'index-pages' : {
+ 'users/download/index.html' : 'site-tools/templates/download-template.html',
+ 'users/history/index.html' : 'site-tools/templates/history-template.html',
+ 'users/news/index.html' : 'site-tools/templates/news-template.html',
+ 'index.html' : 'site-tools/templates/index-src.html'
+ },
+ 'feeds' : {
+ 'feed/downloads.rss' : {
+ 'title': 'Boost Downloads',
+ 'matches': ['feed/history/*.qbk', 'feed/downloads/*.qbk'],
+ 'count': 3
+ },
+ 'feed/history.rss' : {
+ 'title': 'Boost History',
+ 'matches': ['feed/history/*.qbk']
+ },
+ 'feed/news.rss' : {
+ 'title': 'Boost News',
+ 'matches': ['feed/news/*.qbk', 'feed/history/*.qbk'],
+ 'count': 5
+ }
+ }
+}

Copied: website/public_html/beta/site-tools/boost_site/site_tools.py (from r73076, /website/public_html/beta/site-tools/site-tools.py)
==============================================================================
--- /website/public_html/beta/site-tools/site-tools.py (original)
+++ website/public_html/beta/site-tools/boost_site/site_tools.py 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
@@ -1,89 +1,17 @@
-#!/usr/bin/env python
 # Copyright 2007 Rene Rivera
 # Copyright 2011 Daniel James
 # Distributed under the Boost Software License, Version 1.0.
 # (See accompanying file LICENSE_1_0.txt or http://www.boost.org/LICENSE_1_0.txt)
 
-"""Usage: python site-tools.py [command]
-
-Commands:
-
-update Update the html pages and rss feeds for new or updated
- quickbook files.
-
-refresh Reconvert all the quickbook files and regenerate the html
- pages. Does not update the rss feeds or add new pages.
- Useful for when quickbook, the scripts or the templates have
- been updated.
-
-"""
-
 import os, sys, subprocess, glob, re, time, xml.dom.minidom, codecs
 import boost_site.templite, boost_site.pages, boost_site.boostbook_parser, boost_site.util
+from boost_site.settings import settings
 
 ################################################################################
 
-settings = {
- 'downloads' : [
- 'feed/history/boost_1_47_0.qbk'
- ],
- 'pages': {
- 'users/history/': {
- 'src_files' : ['feed/history/*.qbk'],
- 'template' : 'site-tools/templates/entry-template.html'
- },
- 'users/news/': {
- 'src_files' : ['feed/news/*.qbk'],
- 'template' : 'site-tools/templates/entry-template.html'
- },
- 'users/download/': {
- 'src_files' : ['feed/downloads/*.qbk'],
- 'template' : 'site-tools/templates/entry-template.html'
- }
- },
- 'index-pages' : {
- 'users/download/index.html' : 'site-tools/templates/download-template.html',
- 'users/history/index.html' : 'site-tools/templates/history-template.html',
- 'users/news/index.html' : 'site-tools/templates/news-template.html',
- 'index.html' : 'site-tools/templates/index-src.html'
- },
- 'feeds' : {
- 'feed/downloads.rss' : {
- 'title': 'Boost Downloads',
- 'matches': ['feed/history/*.qbk', 'feed/downloads/*.qbk'],
- 'count': 3
- },
- 'feed/history.rss' : {
- 'title': 'Boost History',
- 'matches': ['feed/history/*.qbk']
- },
- 'feed/news.rss' : {
- 'title': 'Boost News',
- 'matches': ['feed/news/*.qbk', 'feed/history/*.qbk'],
- 'count': 5
- }
- }
-}
-
-################################################################################
-
-def main(argv):
+def init():
     os.chdir(os.path.join(os.path.dirname(sys.argv[0]), "../"))
 
- if len(argv) != 1:
- print __doc__
- return
-
- command = argv[0]
-
- if command == 'update':
- return update_quickbook(False)
- elif command == 'refresh':
- return update_quickbook(True)
- else:
- print __doc__
- return
-
 def load_hashes(hash_file):
     qbk_hashes = {}
 
@@ -94,9 +22,12 @@
             qbk_hashes[qbk_file] = {'qbk_hash': qbk_hash, 'rss_hash': rss_hash}
         return qbk_hashes
     finally:
- file.close()
+ file.close()
+
+def refresh_quickbook():
+ update_quickbook(True)
 
-def update_quickbook(refresh):
+def update_quickbook(refresh = False):
     # Now check quickbook files.
     
     pages = boost_site.pages.Pages('site-tools/state/feed-pages.txt')
@@ -218,6 +149,3 @@
     })
 
 ################################################################################
-
-if __name__ == "__main__":
- main(sys.argv[1:])

Added: website/public_html/beta/site-tools/refresh.py
==============================================================================
--- (empty file)
+++ website/public_html/beta/site-tools/refresh.py 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
@@ -0,0 +1,17 @@
+#!/usr/bin/env python
+# Copyright 2007 Rene Rivera
+# Copyright 2011 Daniel James
+# Distributed under the Boost Software License, Version 1.0.
+# (See accompanying file LICENSE_1_0.txt or http://www.boost.org/LICENSE_1_0.txt)
+
+"""Usage: python refresh.py
+
+Reconvert all the quickbook files and regenerate the html pages. Does
+not update the rss feeds or add new pages. Useful for when quickbook,
+the scripts or the templates have been updated.
+"""
+
+import boost_site.site_tools
+
+boost_site.site_tools.init()
+boost_site.site_tools.update_quickbook()

Deleted: website/public_html/beta/site-tools/site-tools.py
==============================================================================
--- website/public_html/beta/site-tools/site-tools.py 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
+++ (empty file)
@@ -1,223 +0,0 @@
-#!/usr/bin/env python
-# Copyright 2007 Rene Rivera
-# Copyright 2011 Daniel James
-# Distributed under the Boost Software License, Version 1.0.
-# (See accompanying file LICENSE_1_0.txt or http://www.boost.org/LICENSE_1_0.txt)
-
-"""Usage: python site-tools.py [command]
-
-Commands:
-
-update Update the html pages and rss feeds for new or updated
- quickbook files.
-
-refresh Reconvert all the quickbook files and regenerate the html
- pages. Does not update the rss feeds or add new pages.
- Useful for when quickbook, the scripts or the templates have
- been updated.
-
-"""
-
-import os, sys, subprocess, glob, re, time, xml.dom.minidom, codecs
-import boost_site.templite, boost_site.pages, boost_site.boostbook_parser, boost_site.util
-
-################################################################################
-
-settings = {
- 'downloads' : [
- 'feed/history/boost_1_47_0.qbk'
- ],
- 'pages': {
- 'users/history/': {
- 'src_files' : ['feed/history/*.qbk'],
- 'template' : 'site-tools/templates/entry-template.html'
- },
- 'users/news/': {
- 'src_files' : ['feed/news/*.qbk'],
- 'template' : 'site-tools/templates/entry-template.html'
- },
- 'users/download/': {
- 'src_files' : ['feed/downloads/*.qbk'],
- 'template' : 'site-tools/templates/entry-template.html'
- }
- },
- 'index-pages' : {
- 'users/download/index.html' : 'site-tools/templates/download-template.html',
- 'users/history/index.html' : 'site-tools/templates/history-template.html',
- 'users/news/index.html' : 'site-tools/templates/news-template.html',
- 'index.html' : 'site-tools/templates/index-src.html'
- },
- 'feeds' : {
- 'feed/downloads.rss' : {
- 'title': 'Boost Downloads',
- 'matches': ['feed/history/*.qbk', 'feed/downloads/*.qbk'],
- 'count': 3
- },
- 'feed/history.rss' : {
- 'title': 'Boost History',
- 'matches': ['feed/history/*.qbk']
- },
- 'feed/news.rss' : {
- 'title': 'Boost News',
- 'matches': ['feed/news/*.qbk', 'feed/history/*.qbk'],
- 'count': 5
- }
- }
-}
-
-################################################################################
-
-def main(argv):
- os.chdir(os.path.join(os.path.dirname(sys.argv[0]), "../"))
-
- if len(argv) != 1:
- print __doc__
- return
-
- command = argv[0]
-
- if command == 'update':
- return update_quickbook(False)
- elif command == 'refresh':
- return update_quickbook(True)
- else:
- print __doc__
- return
-
-def load_hashes(hash_file):
- qbk_hashes = {}
-
- file = open(hash_file)
- try:
- for line in file:
- (qbk_file, qbk_hash, rss_hash) = line.strip().split(',')
- qbk_hashes[qbk_file] = {'qbk_hash': qbk_hash, 'rss_hash': rss_hash}
- return qbk_hashes
- finally:
- file.close()
-
-def update_quickbook(refresh):
- # Now check quickbook files.
-
- pages = boost_site.pages.Pages('site-tools/state/feed-pages.txt')
-
- if not refresh:
- for location in settings['pages']:
- pages_data = settings['pages'][location]
- for src_file_pattern in pages_data['src_files']:
- for qbk_file in glob.glob(src_file_pattern):
- pages.add_qbk_file(qbk_file, location)
-
- pages.save()
-
- # Translate new and changed pages
-
- pages.convert_quickbook_pages(refresh)
-
- # Generate 'Index' pages
-
- index_page_variables = {
- 'pages' : pages,
- 'downloads' : pages.match_pages(settings['downloads'], sort = False)
- }
-
- for index_page in settings['index-pages']:
- boost_site.templite.write_template(
- index_page,
- settings['index-pages'][index_page],
- index_page_variables)
-
- # Generate RSS feeds
-
- if not refresh:
- for feed_file in settings['feeds']:
- feed_data = settings['feeds'][feed_file]
- rss_feed = generate_rss_feed(feed_file, feed_data)
- rss_channel = rss_feed.getElementsByTagName('channel')[0]
- old_rss_items = pages.load_rss(feed_file, rss_feed)
-
- feed_pages = pages.match_pages(feed_data['matches'])
- if 'count' in feed_data:
- feed_pages = feed_pages[:feed_data['count']]
-
- for qbk_page in feed_pages:
- if qbk_page.loaded:
- item = generate_rss_item(rss_feed, qbk_page.qbk_file, qbk_page)
- pages.add_rss_item(item)
- rss_channel.appendChild(item['item'])
- elif qbk_page.qbk_file in old_rss_items:
- rss_channel.appendChild(old_rss_items[qbk_page.qbk_file]['item'])
- else:
- print "Missing entry for %s" % qbk_page.qbk_file
-
- output_file = open(feed_file, 'w')
- try:
- output_file.write(rss_feed.toxml('utf-8'))
- finally:
- output_file.close()
-
- pages.save()
-
-################################################################################
-
-def generate_rss_feed(feed_file, details):
- rss = xml.dom.minidom.parseString('''<?xml version="1.0" encoding="UTF-8"?>
-<rss version="2.0" xmlns:boostbook="urn:boost.org:boostbook">
- <channel>
- <generator>BoostBook2RSS</generator>
- <title>%(title)s</title>
- <link>%(link)s</link>
- <description>%(description)s</description>
- <language>%(language)s</language>
- <copyright>%(copyright)s</copyright>
- </channel>
-</rss>
-''' % {
- 'title' : details['title'],
- 'link' : "http://www.boost.org/" + feed_file,
- 'description' : '',
- 'language' : 'en-us',
- 'copyright' : 'Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or http://www.boost.org/LICENSE_1_0.txt)'
- } )
-
- return rss
-
-def generate_rss_item(rss_feed, qbk_file, page):
- assert page.loaded
-
- item = rss_feed.createElement('item')
-
- title = xml.dom.minidom.parseString('<title>%s</title>' % page.title_xml)
- item.appendChild(rss_feed.importNode(title.documentElement, True))
-
- title = xml.dom.minidom.parseString('<link>http://www.boost.org/%s>' % page.location)
- item.appendChild(rss_feed.importNode(title.documentElement, True))
-
- # TODO: Convert date format?
- node = rss_feed.createElement('pubDate')
- node.appendChild(rss_feed.createTextNode(page.pub_date))
- item.appendChild(node)
-
- node = rss_feed.createElement('boostbook:purpose')
- node.appendChild(rss_feed.createTextNode(page.purpose_xml))
- item.appendChild(node)
-
- if page.download_item:
- node = rss_feed.createElement('boostbook:downlaod')
- node.appendChild(rss_feed.createTextNode(page.download_item))
- item.appendChild(node)
-
- node = rss_feed.createElement('description')
- node.appendChild(rss_feed.createTextNode(page.description_xml))
- item.appendChild(node)
-
- return({
- 'item': item,
- 'quickbook': qbk_file,
- 'last_modified': page.last_modified
- })
-
-################################################################################
-
-if __name__ == "__main__":
- main(sys.argv[1:])

Added: website/public_html/beta/site-tools/update.py
==============================================================================
--- (empty file)
+++ website/public_html/beta/site-tools/update.py 2011-07-13 18:16:35 EDT (Wed, 13 Jul 2011)
@@ -0,0 +1,16 @@
+#!/usr/bin/env python
+# Copyright 2007 Rene Rivera
+# Copyright 2011 Daniel James
+# Distributed under the Boost Software License, Version 1.0.
+# (See accompanying file LICENSE_1_0.txt or http://www.boost.org/LICENSE_1_0.txt)
+
+"""Usage: python update.py
+
+Update the html pages and rss feeds for new or updated quickbook files.
+
+"""
+
+import boost_site.site_tools
+
+boost_site.site_tools.init()
+boost_site.site_tools.update_quickbook()

Date view Thread view Subject view Author view

Boost-Commit list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk