Subject: [boost] Designing a multi-threaded file parser
From: Aaron Boxer (boxerab_at_[hidden])
Date: 2016-04-21 20:47:59
I hope this is not off-topic, but I am interested in hearing about boost
classes that will help
me design the following:
I am designing a multi-threaded file parser for C++11 threads.
The file format I am interested in parsing is a tagged format, with some
requiring further processing, and others not.
My working model right now is to have one thread through the file, and if
a tag that requires further processing, to push the file offset of this tag
into a queue,
to be processed by a thread pool.
The read thread will spend a lot of time waiting for file IO to complete.
Would it be
faster to us the ASIO classes to do the reading?
I would like to design the following thread scheduling library to help
process the tags:
The library would have the following features:
smallest unit of work for a thread is a job
jobs have dependencies - job FOO cannot complete until job BAR has
threads are divided into groups
groups can be divided into bundles (a bundle can share the same
processor affinity, for example)
groups are assigned to domains - domains contain jobs with similar
characteristics, for example slow I/O activity
domains contain queues of jobs
threads in a group assigned to a domain can execute any jobs in any of
the domain's queues
Does boost support this type of thread scheduling framework? Can boost help
me manage dependencies between jobs?
Sorry for these broad design questions - just hoping for some insight into
best design here.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk