You build 64-bit stuff (including boost if you wish) in 64-bit environment, 32-bit stuff (including boost if you wish) in a separate 32-bit environment. The environment is native for build tools, so you need no crosscompiling (-m32), no special library lookups. Google for 'linux 32-bit chroot' for more information.