[cmake-developers] Faster automoc with parallel processing?

[Thomas Sondergaard]

Hi Alexander,

Thanks for replying.


On 2012-06-18 21:58, Alexander Neundorf wrote:
> On Sunday 17 June 2012, Thomas Sondergaard wrote:

>  > On Windows with MSVC 2010 Generator, moc seems to be very slow and it is
>  > a paint to see it run sequentially. Could automoc be taught to do this
>  > in parallel using all the cores on the machine?
>
> is "certainly" good enough as an answer ? ;-)

It's a good start :-)

> The thing is, for each target there is an accompanying automoc target.
> So if you have 4 cores, and 4 targets are built in parallel, then also 4
> (auto)mocs should be running in parallel.
>
> If you have only 1 target, consisting of many files, only 1 automoc
> target will be executed, processing one moc after the other.

Interesting. I haven't actually fully switched to automoc, I have only 
tested it on one target in a project with +50 targets. Perhaps the 
situation will look better if I automoc everything.

> Parallelizing this is not obvious. Simply running as many mocs on a
> machine as it has cores might lead to 16 mocs running on 4 cores (for 4
> different targets).

I think overparallization is better than underparallization. I don't 
think running 16 parallel mocs at a time on a 4 core machine should be 
dramatically slower than running 4. An experiment could show.

> So, it is possible, it is some implementation work, and I'm not clear on
> how to do the logic.
>
> Do you have some ideas how to do this in a good way ?

automoc could start a background process and use that to handle all 
moc'ing. The process could stop after a few seconds of inactivity. That 
way moc would never use more than the number of cores in the machine, 
but it might overlap with other build activity. It doesn't seem sensible 
that automoc should only be concerned about overutilization from other 
mocs. Perhaps the best way is simply to look at the load-average, like 
make --load-average?

Regards,

Thomas