[CMake] Converting a large C++-Project to CMake

[Michael Wild]

On 1. Nov, 2010, at 5:53 , Benjamin King wrote:

> Hi Karl!
> 
> 
>> I'm currently (as a consultant) converting ~2.5 Mio LOC C++/C from
>> vcproj/Makefiles/qmake to cmake.
> 
> Can you say something of the motivations for  your client to do this?
> 
> I suspect that you are expected to keep most of their development workflow and processes in order. Which parts are you allowed to redesign and which have to stay?
> 
> For us, there are also strong non-technical factors that will affect the decision to improve the build system, such as:
> 1) Is there some migration path from the old system?

Well, IMHO this really depends on your current build system. If it is flexible enough you should be able to port the parts of your project one by one.

> 2) Can we get around taking two weeks off where everybody needs to learn a new tool?

I don't think that it takes that long for everybody. There will be a few key-persons who will have to dig in deep, but the majority of the developers will be able to add and remove sources from the CMakeLists.txt files without even knowing the language. The same goes for include directories and link-libraries. Running CMake is another matter, but that should take a really short time to get a grasp on, and you could also provide a convenient wrapper script.

> 3) Can we keep the knowledge that went into the scripts, makefiles and qmake-projects of the current system?

I don't quite understand the point of this question. Also, it seems to be motivated by the "sunk costs fallacy". Apparently the current build system doesn't fulfill its requirements anymore and is fragile and from your statements I get the feeling that you don't think the issues can be easily rectified in the current framework. Cut your losses and move on. Whether that is CMake or something else that fits the bill doesn't matter so much.

> 4) Does it offer enough advantages to take the risk to change a matured and operative build system (albeit the latter is running at its limits)?

CMake itself is also quite mature and used for a large variety of projects, some of them huge (take KDE). Of course, the build-system you create with CMake will take some time to settle and grow into its full potential, but that is true for every other build system.

> 
> There are more points, but I'll need to address them all in order to convince anyone to even try cmake.
> 
> Right now, my plan is to convert a substantial part of the project to cmake. It needs to work with all our code generators.

What do you mean by a "code generator"?

> It will include a subproject that has a substantial amount of unit, integration and regression test that all need to be passed afterwards. And the migration path could be to just check in the CMakeLists.txt into the repository and be grateful that cmake builds out of source.

Keeping the two systems side-by-side seems to be a reasonable approach. If you used a decent SCM (sorry, I'm extremely biased against CVS/SVN), you could easily do that in a feature-branch, not disturbing current development. But with CVS I think it's best to have them side-by-side.

> 
> Part of my non-technical review will be the amount of time and pain it took me to do the conversion. The book (Mastering CMake) and the superfriendly and professional mailing lists will surely be big selling points there :-)

Just remember that the book is a bit out-dated. Although everything in there should still work with current CMake (AFAIK), there are some notable new features which could make your life easier (such as the ExternalProject module).

> 
> I'd be really interesting to know what non-technical issues you are facing with your current project!
> 
> 
>> - Automate everything (You need: continuous build and delivery, use
>> cdash or hudson)
>> - Complete build process (You need: checkout, build, packaging and install)
> 
> We do that with bash scripts and virtual machines.
> Which is why running cmake must be fully scriptable.
> I understand that it is if you are passing it the required configuration
> parameters with cmake -DTHIS_AND_THAT.

If you're willing to use CTest+CDash and CPack, you've got it all. Also, for the scripting them -C option might be of interest for you. Instead of passing tons of -D options, you write a script containing "set(... CACHE ...)" instructions to populate the cache.

> 
> 
>> There are others ways to build faster, like ccache
>> (http://ccache.samba.org/) or distcc (http://distcc.samba.org/).
> 
> We are using both for regular developer builds in linux. With release builds, we are a little more reluctant and expect everything to build in bearable time from scratch on a single machine. This works, just short of the bearable part.

CMake can speed up builds from scratch in my experience, but there's only so much you can do when building sequentially. What really helps is doing parallel builds with GNU Make (the -j option).

> 
> 
>> If you are going to change tools (introduce cmake), it is also time to
>> review your development process (If in the business world e.g. a
>> SAP-Systems is introduced the complete work flow and business processes
>> will get adapted).
> 
> I'll collect all the suggestions from this list and summarize them in a report.
> 
> Thanks a lot for being so helpful, that's much appreciated!
> 
> 
> Cheers,
> Benjamin

It also might help to get some inspiration from the many open-source projects using CMake out there. Of course, if you're developing some proprietary software, you'll have to be careful to not violate any licenses (i.e. don't just copy-paste, which often-times isn't a good solution in the first place). But you'll see a lot of examples on how to tackle a particular problem and in time you will also be able to improve on the ideas you collected.

HTH

Michael

--
There is always a well-known solution to every human problem -- neat, plausible, and wrong.
H. L. Mencken

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <http://www.cmake.org/pipermail/cmake/attachments/20101101/98aa92e8/attachment-0001.pgp>