[cmake-developers] CMake daemon for user tools

[Milian Wolff]

On Donnerstag, 21. Januar 2016 22:03:23 CET Daniel Pfeifer wrote:
> On Wed, Jan 20, 2016 at 10:03 PM, Stephen Kelly <steveire at gmail.com> wrote:
> > Milian Wolff wrote:
> >>> I'm concerned that the memory usage of a daemon implementing the
> >>> proposed
> >>> capabilities may be too large to be practical (at least without a major
> >>> redesign of certain structures that tend to duplicate substrings, or
> >>> some kind of out-of-core approach).
> >> 
> >> This sounds like optimizations to that daemon will benefit CMake.
> > 
> > FYI I merged a reduce-allocations branch to next for testing. It came from
> > 
> > Milian here:
> >  https://github.com/steveire/CMake/pull/1
> 
> Very nice! It is impressive what kind of an impact a simple 'const&'
> can have. Also, nice work identifying the hotspots.
> 
> > Milian also mentioned the possibility of using something like a sqlite
> > database (probably what you meant by out-of-core above) for definitions
> > for
> > querying by the daemon.
> > 
> > I also mentioned some possibile optimization possibilities, such as
> > removing the closure of definitions created for all directories. I wrote
> > a branch which does that some months ago, but it resulted in a slow down.
> > I'll see if I can rebase the commit to master and push it to github,
> > together with a patch for avoiding computing the hash multiple times in
> > cmDefinitions.
>
> What do you think about string interning? I started a cmString class
> [2] that stores short strings inplace and puts long strings into a
> pool. Copies never allocate extra memory and equality comparison is
> always O(1).
> In addition to `operator<` (which is lexicographical) there is a O(1)
> comparison that can be used where lexicographical ordering is not
> required (ie. for lookup tables).
> 
> [1] https://en.wikipedia.org/wiki/String_interning
> [2] https://github.com/purpleKarrot/CMake/commits/string-pool

Imo, you should replace this custom code by a boost::flyweight of std::string.
That said, I think this can have a significant impact on the memory footprint 
of CMake, considering how heavy it relies on strings internally. But it also 
seems to mutate strings a lot. I've seen places e.g. where a list of compile-
time known identifiers is prepended with "CMAKE_" at runtime. This is slow 
with normal strings (reallocations), but will also be slow with a flyweight or 
interning, possibly even leading to the pollution of the internal pool with 
temporary strings.

Did you measure any impact on both, runtime speed and memory footprint yet?
-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part.
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20160121/e60d8f2b/attachment.sig>