rebuilding packages that haven't changed

[Chris Whitehouse]

Ion-Mihai Tetcu wrote:
> On Sat, 17 Apr 2010 04:17:56 -0400
> Joe Marcus Clarke <marcus at marcuscom.com> wrote:
> 
>> On Sun, 2010-04-11 at 23:56 +0100, Chris Whitehouse wrote:
>>> Hi
>>>
>>> I set up a test tinderbox and built a small package (shells/bash).
>>> ../packages/test/All contains
>>> Makefile
>>> bash-4.0.35.tbz
>>> bison-2.4.1,1.tbz
>>> gettext-0.17_1.tbz
>>> libiconv-1.13.1_1.tbz
>>> libtool-2.2.6b.tbz
>>> m4-1.4.14,1.tbz
>>>
>>> Now I do something like:
>>>
>>> # rm ../packages/test/All/libiconv-1.13.1_1.tbz
>>> # ./tc rmBuildPortsQueueEntry -i 21             # 21 = bash
>>> # ./tc addBuildPortsQueueEntry -b test -d shells/bash
>>>
>>> restart tinderd
>>>
>>> then other dependencies get rebuilt, here it's everything below
>>> Makefile # ls -ltr ../packages/test/All/
>>> total 5898
>>> -rw-r--r--  1 root  wheel   189463 Apr 11 22:14 m4-1.4.14,1.tbz
>>> -rw-r--r--  1 root  wheel   387674 Apr 11 22:32 libtool-2.2.6b.tbz
>>> lrwxr-xr-x  1 root  wheel       37 Apr 11 23:13 Makefile -> 
>>> /sata2/tinderbox/builds/test/Makefile
>>> -rw-r--r--  1 root  wheel  1491154 Apr 11 23:14
>>> libiconv-1.13.1_1.tbz -rw-r--r--  1 root  wheel  2558691 Apr 11
>>> 23:18 gettext-0.17_1.tbz -rw-r--r--  1 root  wheel   396562 Apr 11
>>> 23:18 bison-2.4.1,1.tbz -rw-r--r--  1 root  wheel   884451 Apr 11
>>> 23:20 bash-4.0.35.tbz
>>>
>>> The website says 'Tinderbox supports automatic tracking of
>>> dependencies and only rebuilds packages that changed since last
>>> run'. In fact a number of packages that haven't changed since last
>>> run have been rebuilt.
>>>
>>> Am I doing something wrong or just misunderstanding the statement?
>> Since you deleted libiconv, that port needs to be rebuilt as does all
>> of the ports which depend on it.  So, TB is behaving as designed.
>> Had you left the libiconv package in place, and just rebuilt bash,
>> then only bash would have been rebuilt.
> 
> Since this is something that confused me for long enough time, up to
> the point when I nudge marcus to do some changes in tindy's code so
> that I could avoid this, here's a little more verbose explanation:
> 
> For A depends on B, B depends on C:
> 
> - none built, schedule A ==> C build, then B, then A
> - C built, schedule A ==> B built, then A built
> - B and C built, schedule A ==> only A built
> - A, B and C built, schedule A ==> only A built
> 
> Now, suppose you have A, B, C built, but you remove or rebuilt C and
> schedule A (your case): tindy produces a Makefile, which each package
> as a target (I'm simplifying things here for didactic proposes):
> 
> A.tbz:
> 	B.tgz
> 	cd ${PORTSDIR}/A_categ/A && make package
> 
> B.tgz:
> 	C.tgz
> 	cd ${PORTSDIR}/B_categ/B && make package
> 
> C.tgz:
> 	cd ${PORTSDIR}/C_categ/C && make package
> 
> this is really make(1), so if you delete C, or rebuild it, the
> resulting package C.tgz from 'C.tgz:' target is *newer* that B.tgz
> package, so the target B.tgz *is out of date* (see make(1) for details).
> Tindy being a testing system first, this is correct, as you might have
> changed something in C, without bumping PORT{VERSION,REVISION,EPOCH}, so
> in order to be sure of the result, all packages that depend on C must
> be rebuilt.
> 
> I considered hacking around this on QAT, but after a careful analysis
> the number of potential problem introduced greatly overcome the
> benefits. Maybe YMMV.
> 
> If you are interested in speed, setup ccache, it will help up to 60-70%.
> 
Yes makes sense now thank you, and for the ccache tip.

thanks

Chris