feature request: record build space used

[Ion-Mihai Tetcu]

On Thu, 24 Jul 2008 22:11:06 -0400
Joe Marcus Clarke <marcus at marcuscom.com> wrote:

> On Wed, 2008-07-23 at 21:46 +0300, Ion-Mihai Tetcu wrote:
> > On Wed, 23 Jul 2008 12:18:16 -0400
> > Wesley Shields <wxs at FreeBSD.org> wrote:
> > 
> >  On Wed, Jul 23, 2008 at 12:11:48PM -0400, Joe Marcus Clarke wrote:
> > > > On Wed, 2008-07-23 at 13:59 +0300, Ion-Mihai Tetcu wrote:
> > > > > Yeh, an other one ;)
> > > > > 
> > > > > 
> > > > > It would be very useful to me to have, for each port, stored
> > > > > in the database the total size of the space it occupies,
> > > > > together with base OS and dependencies, at build time. This
> > > > > value should be updated, if bigger that current values, even
> > > > > if the build fails.
> > > > 
> > > > How would one know the size before the port builds?
> > > > Dependencies are one thing, but we have no concept of size
> > > > (even distfile size) before buildscript is called.
> > > 
> > > You can get a rough idea of it based upon the previous build?
> > > Take the previous build and add on N% of space for growth?
> > 
> > Exactly. While it might fail for a few ports that suddenly grow, it
> > will work for 99%.
> > 
> > From my educated guess:
> > - for at least 50% of the ports tindy is spending more time untaring
> > the jail that doing anything else. 
> > - for 95% of the ports 2GB of RAM-based build space is more that
> > enough;
> > 
> > What I'd like is to be able to have everything in RAM for those 95%
> > without maintaining an exceptions list by hand. For the current
> > hardware QA Tindy is running on I should get at least 35% "speed"
> > improvement for a full run. (The jail tarball is cached by the fs in
> > memory for me).
> 
> Any word on my follow up questions?  I am interested in implementing
> these features, but I need more details.

Yeh, sorry, tough days.

Let me try to rephrase it:
What I want to know is how much space is required to build a port so I
can have all that done in memory-backed file system.

If I'm understanding the way tindy works correctly that space is:
- the untar'ed jail
- the untar'ed dependecies
- the untar'ed sources of the port to be built
- the results of the build process (.o files, binaries, docs, etc.)

The distfiles and packages are kept outside the build dir (which looks
something like /TINDERBOX_ROOT/BUILD_NAME) so they shouldn't be counted
in the necessary space.

The way I see it implemented:
- record the space size for the last build if successful.
- record the space size of the last build if unsuccessful and bigger
that the one currently recorded.
- add a ports_fail_pattered to look for 'out of space'
- have the recorded space exported in prePortBuild hook so that one
could use his favorite mount command for /TINDERBOX_ROOT/BUILD_NAME


Of course, this implementation will be far from perfect:
- needed space may vary because of non-default OPTIONS, NOPORT*, etc.
- it may change because of new/changed depedns
- etc.

But it would give a good indication of the magnitude of space needed.


-- 
IOnut - Un^d^dregistered ;) FreeBSD "user"
  "Intellectual Property" is   nowhere near as valuable   as "Intellect"
FreeBSD committer -> itetcu at FreeBSD.org, PGP Key ID 057E9F8B493A297B
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://marcuscom.com/pipermail/tinderbox-list/attachments/20080726/6835559f/attachment.bin>