fetch / checksum problems with distcache

[Joe Marcus Clarke]

On Sun, 2009-03-01 at 09:09 +0200, Ion-Mihai Tetcu wrote:
> On Sat, 28 Feb 2009 13:52:04 -0500
> Joe Marcus Clarke <marcus at marcuscom.com> wrote:
> 
> > On Fri, 2009-02-27 at 21:02 +0200, Ion-Mihai Tetcu wrote:
> > > Hi,
> > > 
> > > 
> > > I wonder if:
> > > - if checksum fails and file was fetched from distcache it
> > > shouldn't be tried a second time w/o distcache
> > > - if checksum fails and there is a MASTER_SITE_OVERRIDE defined, a
> > >   second fetch should be tried ignoring MASTER_SITE_OVERRIDE
> > > 
> > > Unfortunately I don't see an easy way to do this in tindy's code.
> > > 
> > > A solution could be to override do-fetch:/do-checksum but I don't
> > > know if we want to do that since it would be equivalent to run a
> > > patched bsd.port.mk with all potential problems from that could
> > > come out of that.
> > > Not that I think that this kind of behaviour should be the default
> > > in bsd.port.mk (together with dropping -R from FETCH_ARGS).
> > > 
> > > 
> > > > ====================<phase 1: make checksum>====================
> > > > => uftp.tar doesn't seem to exist in /tmp/distfiles/.
> > > > => Attempting to fetch from file:///distcache//.
> > > > uftp.tar                                               130 kB 4476
> > > > kBps => MD5 Checksum mismatch for uftp.tar.
> > > > => SHA256 Checksum mismatch for uftp.tar.
> > > > ===>  Refetch for 1 more times files: uftp.tar uftp.tar 
> > > > => uftp.tar doesn't seem to exist in /tmp/distfiles/.
> > > > => Attempting to fetch from file:///distcache//.
> > > > => MD5 Checksum mismatch for uftp.tar.
> > > > => SHA256 Checksum mismatch for uftp.tar.
> > > > ===>  Giving up on fetching files: uftp.tar uftp.tar 
> > > > Make sure the Makefile and distinfo file
> > > > (/a/ports/ftp/uftp/distinfo) are up to date.  If you are
> > > > absolutely sure you want to override this check, type "make
> > > > NO_CHECKSUM=yes [other args]". *** Error code 1
> > > 
> > > 
> > > Any better ideas?
> > 
> > I agree this would make a good knob for bsd.port.mk.
> 
> Have to play with it, then :)
> 
> >  In the meantime, what about performing a cache removal in the case
> > that state 1 of a portbuild fails?  This way, a port will be
> > successfully rebuild on tinderbuild pass 2.

Well, you could schedule a rebuild of the ports that failed.  At least
by purging the case, manual intervention is not required.

> 
> It would do, but I'm trying to avoid the second pass on QAT, since it
> would also try to rebuild all ports that fail for other reason, thus
> increasing the response time drastically.
> 
> > This patch should handle that:
> > 
> > http://www.marcuscom.com/downloads/portbuild.diff
> 
> 

> I'm not sure about those 10 minutes, isn't it to much by far?

It's a worse-case scenario.  It means that if a lock cannot acquired
after 10 minutes, then give up.  This is the same timeout applied to the
cache insertion.

Joe

> 
> 
-- 
PGP Key : http://www.marcuscom.com/pgp.asc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <http://marcuscom.com/pipermail/tinderbox-list/attachments/20090301/c71807eb/attachment.bin>