[Udpcast] Scaling udpcast
Michael D. Setzer II
mikes at kuentos.guam.net
Thu Apr 24 23:24:10 CEST 2008
-----BEGIN PGP SIGNED MESSAGE-----
Just a quick addition to my earlier message. I just did a retest of doing the
imaging after opening both port 9000 and 9001 on both senders and
receivers. Sent the image to 19 receivers, and this time, it went thru with no
errors, and the files were the correct size? So, at least in this case, it would
appear that port 9001 is playing a part in this.
On 24 Apr 2008 at 17:14, Michael Holroyd wrote:
Date sent: Thu, 24 Apr 2008 17:14:37 -0400
From: Michael Holroyd <meekohi at cs.virginia.edu>
To: "Richard W.M. Jones" <rjones at redhat.com>
Copies to: udpcast at udpcast.linux.lu
Subject: Re: [Udpcast] Scaling udpcast
> Hi Rich,
> I'd be very interested in a patch. I might have worked on one myself
> (I'd already edited the source to turn off the absurd stdout spam), but
> unfortunately my sender is a windows box and I wasn't in the mood to
> battle compiling udpcast for windows. The behavior I saw was that 50-55
> of my recievers would have the same md5sum, but it was still the *wrong*
> md5sum. The outliers could very plausibly be due to checksums being
> unreliable, and perhaps there was some bit-flipping that occurred before
> the first router sent everything out on multicast.
> Let me know how it goes if you decide to try it out,
> Richard W.M. Jones wrote:
> > On Thu, Apr 24, 2008 at 11:06:03AM -0400, Michael Holroyd wrote:
> >> I tried to solve a problem much smaller than yours but still had
> >> incredible difficulty. I was moving 10GB datasets out to 64 receivers over
> >> a flat switched network using multicast. Unfortunately, for reasons I never
> >> tracked down, files of this size would always get corrupted along the way
> >> even though all the receivers had received all packets (i.e. the md5sum
> >> would be different across all the different machines).
> > I haven't seen this problem (my tests are too small-scale probably)
> > but I notice that the protocol doesn't do any sort of error detection
> > for the dataBlocks. So we're relying on UDP's 16 bit checksum and
> > maybe ethernet's CRC32. Both types of checksum are known to be very
> > weak, and ethernet checksumming is even sometimes turned off.
> > Shouldn't be too hard to add a more robust checksum to the packets.
> > Is anyone interested in a patch? I might have a go at one later.
> > Rich.
> > PS. My cheap-ish consumer switch slows down from gigabit-ethernet to
> > 10 Mbps as soon as I ask it to do multicast or broadcast. Is this
> > normal?
> Udpcast mailing list
> Udpcast at udpcast.linux.lu
Michael D. Setzer II - Computer Science Instructor
Guam Community College Computer Center
mailto:mikes at kuentos.guam.net
mailto:msetzerii at gmail.com
Guam - Where America's Day Begins
Number of Seti Units Returned: 19,471
Processing time: 32 years, 290 days, 12 hours, 58 minutes
(Total Hours: 287,489)
BOINC at HOME CREDITS
SETI 5,269,727.070797 | EINSTEIN 1,573,038.609732 | ROSETTA
-----BEGIN PGP SIGNATURE-----
Version: PGP 6.5.8 -- QDPGP 2.61c
-----END PGP SIGNATURE-----
More information about the Udpcast