[Udpcast] Scaling udpcast

Richard W.M. Jones rjones at redhat.com
Thu Apr 24 20:51:18 CEST 2008


On Thu, Apr 24, 2008 at 11:06:03AM -0400, Michael Holroyd wrote:
>    I tried to solve a problem much smaller than yours but still had 
> incredible difficulty. I was moving 10GB datasets out to 64 receivers over 
> a flat switched network using multicast. Unfortunately, for reasons I never 
> tracked down, files of this size would always get corrupted along the way 
> even though all the receivers had received all packets (i.e. the md5sum 
> would be different across all the different machines).

I haven't seen this problem (my tests are too small-scale probably)
but I notice that the protocol doesn't do any sort of error detection
for the dataBlocks.  So we're relying on UDP's 16 bit checksum and
maybe ethernet's CRC32.  Both types of checksum are known to be very
weak, and ethernet checksumming is even sometimes turned off.

Shouldn't be too hard to add a more robust checksum to the packets.
Is anyone interested in a patch?  I might have a go at one later.

Rich.

PS. My cheap-ish consumer switch slows down from gigabit-ethernet to
10 Mbps as soon as I ask it to do multicast or broadcast.  Is this
normal?

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v



More information about the Udpcast mailing list