[Udpcast] Scaling udpcast

Michael Holroyd meekohi at cs.virginia.edu
Thu Apr 24 23:15:22 CEST 2008


Hi Rich,
    I'd be very interested in a patch. I might have worked on one myself 
(I'd already edited the source to turn off the absurd stdout spam), but 
unfortunately my sender is a windows box and I wasn't in the mood to 
battle compiling udpcast for windows. The behavior I saw was that 50-55 
of my recievers would have the same md5sum, but it was still the *wrong* 
md5sum. The outliers could very plausibly be due to checksums being 
unreliable, and perhaps there was some bit-flipping that occurred before 
the first router sent everything out on multicast.
    Let me know how it goes if you decide to try it out,
    -Michael

Richard W.M. Jones wrote:
> On Thu, Apr 24, 2008 at 11:06:03AM -0400, Michael Holroyd wrote:
>   
>>    I tried to solve a problem much smaller than yours but still had 
>> incredible difficulty. I was moving 10GB datasets out to 64 receivers over 
>> a flat switched network using multicast. Unfortunately, for reasons I never 
>> tracked down, files of this size would always get corrupted along the way 
>> even though all the receivers had received all packets (i.e. the md5sum 
>> would be different across all the different machines).
>>     
>
> I haven't seen this problem (my tests are too small-scale probably)
> but I notice that the protocol doesn't do any sort of error detection
> for the dataBlocks.  So we're relying on UDP's 16 bit checksum and
> maybe ethernet's CRC32.  Both types of checksum are known to be very
> weak, and ethernet checksumming is even sometimes turned off.
>
> Shouldn't be too hard to add a more robust checksum to the packets.
> Is anyone interested in a patch?  I might have a go at one later.
>
> Rich.
>
> PS. My cheap-ish consumer switch slows down from gigabit-ethernet to
> 10 Mbps as soon as I ask it to do multicast or broadcast.  Is this
> normal?
>
>   




More information about the Udpcast mailing list