[Udpcast] Scaling udpcast

Michael Holroyd meekohi at cs.virginia.edu
Thu Apr 24 17:06:17 CEST 2008


Hi Rich,
    I tried to solve a problem much smaller than yours but still had 
incredible difficulty. I was moving 10GB datasets out to 64 receivers 
over a flat switched network using multicast. Unfortunately, for reasons 
I never tracked down, files of this size would always get corrupted 
along the way even though all the receivers had received all packets 
(i.e. the md5sum would be different across all the different machines). 
Eventually I ended up using small bittorrent clients instead of udpcast 
since it checks the hash of each block. This also makes the process take 
about twice as long, but better to get correct data slow than corrupt 
data fast!

    Hope you have better luck,
    -Michael

Richard W.M. Jones wrote:
> I'm looking at using udpcast to broadcast large disk images (10+ GB)
> to a very large network of machines (1,000-10,000 receivers) over a
> mostly switched, partially segmented gig-ethernet network.
>
> Needless to say, the network of machines is all production-critical
> and I cannot get access to perform real testing.  However testing it
> on my home network I can see some potential problems:
>
>  - If _any_ receiver is misbehaving or unreachable then this stops
>    all transmissions.  Is there a way to get udpcast to drop
>    troublesome receivers in this situation (other than unicast)?
>
>  - Has anyone used the --ttl option to multicast over routers?
>    Does it work (the manpage is unclear)?  Does it need special
>    routers?
>
>  - Any other scaling tips?  Should I try to go for the full set of
>    machines at once or break up the broadcast into groups of machines?
>
> If anyone has used udpcast on such large networks, can you share any
> experiences.
>
> Rich.
>
>   




More information about the Udpcast mailing list