On Thursday 23 September 2004 09:31, Ramon Bastiaans wrote:

Hi all,

I was wondering if anyone knows or could tell what the bottleneck is for udpcast multicast speeds?

We have been using udpcast satisfactory for a while now in combination with systemimager. In our previous situation the maximum speed seemed to be around 20 Mbps (MAX_BITRATE). Setting the speed any higher would result in dropped slices in the cast and some receivers not getting the image in the first cast. This was on a 100 Mbps network.

Wow, that's slow. In comparison, Udpcast can almost saturate a 100 Mbps network (80 Mbps or beyond are easily achievable). Often, the bottleneck is not the network, but the local hard disk, especially when operating in compressed mode.

We now have a new setup where we image machines over a 1000 Mbps network, and the maximum speed seems to be 40 Mbps for udpcast? If we cast any faster the number of machines failing to successfully receive the cast dramatically increases (even though in the first tests, even at 40 Mbps there are 3 out of 40 still dropping).

I've no personal experience with 1000 Mbps networks, but from reports I got it seems that: * it cannot saturate the network * but 500 Mbps is achievable * paradoxically, "optimum" speed is achieved by limiting the bitrate (!). i.e. add --max-bitrate 600m

The reason for this seems to be that if data is sent too fast, 1000 Mbps equipment randomly drops frames, rather than using flow control, which seems to affect speed in a much worse way than if the bitrate is already limited at the source. Fortunately, no such phenomenon exists on most 100 Mbps switches.

To find the optimal limit on a 1000 Mbps network, start with 400m, and then increase the limit (by increments of 100m for instance), until the speed you get out of it no longer raises (or even falls).

Now this is on machines with 15.000 rpm scsi disk's. The hd and network both should be able to go as fast as 100 MByte/sec in theory. However we can't even get a stable cast at a 10th of this speed.

A tenth of the speed? This is very suspicious, as it puts the speed at 80 Mbps, i.e. a speed you could get on a 100 Mbps network. Make sure that there are no 100 Mbps devices directly or indirectly connected to the switch (even if they are not actually participating in the cast). Because if flow control _is_ enabled, the switch might slow down communication so that even these "slow" ports can follow. Yes, if IGMP snooping is enabled this should theoretically not be an issue, but there are a scary number of switches out there with buggy or missing IGMP support (i.e. even if a menu item "IGMP snooping" is present in the switch's management interface, it might be a no-op!). To find out whether this is the case, try disconnecting any equipment from the switch which is not directly participating in the multicast. If this is not feasible, try to swithch off flow control just on the "slow" ports (many switches support a per-port flow control setting).

It's not a very big issue since we save a lot of time by imaging all machines at the same time, but I was still wondering what exactly the bottleneck for the multicast's speed is.

If the issue is not 100 Mbps devices slowing down the transfer, try adding --max-bitrate 600m to the sender, and it should work.

Is it the udp protocol, or the multicast technique, or could it still be a hardware issue?

[if the issue is not the 100 Mbps device issue], probably some kind of hardware issue, but I couldn't yet exactly pinpoint what the issue is. As said, the problem mostly seems to arise on Gbps networks. On 100 Mbps networks, udpcast can saturate the network just fine, without any special tweaks.

[the "slow device" issue, OTOH, also happens on 100 Mbps networks, mostly in connection with network printers that only support 10 Mbps]

Any opinions on the subject are appreciated, perhaps some of the authors of udpcast could give some insight?

Kind regards,

Regards,

Alain

I think Alain provided an excellent tip on how to deal with the megabit ethernet. However I just want to add that in my experience the target client disk drive is always the bottleneck. It really depends on what you are imaging and whether compression is used.

Remember that the transfer rate is showing the packets transfer rate, and not the rate being written to the disk.

We use a 100 Mbit network to image 40 GB client notebook drives. Given that most of the drive on the master machine has been zero'ed, it compresses very well to less than 3GB of image. When we send out the image to the client notebooks, the sender machine disconnects from all clients, and the clients remain writing data (mostly highly compressed zeros) to disk from their 512 MB available RAM for the next 4 or 5 minutes, which covers about 5 or 6 GB of client disk writing.

The only way I can imagine the network being a bottleneck is if: - the network is already saturated with other traffic - the image is not created as compressed - the master disk is nearly full - the master disk is not zeroed in the empty sectors

--Donald Teed

On Thu, 23 Sep 2004, Ramon Bastiaans wrote:

Hi all,

I was wondering if anyone knows or could tell what the bottleneck is for udpcast multicast speeds?

We have been using udpcast satisfactory for a while now in combination with systemimager. In our previous situation the maximum speed seemed to be around 20 Mbps (MAX_BITRATE). Setting the speed any higher would result in dropped slices in the cast and some receivers not getting the image in the first cast. This was on a 100 Mbps network.

We now have a new setup where we image machines over a 1000 Mbps network, and the maximum speed seems to be 40 Mbps for udpcast? If we cast any faster the number of machines failing to successfully receive the cast dramatically increases (even though in the first tests, even at 40 Mbps there are 3 out of 40 still dropping).

Now this is on machines with 15.000 rpm scsi disk's. The hd and network both should be able to go as fast as 100 MByte/sec in theory. However we can't even get a stable cast at a 10th of this speed. It's not a very big issue since we save a lot of time by imaging all machines at the same time, but I was still wondering what exactly the bottleneck for the multicast's speed is.

Is it the udp protocol, or the multicast technique, or could it still be a hardware issue?

Any opinions on the subject are appreciated, perhaps some of the authors of udpcast could give some insight?

Kind regards,

-- Ramon Bastiaans SARA - Academic Computing Services Kruislaan 415 1098 SJ Amsterdam

---

Udpcast mailing list Udpcast@udpcast.linux.lu http://udpcast.linux.lu/mailman/listinfo/udpcast