Dear list members,
I'm interested in the best case performance of UDPcast on fast networks like Fast Ethernet or Gigabit Ethernet. So far my results are not so good, but maybe I'm doing something wrong and maybe somebody can give me any hints.
For Fast Ethernet, I get the following performance (without disk access):
Clients Performance [MB/s] re-xmits 1 11.8 12 2 0.89 142290 4 0.93 103320 8 1.0 ?
So, while I get wire speed with only one client and server, I only get about 10% of the best performance for more than 1 client. Am I missing something? Are the high re-xmit-numbers normal or do they point to a problem? The commands I use are as follows:
node01:~/src/udpcast> ./udp-sender --file /dev/zero --full-duplex node02:~/src/udpcast> ./udp-receiver --file /dev/null
On Gigabit Ethernet the performance seems even worse, I can barely transmit any data. Here's what happens on the sender during 1 minute:
node01:~/src/udpcast> time ./udp-sender --file /dev/zero --full-duplex --interface eth1 --log /tmp/udpcast.log Udp-sender 2001-12-31 172.25.1.101 Using mcast address 236.25.1.101 UDP sender for /dev/zero at 172.25.1.101 172.25.1.101 on eth1 Broadcasting control to 172.25.255.255 New connection from 172.25.1.102 (#0) 00000019 Ready. Press any key to start sending data.
Starting transfer: 00000019 bytes= 163 072 re-xmits=000000 ( 0.0%) slice=0112 - 0 Received retransmittal request for 1 from 0: Retransmitting 1.41 Retransmitting 1.42 Retransmitting 1.43 Retransmitting 1.90 [CTRL-C] 0.000u 0.000s 1:06.53 0.0%mits=00+0k 0+0io 138pf+0we=0112 - 0
And on the receiver: node02:~/src/udpcast> time ./udp-receiver --file /dev/null --interface eth1 --log /tmp/udpcast.log Udp-receiver 2001-12-31 UDP receiver for /dev/null at 172.25.1.102 172.25.1.102 on eth1 received message, cap=00000019 Connected as #0 to 172.25.1.101 172.25.1.102 Listening to multicast on 236.25.1.101 Press any key to start receiving data! bytes= 326 144 ( 0.29 Mbps) Cancelled by user 0.000u 0.020s 1:15.49 0.0% 0+0k 0+0io 140pf+0w
Any ideas?
Regards, Felix
Hallo Felix :v) !
On Tue, 17 Dec 2002, Felix Rauch wrote:
Clients Performance [MB/s] re-xmits 1 11.8 12 2 0.89 142290
Argh!
I did some tests with 1:1 copying and came out with 85Mbps on FastEthernet which corresponds to your results. I did not though test with multiple clients assuming that the results would compare. I dare not hope they won't... :-/
*t
-- ----------------------------------------------------------- Tomas Pospisek SourcePole - Linux & Open Source Solutions http://sourcepole.ch Elestastrasse 18, 7310 Bad Ragaz, Switzerland Tel: +41 (81) 330 77 11 -----------------------------------------------------------
Le Mar 17 Déc 2002 15:56, Felix Rauch a écrit :
Dear list members,
I'm interested in the best case performance of UDPcast on fast networks like Fast Ethernet or Gigabit Ethernet. So far my results are not so good, but maybe I'm doing something wrong and maybe somebody can give me any hints.
For Fast Ethernet, I get the following performance (without disk access):
Clients Performance [MB/s] re-xmits 1 11.8 12 2 0.89 142290 4 0.93 103320 8 1.0 ?
So, while I get wire speed with only one client and server, I only get about 10% of the best performance for more than 1 client.
10% ? That suspiciously looks as if your switch is dropping down from 100Mb/s to 10Mb/s. This happens often on 10/100 switches, if one device is connected to it that cannot handle 100. Many switches don't understand IGMP, and thus do not know the composition of the multicast group. Multicast transmission goes out of _all_ ports (rather than only those where receivers are), and if one of the ports has a 10Mbps device attached, the switch slows down the communication.
The problem does not happen if there is only one receiver connected, because in that case udpcast uses unicast, rather than multicast. With unicast, switches usually know which ports should get the traffic, and which ones shouldn't (or else, it would be a hub, not a switch).
There are several ways around this: 1. Maybe your switch does understand IGMP, but this is just switched off? Can you check what happens if you enable it (IGMP snooping)? N.B. Some 3com switches, and maybe others as well, do have a config option for IGMP snooping, but it is not actually implemented... (i.e. even if enabled, it doesn't work).
2. Or if the above doesn't work, try disabling flow control in the switch. With that config, the switch will just drop the packets that it cannot transmit on the 10Mb/s ports (but the 100 Mb/s ports still enjoy the complete transmission).
3. If that doesn't help either (or is unavailable), try unplugging all devices that do not participate in the communication. It might be necessary to unplug the "uplink" to the root switch as well, if you have a multi-level switched network. In that case, DHCP may be unavailable, if your DHCP server is sitting on a different switch. Just configure the IP addresses by hand. What also works is to leave the uplink in during IP discovery, and to unplug it only when all stations have gotten their IP address.
With switches appropriately configured (i.e. "no flow control", our switches unfortunately don't understand IGMP), we get up to 90 Mb/s on our machines (if hard disks are fast enough).
Am I missing something? Are the high re-xmit-numbers normal or do they point to a problem? The commands I use are as follows:
node01:~/src/udpcast> ./udp-sender --file /dev/zero --full-duplex node02:~/src/udpcast> ./udp-receiver --file /dev/null
On Gigabit Ethernet the performance seems even worse, I can barely transmit any data. Here's what happens on the sender during 1 minute:
Not sure that this is the same problem as on 100Mbit/s. Indeed, as this technology is much more recent, most Gigabit switches (hopefully) should know how to correctly switch multicast traffic.
But maybe a different problem is occuring (but it might still be worth trying to disconnect from the Gigabit switch all devices that are unrelated to the transfer).
node01:~/src/udpcast> time ./udp-sender --file /dev/zero --full-duplex --interface eth1 --log /tmp/udpcast.log Udp-sender 2001-12-31
Can you also retry this with the half-duplex flag, just in case?
Alain
On Tue, 17 Dec 2002, Alain Knaff wrote:
There are several ways around this:
- Maybe your switch does understand IGMP, but this is just switched
off? Can you check what happens if you enable it (IGMP snooping)? N.B. Some 3com switches, and maybe others as well, do have a config option for IGMP snooping, but it is not actually implemented... (i.e. even if enabled, it doesn't work).
- Or if the above doesn't work, try disabling flow control in the
switch. With that config, the switch will just drop the packets that it cannot transmit on the 10Mb/s ports (but the 100 Mb/s ports still enjoy the complete transmission).
Unfortunately our switch is unmanaged, so I couldn't change the above.
- If that doesn't help either (or is unavailable), try unplugging all
devices that do not participate in the communication.
I did this and I got 94 Mbps for 2, 4 and 8 clients. So indeed, it seems to be a problem with the switch.
Not sure that this is the same problem as on 100Mbit/s. Indeed, as this technology is much more recent, most Gigabit switches (hopefully) should know how to correctly switch multicast traffic.
The switch is somewhat old (from the days when Gigabit Ethernet was still a new technology), but it should do the multicasting right. But it seems that the problem is not related to multicast (see below).
But maybe a different problem is occuring (but it might still be worth trying to disconnect from the Gigabit switch all devices that are unrelated to the transfer).
There are only 16 Gigabit Ethernet ports used on the switch (plus a management port).
Can you also retry this with the half-duplex flag, just in case?
I did so but it didn't help. What surprises me somewhat, is the fact the the performance is also very low for only 1 sender and 1 receiver. Here's what I tried:
On the receiver: node02:~/src/udpcast> time ./udp-receiver --file /dev/null --interface eth1 Udp-receiver 2001-12-31 UDP receiver for /dev/null at 172.25.1.102 172.25.1.102 on eth1 received message, cap=00000019 Connected as #0 to 172.25.1.101 172.25.1.102 Listening to multicast on 236.25.1.101 Press any key to start receiving data! bytes= 425 152 ( 0.42 Mbps) Cancelled by user 0.000u 0.010s 1:25.11 0.0% 0+0k 0+0io 138pf+0w
And on the sender: node01:~/src/udpcast> time ./udp-sender --file /dev/zero --half-duplex --interface eth1 Udp-sender 2001-12-31 172.25.1.101 Using mcast address 236.25.1.101 UDP sender for /dev/zero at 172.25.1.101 172.25.1.101 on eth1 Broadcasting control to 172.25.255.255 New connection from 172.25.1.102 (#0) 00000019 Ready. Press any key to start sending data. [RETURN] Starting transfer: 00000019 [CTRL-C about 80 seconds later] 0.000u 0.000s 1:22.93 0.0%mits=00+0k 0+0io 155pf+0we=0202 - 0
This should be a unicast transmission if I understand you right, but still I see much less than even 1 Mbps.
Regards, Felix