I just got a 24 port 1GB switch and my first attempt definitely didn't give the results I was hoping for. Perhaps there are some tuning options that might improve the options.
First, I created an ntfsclone lzop compressed image of the Windows partition that is uploaded to one of the machines connected with the 1GB switch. That took about 11 minutes.
I then made sure that all the machines had the nics running at 1G. Originally, two machines were only connecting at 100M, but replacing the connector on 1 and just recrimping the other got them all to connect at 1G. The cables are all Cat5e, but they are a few years old.
I have a script that using udpcast to transfer this image file to all the other 19 machines in the lab connected to the machine. I was hoping to get a better speed when switching to the 100M to the 1G, but wasn't looking for a 10 times increase, since I knew that the hard disks only will do about 55MB/ second accordng to hdparm. But in my first test it took 31 minutes to transfer the file using udpcast??
Image file is about 12GB.
Here is the end of the listing. bytes= 12 221 254K re-xmits=32506028 (378.1%) slice=0112 - 15 Timeout notAnswered=[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18] notReady=[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18] nrAns=0 nrRead=0 nrPart=19 avg=252 bytes= 12 229 698K re-xmits=32529469 (378.2%) slice=0112 - 3 Transfer complete.
Didn't notice any machine giving errors, since it seemed to always list all 19 machines. The re-xmits seems to be an issue, but perhaps some other setting with the Linux (Fedora 16) or udpcast options.
The script is run from the machine machine, and it links to the other 19 machines and then transfers.
Partition /dev/sda6 is is a ext3 partition that is separate on all machines to store images, that can then be used to reimage individual machines with no network traffic.
if [ -e /sda6/d7-xp.lzo ] ; then echo "SDA6 already mounted" else sudo mount /dev/sda6 /sda6 fi cmd="hostname && mount /dev/sda6 /sda6 && cd /sda6 && ./udp-receiver --no-progress --file d7-xp.lzo && cd / && umount /sda6 " #cmd="udp-receiver --file d7-xp.lzo " plink -t 192.168.7.201 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.202 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.203 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.204 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.205 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.206 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.207 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.208 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.209 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.210 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.211 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.212 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.213 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.214 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.215 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.216 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.217 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.218 -l root -pw xxxxxx "$cmd" & plink -t 192.168.7.220 -l root -pw xxxxxx "$cmd" & sleep 20 cd /sda6 time udp-sender --full-duplex --file d7-xp.lzo --min-receivers 19 --min-wait 9
I'm hoping there are some more experience with using udpcast in 1G environments.
Currently running another test with sending to just two machines, but seems be showing similar results.
+----------------------------------------------------------+ Michael D. Setzer II - Computer Science Instructor Guam Community College Computer Center mailto:mikes@kuentos.guam.net mailto:msetzerii@gmail.com http://www.guam.net/home/mikes Guam - Where America's Day Begins G4L Disk Imaging Project maintainer http://sourceforge.net/projects/g4l/ +----------------------------------------------------------+
http://setiathome.berkeley.edu (Original) Number of Seti Units Returned: 19,471 Processing time: 32 years, 290 days, 12 hours, 58 minutes (Total Hours: 287,489)
BOINC@HOME CREDITS SETI 11801304.490587 | EINSTEIN 7206065.819851 ROSETTA 4194793.509365 | ABC 10747635.646984
What "gigabit" switch did you get? I've had several lemons. Check ethtool -S eth0 if you can, if you see lots of RX_PAUSE, your switch is cheating, spamming pause packets to slow down the traffic to rates it can handle. It also shows receive errors.
The client which didn't negotiate gigabit on the first try is strange. Having to force speed+duplex is one thing, but if the problem GOES AWAY, that suggests cable trouble to me. I haven't had negotiation problems that just go away when you try again. Negotiation doesn't test cable quality, it selects the maximum both support; it should've picked the maximum speed, first try, unless the cable was so bad negotiation just plain failed; after which both ends could've picked the same sane default.
Also, if you have one slow receiver -- not necessarily a network problem, a slow disk could do it -- everything will wait for the slow node.
-----Original Message----- From: udpcast-bounces@udpcast.linux.lu [mailto:udpcast-bounces@udpcast.linux.lu] On Behalf Of Michael D. Setzer II Sent: Wednesday, February 01, 2012 6:13 AM To: udpcast@udpcast.linux.lu Subject: [Udpcast] using udpcast with 1G ethernet.
I just got a 24 port 1GB switch and my first attempt definitely didn't give the results I was hoping for. Perhaps there are some tuning options that might improve the options.
First, I created an ntfsclone lzop compressed image of the Windows partition that is uploaded to one of the machines connected with the 1GB switch. That took about 11 minutes.
I then made sure that all the machines had the nics running at 1G. Originally, two machines were only connecting at 100M, but replacing the connector on 1 and just recrimping the other got them all to connect at 1G. The cables are all Cat5e, but they are a few years old.
I have a script that using udpcast to transfer this image file to all the other 19 machines in the lab connected to the machine. I was hoping to get a better speed when switching to the 100M to the 1G, but wasn't looking for a 10 times increase, since I knew that the hard disks only will do about 55MB/ second accordng to hdparm. But in my first test it took 31 minutes to transfer the file using udpcast??
Image file is about 12GB.
On 1 Feb 2012 at 12:09, Tyler Montbriand wrote:
From: "Tyler Montbriand" tyler@marieval.com To: udpcast@udpcast.linux.lu Date sent: Wed, 1 Feb 2012 12:09:52 -0600 Subject: Re: [Udpcast] using udpcast with 1G ethernet.
What "gigabit" switch did you get? I've had several lemons. Check ethtool -S eth0 if you can, if you see lots of RX_PAUSE, your switch is cheating, spamming pause packets to slow down the traffic to rates it can handle. It also shows receive errors.
It is a Dell 2724 switch I got off ebay and just downloaded the manual, so am complete new to it.
I did a test trying to use just two receivers to see if I could get better results, but was getting about the same results. Then did a single client test and got speeds of about 290MB with the two systems doing each separately, but when I tried to do them together afterwards it was giving like 2MB. I know that single transfers are unicast, so it seems to be something with the multicast setup. Have to look into getting in the switches management to setup the storm settings, but its default IP is not what this lab is on. Manual says 192.168.2.1
The client which didn't negotiate gigabit on the first try is strange. Having to force speed+duplex is one thing, but if the problem GOES AWAY, that suggests cable trouble to me. I haven't had negotiation problems that just go away when you try again. Negotiation doesn't test cable quality, it selects the maximum both support; it should've picked the maximum speed, first try, unless the cable was so bad negotiation just plain failed; after which both ends could've picked the same sane default.
The cables were made by our MIS department long ago, and the one I replaced had the wires to far out, and the other one I just recrimped and it then connected. Since I got a 280MB speed on the two other machines I selected, it doesn't seem to be a cable issue with those 3 at least.
Also, if you have one slow receiver -- not necessarily a network problem, a slow disk could do it -- everything will wait for the slow node.
I have seen this in the past with machines, where one system would slow down the others, but it generally would just report that one machine not responding. In this case, the messages always seem to say that all the machines not responding, so it looks like it is the switch that is doing something. Even with just two machines it kept reporting [0,1].
Thanks. Might even try using the broadcast options, and see what that does. This lab is on its own 192.168.7.x network off a linux server that has 9 ethernet ports, so it should isolate from rest of campus, or just disconnect it for tests?
-----Original Message----- From: udpcast-bounces@udpcast.linux.lu [mailto:udpcast-bounces@udpcast.linux.lu] On Behalf Of Michael D. Setzer II Sent: Wednesday, February 01, 2012 6:13 AM To: udpcast@udpcast.linux.lu Subject: [Udpcast] using udpcast with 1G ethernet.
I just got a 24 port 1GB switch and my first attempt definitely didn't give the results I was hoping for. Perhaps there are some tuning options that might improve the options.
First, I created an ntfsclone lzop compressed image of the Windows partition that is uploaded to one of the machines connected with the 1GB switch. That took about 11 minutes.
I then made sure that all the machines had the nics running at 1G. Originally, two machines were only connecting at 100M, but replacing the connector on 1 and just recrimping the other got them all to connect at 1G. The cables are all Cat5e, but they are a few years old.
I have a script that using udpcast to transfer this image file to all the other 19 machines in the lab connected to the machine. I was hoping to get a better speed when switching to the 100M to the 1G, but wasn't looking for a 10 times increase, since I knew that the hard disks only will do about 55MB/ second accordng to hdparm. But in my first test it took 31 minutes to transfer the file using udpcast??
Image file is about 12GB.
Udpcast mailing list Udpcast@udpcast.linux.lu https://udpcast.linux.lu/cgi-bin/mailman/listinfo/udpcast
+----------------------------------------------------------+ Michael D. Setzer II - Computer Science Instructor Guam Community College Computer Center mailto:mikes@kuentos.guam.net mailto:msetzerii@gmail.com http://www.guam.net/home/mikes Guam - Where America's Day Begins G4L Disk Imaging Project maintainer http://sourceforge.net/projects/g4l/ +----------------------------------------------------------+
http://setiathome.berkeley.edu (Original) Number of Seti Units Returned: 19,471 Processing time: 32 years, 290 days, 12 hours, 58 minutes (Total Hours: 287,489)
BOINC@HOME CREDITS SETI 11801304.490587 | EINSTEIN 7206654.049851 ROSETTA 4195943.642083 | ABC 10760546.984548
-
Michael D. Setzer II -
Tyler Montbriand