Udpcast

udpcast@udpcast.linux.lu

554 discussions

maximum asynchronous speed and Jumbo Frames
by Martin Tullier 07 Mar '16

07 Mar '16

Hi there Richard, I have been looking into Data-Diodes using UDPcast as well over the past few weeks. I have seen speeds as high as 70Mbps on a 100Mbps ethernet link. It is usually more like 50+Mbps as this example shows. The file sent is 29,428,629 after uuencoding. The receiver side CMD: # time udp-receiver --interface p3p1 --start-time 30 --nosync --stat-period 2 --pipe /bin/uudecode Udp-receiver 20120424 Compressed UDP receiver for (stdout) at 10.37.49.21 on p3p1 Connected as #0 to 10.37.49.20 bytes= 29 428 629 ( 54.48 Mbps)) Transfer complete. real 0m4.680s user 0m0.166s sys 0m0.200s The sender side CMD: ]# time uuencode -m ./bigtestfile test33file | udp-sender --interface p1p1 --pointopoint --stat-period 3 --async --max-bitrate 100M --fec 16x16 --mcast-data-addr 10.37.49.21 stripes=16 redund=16 stripesize=128 Udp-sender 20120424 UDP sender for (stdin) at 10.37.49.20 on p1p1 Broadcasting control to 10.37.49.255 Ready. Press any key to start sending data. Starting transfer: 00000029 bytes= 29 428 629 re-xmits=0000000 ( 0.0%) slice=1024 - 00 Transfer complete. I am using 3 ethernet NICs to accomplish this: Two on the sender side, and One on the receiver. This is point to point and only about 8 feet in distance. The purpose of the 2 NICs on the sender is this: only one actually send the data 2 wires Tx. This NIC must receive "carrier" or somehow know that it is in fact connected to a network. Therefore the second NIC has its 2 wires Tx connected to the Rx side of the transmitting or sending NIC. Additionally all NICs involved must have their ethernet device attributes set the same manually. In my case that is 100Mbps half duplex with no autoneg there may be other attributes involve depending on the actual device used. All 3 NICs have and Ethernet ip address and ifconfig {device} up to bring them up. Since you are working with an actual Data Security Diode is a "Canary GT-10SD". How would you characterize this poor man's data diode? Also FYI: , no joy with upd-sender option "--nokbd". the sender is sending but with checksum errors. ______________________________ Martin Tullier

1 0

Maximum async..
by Brent Kimberley 26 Feb '16

26 Feb '16

‎Hi Rich, Your implementation should be run well below "5GT/s".‎ * Look at the name plate of your back plane, storage‎, adapter, network adapter, logic, etc.‎.. * Check the data flow(s). * Determine Who is chewing up What - Where - Why - When - How - Rest ... * perhaps one or more of components are running in an energy efficient mode ;-) ‎ THIS MESSAGE IS FOR THE USE OF THE INTENDED RECIPIENT(S) ONLY AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, PROPRIETARY, CONFIDENTIAL, AND/OR EXEMPT FROM DISCLOSURE UNDER ANY RELEVANT PRIVACY LEGISLATION. No rights to any privilege have been waived. If you are not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying, conversion to hard copy, taking of action in reliance on or other use of this communication is strictly prohibited. If you are not the intended recipient and have received this message in error, please notify me by return e-mail and delete or destroy all copies of this message.

1 0

Re: [Udpcast] Udpcast Digest, Vol 123, Issue 1
by Brent Kimberley 25 Feb '16

25 Feb '16

>> We can't get the network transmission [to exceed] 2-3 % [of nameplate] What does iperf report? Have you tried a link local subnet? >> 1000Mbps Have you tried Send Unreliable Datagram(UD) with immediate (or multicast)? -----Original Message----- From: Brent Kimberley Sent: Wednesday, February 24, 2016 12:28 PM To: 'richard.k.evans(a)nasa.gov' Cc: udpcast(a)udpcast.linux.lu Subject: RE: Udpcast Digest, Vol 123, Issue 1 Hi Richard, Your performance on a general purpose machine should exceed the following. 384mbps @9000 395mbps @4096 352mbps @1456 312mbps @1024 193mbps @256 63mbps @64 Best Regards, Brent <snippet> @REM cd %TEMP% @REM fsutil file createnew test1.tx 2199023255552 @REM dir test1.tx @REM cd %TEMP% .\udp-sender.exe -b <blocksize> --autostart 5 --stat-period 2000 --max-bitrate 1000m --async --fec 8x8 --mcast-rdv-addr 127.0.0.1 --mcast-data-addr 127.0.0.1 --interface 127.0.0.1 --file test1.tx @REM cd %TEMP% .\udp-receiver.exe --file test.<blocksize>.rx --nosync --mcast-rdv-addr 127.0.0.1 --stat-period 2000 --interface 127.0.0.1 @REM del test*.* udp*.exe </snippet> < receiver observations> UDP receiver for test.64.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K ( 63.23 Mbps)) Transfer complete. UDP receiver for test.256.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (193.83 Mbps)) Transfer complete. UDP receiver for test.1024.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (312.40 Mbps) Transfer complete. UDP receiver for test.4096.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (395.57 Mbps)) Transfer complete. UDP receiver for test.9000.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (384.52 Mbps) Transfer complete. UDP receiver for test.1456.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (352.54 Mbps) Transfer complete. </receiver observations> Message: 1 Date: Thu, 18 Feb 2016 18:39:30 +0000 From: "Evans, Richard K. (GRC-H000)" <richard.k.evans(a)nasa.gov> To: "udpcast(a)udpcast.linux.lu" <udpcast(a)udpcast.linux.lu> Cc: "Evans, Richard K. \(GRC-H000\)" <richard.k.evans(a)nasa.gov> Subject: [Udpcast] maximum asynchronous speed and Jumbo Frames Message-ID: <DC3FB55EE7FEFD409A6FC1EA00D50D0B0F5C4510(a)NDJSMBX203.ndc.nasa.gov> Content-Type: text/plain; charset="iso-8859-1" Hello, I am using the UDPCast in asynchronous mode to transmit large data files through a Data Security Diode (mono-directional network). Here is the command we use to transmit: Udp-sender.exe --async --fec 8x8 --max-bitrate 1000m --mcast-rdv-addr 224.2.2.1 --mcast-data-addr 224.2.2.2 --interface <ipTx> --file myfile.dat The Receive command I use is: udp-receiver.exe --nosync --mcast-rdv-addr 224.2.2.1 --interface <ipRx> --f myfile.dat Overall, this works well but we have noticed that we can't get the network transmission speed up above 2-3 % (22Mbps) of the Gigabit Ethernet adapter's full capability (1000Mbps). We also know that we are not processor limited, nor are we disk speed limited or anything else we can think of. I know that the asynchronous mode application is for satellite transmissions that are understandably slower than what a file transfer purely over Gibabit Ethernet can do, so I am writing here to ask if any one has any thoughts or insights about what might be the core issue that keeps the process running so slow. Fwiw, the Data Security Diode is a Canary GT-10SD, but for tuning UDPCast, we bypass the diode with a direct cable connection between Tx and Rx machines. So the performance of the system I am describing is without the diode in place. Thanks in advance, folks. -Rich - Richard Evans, NASA GRC - Plum Brook Station ? http://www.linkedin.com/in/rkevans ???216.973.4398 ------------------------------ _______________________________________________ Udpcast mailing list Udpcast(a)udpcast.linux.lu https://www.udpcast.linux.lu/mailman/listinfo/udpcast End of Udpcast Digest, Vol 123, Issue 1 *************************************** THIS MESSAGE IS FOR THE USE OF THE INTENDED RECIPIENT(S) ONLY AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, PROPRIETARY, CONFIDENTIAL, AND/OR EXEMPT FROM DISCLOSURE UNDER ANY RELEVANT PRIVACY LEGISLATION. No rights to any privilege have been waived. If you are not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying, conversion to hard copy, taking of action in reliance on or other use of this communication is strictly prohibited. If you are not the intended recipient and have received this message in error, please notify me by return e-mail and delete or destroy all copies of this message.

2 1

Re: [Udpcast] Udpcast Digest, Vol 123, Issue 1
by Brent Kimberley 24 Feb '16

24 Feb '16

Hi Richard, Your performance on a general purpose machine should exceed the following. 384mbps @9000 395mbps @4096 352mbps @1456 312mbps @1024 193mbps @256 63mbps @64 Best Regards, Brent <snippet> @REM cd %TEMP% @REM fsutil file createnew test1.tx 2199023255552 @REM dir test1.tx @REM cd %TEMP% .\udp-sender.exe -b <blocksize> --autostart 5 --stat-period 2000 --max-bitrate 1000m --async --fec 8x8 --mcast-rdv-addr 127.0.0.1 --mcast-data-addr 127.0.0.1 --interface 127.0.0.1 --file test1.tx @REM cd %TEMP% .\udp-receiver.exe --file test.<blocksize>.rx --nosync --mcast-rdv-addr 127.0.0.1 --stat-period 2000 --interface 127.0.0.1 @REM del test*.* udp*.exe </snippet> < receiver observations> UDP receiver for test.64.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K ( 63.23 Mbps)) Transfer complete. UDP receiver for test.256.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (193.83 Mbps)) Transfer complete. UDP receiver for test.1024.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (312.40 Mbps) Transfer complete. UDP receiver for test.4096.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (395.57 Mbps)) Transfer complete. UDP receiver for test.9000.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (384.52 Mbps) Transfer complete. UDP receiver for test.1456.rx at 127.0.0.1 on Software Loopback Interface 1 Connected as #0 to 127.0.0.1 bytes= 2 097 152K (352.54 Mbps) Transfer complete. </receiver observations> Message: 1 Date: Thu, 18 Feb 2016 18:39:30 +0000 From: "Evans, Richard K. (GRC-H000)" <richard.k.evans(a)nasa.gov> To: "udpcast(a)udpcast.linux.lu" <udpcast(a)udpcast.linux.lu> Cc: "Evans, Richard K. \(GRC-H000\)" <richard.k.evans(a)nasa.gov> Subject: [Udpcast] maximum asynchronous speed and Jumbo Frames Message-ID: <DC3FB55EE7FEFD409A6FC1EA00D50D0B0F5C4510(a)NDJSMBX203.ndc.nasa.gov> Content-Type: text/plain; charset="iso-8859-1" Hello, I am using the UDPCast in asynchronous mode to transmit large data files through a Data Security Diode (mono-directional network). Here is the command we use to transmit: Udp-sender.exe --async --fec 8x8 --max-bitrate 1000m --mcast-rdv-addr 224.2.2.1 --mcast-data-addr 224.2.2.2 --interface <ipTx> --file myfile.dat The Receive command I use is: udp-receiver.exe --nosync --mcast-rdv-addr 224.2.2.1 --interface <ipRx> --f myfile.dat Overall, this works well but we have noticed that we can't get the network transmission speed up above 2-3 % (22Mbps) of the Gigabit Ethernet adapter's full capability (1000Mbps). We also know that we are not processor limited, nor are we disk speed limited or anything else we can think of. I know that the asynchronous mode application is for satellite transmissions that are understandably slower than what a file transfer purely over Gibabit Ethernet can do, so I am writing here to ask if any one has any thoughts or insights about what might be the core issue that keeps the process running so slow. Fwiw, the Data Security Diode is a Canary GT-10SD, but for tuning UDPCast, we bypass the diode with a direct cable connection between Tx and Rx machines. So the performance of the system I am describing is without the diode in place. Thanks in advance, folks. -Rich - Richard Evans, NASA GRC - Plum Brook Station ? http://www.linkedin.com/in/rkevans ???216.973.4398 ------------------------------ _______________________________________________ Udpcast mailing list Udpcast(a)udpcast.linux.lu https://www.udpcast.linux.lu/mailman/listinfo/udpcast End of Udpcast Digest, Vol 123, Issue 1 *************************************** THIS MESSAGE IS FOR THE USE OF THE INTENDED RECIPIENT(S) ONLY AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, PROPRIETARY, CONFIDENTIAL, AND/OR EXEMPT FROM DISCLOSURE UNDER ANY RELEVANT PRIVACY LEGISLATION. No rights to any privilege have been waived. If you are not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying, conversion to hard copy, taking of action in reliance on or other use of this communication is strictly prohibited. If you are not the intended recipient and have received this message in error, please notify me by return e-mail and delete or destroy all copies of this message.

1 0

maximum asynchronous speed and Jumbo Frames
by Evans, Richard K. (GRC-H000) 18 Feb '16

18 Feb '16

Hello, I am using the UDPCast in asynchronous mode to transmit large data files through a Data Security Diode (mono-directional network). Here is the command we use to transmit: Udp-sender.exe --async --fec 8x8 --max-bitrate 1000m --mcast-rdv-addr 224.2.2.1 --mcast-data-addr 224.2.2.2 --interface <ipTx> --file myfile.dat The Receive command I use is: udp-receiver.exe --nosync --mcast-rdv-addr 224.2.2.1 --interface <ipRx> --f myfile.dat Overall, this works well but we have noticed that we can't get the network transmission speed up above 2-3 % (22Mbps) of the Gigabit Ethernet adapter's full capability (1000Mbps). We also know that we are not processor limited, nor are we disk speed limited or anything else we can think of. I know that the asynchronous mode application is for satellite transmissions that are understandably slower than what a file transfer purely over Gibabit Ethernet can do, so I am writing here to ask if any one has any thoughts or insights about what might be the core issue that keeps the process running so slow. Fwiw, the Data Security Diode is a Canary GT-10SD, but for tuning UDPCast, we bypass the diode with a direct cable connection between Tx and Rx machines. So the performance of the system I am describing is without the diode in place. Thanks in advance, folks. -Rich - Richard Evans, NASA GRC - Plum Brook Station http://www.linkedin.com/in/rkevans ©216.973.4398

1 0

UDPCast - Windows version question
by Aratea, Ceferino JR CIV NAVAIR 545100D 16 Dec '15

16 Dec '15

Could udp-sender.exe and/or udp-receiver.exe be converted to Windows service? How does it work? Thanks in advance. Very respectfully, JR Aratea "Protect society, the commonwealth, and the infrastructure." FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE This electronic transmission may contain information intended only for the person or persons named above. Any misuse or unauthorized disclosure may result in both civil and criminal penalties. **If you receive this transmission in error, please notify the sender at the telephone number or e-mail address listed. Thank you.

1 0

Windows UDPCast Version
by Aratea, Ceferino JR CIV NAVAIR 545100D 16 Dec '15

16 Dec '15

Do UDP-sender.exe and udp-receiver.exe be converted to Windows service? How does it work? Thanks in advance. Very respectfully, JR Aratea "Protect society, the commonwealth, and the infrastructure." FOR OFFICIAL USE ONLY - PRIVACY SENSITIVE This electronic transmission may contain information intended only for the person or persons named above. Any misuse or unauthorized disclosure may result in both civil and criminal penalties. **If you receive this transmission in error, please notify the sender at the telephone number or e-mail address listed. Thank you.

1 0

LAN insert news
by Adrianodnu 12 Sep '15

12 Sep '15

Good afternoon, I need to add more network adapters in udpcast list, as is the step by step procedure? Adriano das Neves. --- Este email está limpo de vírus e malwares porque a proteção do avast! Antivírus está ativa. https://www.avast.com/antivirus

2 1

A question about how udpcast works.
by Jeff Hali 11 Jun '15

11 Jun '15

Hello to the great team behind udpcast (hope this mail reaches you) I have a question about an odd thing that I have seen while using udpcast. As you can see there are 1544 and 96 length frames mixed together , I suspect that this may cause some errors on my setup. Could you tell me if these mixed lengths are a result of how udpcast works? What exactly is happening here? because this is unexpected for me. mac #5: frame-len=1538 (nzld=0), subframe-len=1544 mac #6: frame-len=1538 (nzld=0), subframe-len=1544 mac #7: frame-len=1538 (nzld=0), subframe-len=1544 mac #8: frame-len=90 (nzld=0), subframe-len=96 mac #9: frame-len=90 (nzld=32), subframe-len=96 mac #10: frame-len=90 (nzld=32), subframe-len=96 ^ sample , this pattern of interwoven lengths 1544/96 repeats itself in the transmission. Thanks and best regards. Jeff.

1 0

udp-receiver --start-timeout exits with 0 even if timed out
by Erik Jacobson 01 Dec '14

01 Dec '14

We use udpcast in an environment somewhat based on systemimager but highly modified. We have run in to a very sad situation where a particular model of switch decides it doesn't want to pass the multicast data packets and none of them get through. It is an extremely hard problem to duplicate because it seems to work 99% of the time. The end result when we hit the problem is the udp-receiver "waits forever". So I've begun to consider setting some additional timeouts in the system to try to work around this switch problem. I investigated receive-timeout and start-timeout. Since I'm never able to reproduce the problem "on demand", I emulated the problem by using iptables to block the data stream packets. I found that receive-timeout isn't in play in a situation where not one data channel packet has been sent. However, start-timeout is in play. There seem to be a couple select()-like situations that use the start-timeout. Prior to the "Connected as" message in udp-receiver, if you hit start-timeout there, udp-receiver will exit with an exit code that can be captured by a script for a re-try. However, the selectWithConsole() call in dispatchMessage(), while returning a 0 on select timeout, the caller (netReceiverMain()) doesn't test the return value of the timed out transfer. It turns out selectWithConsole is where I hit the timeout for the "no multicast data packets transferred" problem. After digging further, I realized this is likely because there are threads involved and this makes it more complicated to handle status. I'm rusty with my C but I came up with a work around to get us going. I am not suggesting that this is a good solution, but it does solve it for me and could be used as an illustration of my problem. Maybe some of you experts can quickly come up with the correct solution. Basically, I exit with 100 if zero bytes were transferred. Then I can test that easily in our scripts and re-try if needed in some sort of loop. I realize the correct solution is to exit with an error code if the selectWithConsole() call in dispatchMessage() times out, but it looked hard to deal with when combined with my C-programming rust. Other suggestions welcome. Attached is my "patch" for illustration reasons and not a suggested "fix" for the problem. Erik

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Udpcast