October 2008 - Udpcast

Network card Error
by Darin MacLachlan 30 Dec '09

30 Dec '09

This was working and now it has stopped. Have a UDPcast server running on a HP DL360G5 with 4 NICS. One connected to an internal infrastructure (eth2) and another connected to a dedicated switch with the client systems on the same switch (eth3). When i run the UDPcast send or receive, using the following command: udp-sender -f /home/base.udpc --interface eth3 --point-to-point --full- duplex --max-bitrate 600m --nokbd --portbase=9922 --min-receivers 1 -- rexmit-hello-interval 5000 --max-wait 300 I get the following error message: Udp-sender 2007-03-23 No suitable network interface found The following interfaces are available: lo 127.0.0.1 eth2 10.46.0.251 eth2:1 10.46.0.1 Interestingly, an ifconfig show eth3 as a valid network adapter, but the udp-sender does not see the device. This was working last week, and nothing has changed on the server I have tried everything, restarting DHCP, removing the interface and adding again. Rebooted the server, restarted the network stack manually. nothing seems to help. Using SUSE linux 10 as the base server OS.

2 7

More on the corrupt file problem
by Kyle Cordes 18 Nov '08

18 Nov '08

To try to track down the udpcast corrupt file problem, I ran some more tests. This time I used a ~50GB file, a sender, and only 1 receiver. side bytes sender: 53687091200 receiver: 53686091776 In all of my large-file runs, udp-receiver comes up a bit "short", it missing some of the data, never "long". I created a 50 GB test file with predictable text data in it, suing this ugly little program: #include <stdio.h> // 16 bytes per entry. // int main(void) { long long gb = 1024 * 1024 * 1024; long long m = 50 * gb; long long i; for(i = 0; i<m; i+= 16) { printf("%.15lld\n", i); } } so that I could easily look at the files. I found that the received file ended with the same data as the sent file; in other words, the problem is *not* a matter of terminating early, or other finishing-out process. Rather, it's much earlier. According to "cmp": differ: byte 2098176010, line 131136001 That's a little under 2 GB of the way in to a 50 GB file. Strangely, I ran repeated tests with 10 GB files, and didn't get any corruption. Alain - it would warm my heart to see you ack these messages, even if you don't have a solution at hand. -- Kyle Cordes http://kylecordes.com

2 10

Quesiton on Kernel builds?
by Michael D. Setzer II 30 Oct '08

30 Oct '08

I just build some new udpcast cd images, and noted that the ATA drives are now showing up as sda instead of (hda) in the older version.The same issue occurred some time ago with Fedora. Don't recall the exact version where it changed. I maintain the g4l disk imaging project, and include the udp-sender and udp-reciever on it, but also recommend using udpcast for doing many machines at once. I actually create images on my servers via g4l and ftp, but the use udpcast to image the lab machines from one to many. For the g4l, I've build the kernel with everything built into the kernel, which makes a single file, and the cd has a number of various kernels since some work with one setup of hardware, while others may not. Except for one scis_wait something modual that always builds as a modual, it is just the single bzImage files that I rename. Have currently created images up the 2.6.27.4. This seems to work fine, and make the cd image easy to add, but it still has the ATA disk coming up as (hda). Not a problem, but I've had a couple of users say they are getting very slow disk performance on some setups. On my own machine, I seem to see no difference in speed, but I'm wondering if on some systems, it is selecting a slower compatible disk driver or something. At one point, I had a user with a SIS5513 controller that required a kernel patch to get higher speeds, but believe that was resolved in an updated to the kernel source that was suppose to get the correct bus speed. Wish I had more info on building kernels. Thanks. +----------------------------------------------------------+ Michael D. Setzer II - Computer Science Instructor Guam Community College Computer Center mailto:mikes@kuentos.guam.net mailto:msetzerii@gmail.com http://www.guam.net/home/mikes Guam - Where America's Day Begins +----------------------------------------------------------+ http://setiathome.berkeley.edu (Original) Number of Seti Units Returned: 19,471 Processing time: 32 years, 290 days, 12 hours, 58 minutes (Total Hours: 287,489) BOINC@HOME CREDITS SETI 6,681,217.6794 | EINSTEIN 2,067,250.6609 | ROSETTA 659,343.5601

2 5

Confusing, troubling UDPcast results
by Kyle Cordes 30 Oct '08

30 Oct '08

I'm using the latest (20080914) UDPcast, which I compiled from source on Ubuntu 8.04 on x64 (xeon). I get the same results I describe here, when I use the ancient (2004) UDPCast available in the Ubuntu packaging system. I also get the same results I describe here, whether I use "--file" or >file. I am transmitting data about 51GB in size from one machine to several others; it is a database backup that I pipe from pg_dump with something like this: pg_dump yadayada | udp-sender --no-kbd --autostart 90 and I receive it on 3 other machines with this: udp-receiver --portbase 10200 --file filenamehere On the sender, the output ends like so: bytes= 54 682 988 148 re-xmits=0166670 ( 0.4%) slice=0032 73 709 551 615 - 2 Transfer complete. Disconnecting #0 (ip-address-redacted) Disconnecting #1 (ip-address-redacted) Disconnecting #2 (ip-address-redacted) On receiver 1, it ends like so: bytes= 54 682 988 148 (123.12 Mbps) 54 682 848 884 Transfer complete. the file is 54682848884 bytes long. On receiver 2, it ends like so: bytes= 54 682 988 148 (123.11 Mbps) 54 682 848 884 Transfer complete. the file is 54682848884 bytes long. On receiver 3, it ends like so: bytes= 54 682 988 148 (123.13 Mbps) 54 682 938 996 Transfer complete. the file is 54682938996 bytes long. On this particular run, I got "lucky" in that two of the 3 receivers got the same file length; most of the time, 3 receivers end up with files of 3 different lengths. I think the correct length is 54 682 988 148, meaning that 0 of the 3 got the file delivered correctly. In the output, udp-receiver appears to know both the correct length and the actual written length, yet not be bothered by them being different (!!). I find this quite surprising and rather disturbing... I had planned to wrap an md5 mechanism around UDPcast somehow as an end-to-end check of the contents, but it's pointless to do that if it doesn't even get the correct *length* of the data. Any help much appreciated, as I find UDPcast to be a fantastic idea and nicely implemented tool, aside from this problem. Could there be some issue in which udp-receiver is confused between what go to the file/stdout (data payload) vs to stderr (absolutely everything else) ? -- Kyle Cordes http://kylecordes.com

1 1

checking write() return value
by Kyle Cordes 29 Oct '08

29 Oct '08

Trying to track down the problem I posted about earlier, I made a few tiny changes, including an assert-style check on write(). This case this code worries about, does not occur in my test with a ~50 GB file. Still, code like this is generally a good idea, rather than assuming that write() wrote the expected amount of data. Kyle diff --git a/receiver-diskio.c b/receiver-diskio.c index 2076ba2..500ac2a 100644 --- a/receiver-diskio.c +++ b/receiver-diskio.c @@ -43,14 +43,18 @@ int writer(struct fifo *fifo, int outFile) { * liberate small chunks one by one rather than attempt to * write out a bigger chunk and block reception for too * long */ - if (bytes > 128 * 1024) + if (bytes > 64 * 1024) bytes = 64 * 1024; - bytes = write(outFile, fifo->dataBuffer + pos, bytes); - if(bytes < 0) { + int bytesWritten = write(outFile, fifo->dataBuffer + pos, bytes); + if(bytesWritten < 0) { perror("write"); exit(1); } + if(bytesWritten != bytes) { + perror("write, bytesWritten != bytes"); + exit(1); + } pc_consumed(fifo->data, bytes); pc_produce(fifo->freeMemQueue, bytes); } -- Kyle Cordes http://kylecordes.com

1 0

Reducing frequency of status messages to stderr
by Kyle Cordes 29 Oct '08

29 Oct '08

Currently, udp-sender and -receiver send many updates per second to stderr describing the current progress and data rate. I'd like to reduce that to perhaps one update per second. Any suggestions on where in the source code to make such a change? -- Kyle Cordes http://kylecordes.com

1 0

Udpcast 2008-09-14: fullbox???
by Stephane Boireau 23 Oct '08

23 Oct '08

Hi, It seems, there is a trouble with building new image using 20080914 version. # ./makeImage -k kernel/vmlinuz-2.6.26.3udpcast --fullbox -c /home/steph/udpcast/udpcast_20081020.iso 24306 blocs Kversion=2.6.26.3udpcast Using ramdisk 01:00 I: -input-charset not specified, using iso-8859-15 (detected in locale settings) Size of boot image is 4 sectors -> No emulation Total translation table size: 2048 Total rockridge attributes bytes: 576 Total directory bytes: 0 Path table size(bytes): 10 Max brk space used 0 3055 extents written (5 MB) # Booting this ISO, no fullbox, not even ls or vi: # ls ls: not found # My host system is Debian etch (but I use alien to convert RPM to DEB because deb versions in http://udpcast.linux.lu/source.html require udpcast dépend de libc6 (>= 2.4-1) ; cependant : La version de libc6 sur le système est 2.3.6.ds1-13etch7. ) Regards. -- Stephane Boireau.

1 1

how to...
by javier.chen 20 Oct '08

20 Oct '08

Hi, alain, sorry, i do not have a clear expression of what I mean. My mean: power on(my client computer) -> show a menu -> choose udp-receiver, then i can receive the large file from server. What can i do? Using udpcast instead of PXE? And how to? Regard, Javier.Chen

2 1

hi all, what can i do now?
by javier.chen 17 Oct '08

17 Oct '08

I have compiled a good busybox(the menu and udpcast added), and also compiled a .rom file for my etherboot. And i want to make udp-sender/receiver run before linux start. Then what should I do? To put busybox into rom? or other things? Please.

2 1

there are some errors in udpcdialog, can u help me?
by javier.chen 17 Oct '08

17 Oct '08

In file included from udpcdialog//cursWrap.c:18: udpcdialog//dialog.h:32:20: error: curses.h: No such file or directory In file included from udpcdialog//cursWrap.c:18: udpcdialog//dialog.h:124: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'use_colors' udpcdialog//dialog.h:125: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'use_shadow' udpcdialog//dialog.h:128: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'attributes' udpcdialog//dialog.h:141: error: expected ')' before '*' token udpcdialog//dialog.h:145: error: expected ')' before '*' token udpcdialog//dialog.h:146: error: expected ')' before '*' token udpcdialog//dialog.h:147: error: expected ')' before '*' token udpcdialog//dialog.h:150: error: expected ')' before '*' token udpcdialog//cursWrap.c: In function 'udpc_clearScreen': udpcdialog//cursWrap.c:21: warning: implicit declaration of function 'attr_clear' udpcdialog//cursWrap.c:21: error: 'stdscr' undeclared (first use in this function) udpcdialog//cursWrap.c:21: error: (Each undeclared identifier is reported only once udpcdialog//cursWrap.c:21: error: for each function it appears in.) udpcdialog//cursWrap.c:21: error: 'LINES' undeclared (first use in this function) udpcdialog//cursWrap.c:21: error: 'COLS' undeclared (first use in this function) udpcdialog//cursWrap.c:21: error: 'attributes' undeclared (first use in this function) udpcdialog//cursWrap.c:22: warning: implicit declaration of function 'move' udpcdialog//cursWrap.c:23: warning: implicit declaration of function 'refresh' make[1]: *** [udpcdialog//cursWrap.o] Error 1 make: *** [udpcdialog/] Error 2

2 1