Hi Alain,
I was studying the UDPCast source code for some experimental purposes. I have the following question.
1. The whole data to be transfered into is divided into slices which again contains a series of blocks to transfer. Can we consider the whole data to be transfered as a single slice ( By setting appropriate parameters in code) and what kind of performance issues we can expect if we do a change of this kind at code level.
Thanks in advance,
Sai
---------------------------------
Do you Yahoo!?
Y! Messenger - Communicate in real time. Download now.
The option of ipappend 1 in the default file and retransmission
of HELLOs in udp-sender (--rexmit-hello-interval) has been
valuable to getting udpcast working with the Dell notebooks
we have with Broadcom 5700 series ethernet.
Today, with a slightly newer broadcom chipset appearing in
the most recent shipment, we noticed that udp-receiver was
intermittantly not showing the "hit any button to start transfer"
message. With some trial and error I found that if I
watched the tail of /var/log/message on the receiver (server)
and waited for the messages on TX and RX flow control to
complete before running udp-receiver, it would always initiate
a good connection. If I had started the udp-receiver
prior to the TX/RX flow control appearing in the message log
(in which case there was no "hit any key" message),
I could ^C the receiver, run it again and it would signify
the ready state with 100% success.
In conclusion, we have a workaround of starting udp-receiver
after a few seconds past the client PXE machine booting and
showing the udp-sender status ready (but not yet "hit any key...").
Another solution would be if udp-receiver also supported
--rexmit-hello-interval. It isn't a flaw in the udpcast system
but a kludge for a network device that is proving itself to be
sluggish in initialization in general.
--Donald Teed
I downloaded your CD image and it worked fine until I used it on a pc with a
USB keyboard. These PCs are in the lab I am trying the program on. Do I need
to compile my own image using my own kernal? I am using debian 3.1 and cannot
find the required files as specified by the -k switch.
Please, let me know what I need to do.
Thanks,
AJ
Quoting "Michael D. Setzer II" <mikes(a)kuentos.guam.net>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Two quick additions. You want to check the duplex status. I think if you are
> using a switch, it should be set to full, and I think the default is half.
I thought it defaulted to full... I'll poke around. It looks like there's a
problem with my images. :-( I'm getting the dreaded BSOD when I
restart saying
there's a problem with the hard drive... grrr.
>
> Second, I looked at Erasor, and it seems to be a security erasor, and did see
> from the page I looked at what pattern is left when it finishes. blank6 makes
> the space all null. If the pattern is a single byte, it should
> probable compress
> about the same, but it wirtes random patterns on the last run, it could make
> things worst.
It's actually configurable. I had it write all zeros instead of random data.
:-)
Thanks for the help,
Ben
>
> On 26 Feb 2006 at 20:48, Benjamin Moore wrote:
>
> Date sent: Sun, 26 Feb 2006 20:48:43 -0800
> From: Benjamin Moore <bmoore(a)tacticallanguage.com>
> To: udpcast(a)udpcast.linux.lu
> Subject: Re: [Udpcast] Lots of Timeouts using udpcast
>
>> Michael D. Setzer II wrote:
>>
>> >>Some questions about the setup. Is it one or two machines that is
>> giving the
>> >>timeouts? I have a lab of 20 machines, one was causing this problem. If I
>> >>imaged all the others together there would be very few timeouts compared
>> >>to thiis one. Turned out to be the hard drive in that machine,
>> which was the
>> >>same brand as all the others. In swapping the hard drive to
>> another machine
>> >>the problem moved from the one machien to the other. Ran diagnostics on
>> >>the drive,and it reports no errors, but it doesn't seem to be able
>> to keep up
>> >>with writing the sections of the drive that are blank, so the compressed
>> >>informaiton is coming in faster than the machine can write it to
>> the disk. I've
>> >>seen that these drives can only do about 45MB after the buffer gets full.
>> >>
>> >>
>> 2 Machines if I'm reading the error message correctly... I don't quite
>> understand what the error is trying to tell me. I'll try imaging them
>> individually and see if the problem goes away. You're theory about the
>> drive writing sounds plausible. I'll experiment, since I'm going to be
>> doing this off and on for the next 6-12 months.
>>
>> >>Also, what IP addresses are you using on the systems. I had a problem,
>> >>whereas the College MIS has 3 Class C blocks all the the same physical
>> >>network with a standard Class C subnet mask (Don't ask why??), and if I
>> >>used dhcp it would only work with machines on the same class C. I ended
>> >>up making an ipmac.txt file for the udpcast, and had it assign 10.0.7.x
>> >>numbers to my machines (Lab D-7).
>> >>
>> >>
>> 192.168.15.* But I'm doing my imaging completely off the external
>> network. All that's on the network is the server and the machines that
>> I'm imaging and they're all plugged into our HP 24 port switch.
>>
>> >>Also, you could try a cross-over cable and directly connect two machines,
>> >>and see if it differs from going thru the switch..
>> >>
>> >>
>> I'll give this a try as well, if I have time. If it is one of the
>> machines... yuck... but if I'm interpretting the error correctly it is
>> having a problem with both of them isn't it?
>>
>> >>Another issue that you might want to look at, are you clearing the unused
>> >>space on the drive before doing the transfer. I have a program
>> that I use with
>> >>my G4L project (at least the last four releases) and udpcast.
>> >>ftp://fedoragcc.dyndns.org/blank6.exe
>> >>Is a little C program (source code at the same site with .cpp),
>> and it works
>> >>for FAT and NTFS partitions. I just copy the file to the machine,
>> and from the
>> >>command prompt you blank6 c (c being the drive to clear). It just creates
>> >>2GB files full of nulls until the disk is full, and deletes them
>> when done. This
>> >>will speed up the compress and greatly reduce the time to
>> transmitt the data.
>> >>When I image my lab machines, I can get about 20+ MB/sec considering the
>> >>uncompressed size of the 80GB drives.
>> >>
>> >>
>> I used Erasor. But I'm using g4l for all the setup. It's a great
>> utility BTW.
>>
>> >>Don't know if that help, or just confused things more. Just yesterday, I
>> >>imaged the other 19 machines in my lab with 80GB drive in about 50
>> >>minutes with a hub.
>> >>
>> >>One other thing that you might want to look at is the
>> --max-bitrate= option.
>> >>I've found that if I set it to 90M it reduces these timeouts for the 18
>> >>machines, and 80M will sometimes have none. With the problem machine, it
>> >>usually takes 60M or lower to stop the errors.
>> >>
>> >>
>> I'll give this a try when I have more of them hooked up. I'm still
>> waiting for Dell to deliver 12 machines.
>>
>> Thanks for the help. (I'll be back at work trying this out soon. :-)
>>
>> Ben
>>
>> _______________________________________________
>> Udpcast mailing list
>> Udpcast(a)udpcast.linux.lu
>> https://lll.lgl.lu/mailman/listinfo/udpcast
>>
>
>
> +----------------------------------------------------------+
> Michael D. Setzer II - Computer Science Instructor
> Guam Community College Computer Center
> mailto:mikes@kuentos.guam.net
> mailto:msetzerii@gmail.com
> http://www.guam.net/home/mikes
> Guam - Where America's Day Begins
> +----------------------------------------------------------+
>
> http://setiathome.berkeley.edu
> Number of Seti Units Returned: 19,471
> Processing time: 32 years, 290 days, 12 hours, 58 minutes
> (Total Hours: 287,489)
>
>
> BOINC Seti@Home Total Credits 473413.537949
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: PGP 6.5.8 -- QDPGP 2.61c
> Comment: http://community.wow.net/grt/qdpgp.html
>
> iQA/AwUBRAIAcizGQcr/2AKZEQJ0EQCfVJePyV0Bh+sc+M3CBgEBQ2TFFdcAnA3W
> 0KnMee3jZAPpFNVSW5z9v1D5
> =31VA
> -----END PGP SIGNATURE-----
>
>
Michael D. Setzer II wrote:
>>Some questions about the setup. Is it one or two machines that is giving the
>>timeouts? I have a lab of 20 machines, one was causing this problem. If I
>>imaged all the others together there would be very few timeouts compared
>>to thiis one. Turned out to be the hard drive in that machine, which was the
>>same brand as all the others. In swapping the hard drive to another machine
>>the problem moved from the one machien to the other. Ran diagnostics on
>>the drive,and it reports no errors, but it doesn't seem to be able to keep up
>>with writing the sections of the drive that are blank, so the compressed
>>informaiton is coming in faster than the machine can write it to the disk. I've
>>seen that these drives can only do about 45MB after the buffer gets full.
>>
>>
2 Machines if I'm reading the error message correctly... I don't quite
understand what the error is trying to tell me. I'll try imaging them
individually and see if the problem goes away. You're theory about the
drive writing sounds plausible. I'll experiment, since I'm going to be
doing this off and on for the next 6-12 months.
>>Also, what IP addresses are you using on the systems. I had a problem,
>>whereas the College MIS has 3 Class C blocks all the the same physical
>>network with a standard Class C subnet mask (Don't ask why??), and if I
>>used dhcp it would only work with machines on the same class C. I ended
>>up making an ipmac.txt file for the udpcast, and had it assign 10.0.7.x
>>numbers to my machines (Lab D-7).
>>
>>
192.168.15.* But I'm doing my imaging completely off the external
network. All that's on the network is the server and the machines that
I'm imaging and they're all plugged into our HP 24 port switch.
>>Also, you could try a cross-over cable and directly connect two machines,
>>and see if it differs from going thru the switch..
>>
>>
I'll give this a try as well, if I have time. If it is one of the
machines... yuck... but if I'm interpretting the error correctly it is
having a problem with both of them isn't it?
>>Another issue that you might want to look at, are you clearing the unused
>>space on the drive before doing the transfer. I have a program that I use with
>>my G4L project (at least the last four releases) and udpcast.
>>ftp://fedoragcc.dyndns.org/blank6.exe
>>Is a little C program (source code at the same site with .cpp), and it works
>>for FAT and NTFS partitions. I just copy the file to the machine, and from the
>>command prompt you blank6 c (c being the drive to clear). It just creates
>>2GB files full of nulls until the disk is full, and deletes them when done. This
>>will speed up the compress and greatly reduce the time to transmitt the data.
>>When I image my lab machines, I can get about 20+ MB/sec considering the
>>uncompressed size of the 80GB drives.
>>
>>
I used Erasor. But I'm using g4l for all the setup. It's a great
utility BTW.
>>Don't know if that help, or just confused things more. Just yesterday, I
>>imaged the other 19 machines in my lab with 80GB drive in about 50
>>minutes with a hub.
>>
>>One other thing that you might want to look at is the --max-bitrate= option.
>>I've found that if I set it to 90M it reduces these timeouts for the 18
>>machines, and 80M will sometimes have none. With the problem machine, it
>>usually takes 60M or lower to stop the errors.
>>
>>
I'll give this a try when I have more of them hooked up. I'm still
waiting for Dell to deliver 12 machines.
Thanks for the help. (I'll be back at work trying this out soon. :-)
Ben
Hello,
I've just recently(as in the last hour) started playing with udpcast to image a
bunch of Windows XP machines.
I seem to have gotten everything working, but the throughput is kind of slow
because of the large number of time out errors I am getting.
This is the error:
Timeout notAnswered=[0,1] notReady=[0,1] nrAns=0 nrRead=0 nrPart=2 avg=<now
hovering around 59k>
Periodically udpcast seems to recover and the throughput goes back up. But then
it starts getting Timeouts again.
THe configuration right now is 2 machines recieving and the machine transmitting
all plugged into the same switch. I will be doing a second run with 20 machines
tomorrow.
It looks like everything is working otherwise, so I'm just curious what's going
on.
Thanks,
Ben
> Of course it does not work if you try to use gzip to decompress lzop
> compressed data (and vice-versa). These are two different compression
> schemes, and should not be mixed.
> If you consistently use lzop it works.
> If you consistently use gzip it also works.
Well I obviously wasnt mixing them. Im no bigginer in unix-like env to not
know gzip and lzop are different schemes. I was obviously not understood.
I'll try to explain it better:
The imaging server runs Gentoo Linux, ive installed udpcast-20050208 on
it. I booted the machine on which I maintain the image using PXE, firstly
chose gzip as my compression format of choice (ive been using udpcast for
over a year, but unfortunatly my old initrd got lost in the process of
upgrading), therefore ive started udp-sender, with gzip on that pc. On the
gentoo server i started udp-receiver. The image was saved no problem, then
i booted another 20 pcs on which i started udp-receiver via as well PXE,
and gzip decompression. and on the server udp-sender. Imaging worked, but
afterwards only NTFS partitions were readable, but none of the linux
partitions.
Then i tried lzop, knowing it is much faster than gzip, saving the image
on the server worked fine, but when i tried to ghost a machine with it
spat out an error (after about 200 000MB of compressed data was sent
across the client ) which i cant remember now but basically it would not
continue ghosting.
Then i tried uncompressed format, took ages, but when it was saved on the
server, i manually ran gzip and lzop on it and tested both(created extra
two files, one .gz and one .lzo), sent them across to the client on which
I chose either lzop or gzip (depending which file of the 2 i was sending)
and it worked perfectly. therefore there doesnt seem to be a problem with
udp-receiver but udp-sender that is compressing the data before it sends
it over the network to the Gentoo server.
The machines are perfectly identical.
Hi, udpcast is a great tool but:
Using gzip compression on udp-sender (2005-12-03) corrupts linux
partitions so they are unreadable, unmountable, unfsckable.
Using lzop compression on udp-sender (2005-12-03) makes udp-receiver fail
on receiving the image with an error.
Using no compression works perfect, but they are too big, so i manually
have to gzip or lzop them. (and they work!)
Im using initrd for PXE made by the web configurator.
Thanks.
Petr Janda