In my scenario I would like to run DHCP and TFTP on one machine and use a second running machine as the sender. Is this possible to do with UDPCast? This would mean copying a live (online) machine, is that a problem? How would I set this up?
Thanks
On Tuesday 20 July 2004 18:24, Lee Kransen wrote:
In my scenario I would like to run DHCP and TFTP on one machine and use a second running machine as the sender. Is this possible to do with UDPCast? This would mean copying a live (online) machine, is that a problem? How would I set this up?
Thanks
Never tried it out, at first glance it should be equivalent of restarting a machine which was improperly shut down. I.e. works in most cases, but occasionally goes fubar.
Actually, udpcasting a live machine could even lead to a worse situation, as the different sectors do not even correspond to one given moment in time (beginning of hard disks corresponds to an earlyer state), and it's quite possible that the filesystem drivers are less prepared with that kind of situation.
Regards,
Alain
Alain Knaff wrote:
On Tuesday 20 July 2004 18:24, Lee Kransen wrote:
In my scenario I would like to run DHCP and TFTP on one machine and use a second running machine as the sender. Is this possible to do with UDPCast? This would mean copying a live (online) machine, is that a problem? How would I set this up?
Actually, udpcasting a live machine could even lead to a worse situation, as the different sectors do not even correspond to one given moment in time (beginning of hard disks corresponds to an earlyer state), and it's quite possible that the filesystem drivers are less prepared with that kind of situation.
Actually, in practice, you'll probably find that most of the time it "just works". However, if you have LVM installed, you could make a snapshot of the image before you udpcast it. Most of the data that's not written to disk is "inoccuous" stuff such as logfiles. However, I wouldn't try this to make a copy of a running database, for instance.
YMMV
Chris
On Tue, 20 Jul 2004, Christopher Curtis wrote:
Actually, in practice, you'll probably find that most of the time it "just works". However, if you have LVM installed, you could make a snapshot of the image before you udpcast it. Most of the data that's not written to disk is "inoccuous" stuff such as logfiles. However, I wouldn't try this to make a copy of a running database, for instance.
Yes, this is what I am assuming. If the person states that the system must be live, it must be a production system of some type, providing an essential service that is accessed at all times of the day. There is a good possibility of files being in a state of being half written, and so on.
It won't even simulate a power loss, because a low level copy of the entire drive will capture parts of the drive at a different time. In the case of something like a database, there are several files in use, and some of them may be large. It might even take a few seconds to minutes to capture a single file on the low level dump, and during that time the file has been updated in several different parts at different times. It would lead to a file of garbage and thus a corrupted database.
A database is a easy example to understand, but the same could be true of any file that is being written on the system, including the filesystem table! Unless you are just fooling around, don't do it!
--Donald Teed
On Tue, 20 Jul 2004, Donald Teed wrote:
On Tue, 20 Jul 2004, Christopher Curtis wrote:
Actually, in practice, you'll probably find that most of the time it "just works". However, if you have LVM installed, you could make a snapshot of the image before you udpcast it. Most of the data that's not written to disk is "inoccuous" stuff such as logfiles. However, I wouldn't try this to make a copy of a running database, for instance.
Yes, this is what I am assuming. If the person states that the system must be live, it must be a production system of some type, providing an essential service that is accessed at all times of the day. There is a good possibility of files being in a state of being half written, and so on.
[...]
A database is a easy example to understand, but the same could be true of any file that is being written on the system, including the filesystem table! Unless you are just fooling around, don't do it!
I had a few occasions where I had to clone a life machine, e.g. because there was no place to store the image to and I needed to have a bootable system on a second machine, or just because I was too lazy to create an image first. As has been said before, most of the time it works fine, but the cloned machine will go through a file-system check during its first boot, because the file system has not been unmounted properly.
Of course, the system to be cloned should be as idle as possible. Certainly I would not recommend to clone a machine with active users on it or to use the cloned machine as a critical system.
- Felix
The machine I need to copy will be totally idle, except for normal system activity, with no active users, connections or intentionally running programs. The reason I thought to do it "live" is that the machine has no CD or diskette drive and not even a USB port. I need to update the machine every few days with the newest version of our system and then copy it to several 10's of new machines.
If I understand correctly, it seems that the best option is for me to take out the hard drive, put it in the DHCP/TFTP server and run the udp-sender from there. Is that correct?
If I decide to copy the disk to an image file and want to zero-ize the empty part of the drive, I understand that I can do this with dd. Could you tell me the command?
If I want to use lzop or gzip to UDPCast a saved image, do I have to save the image in the appropriate compressed format before the transfer or does UDPCast compress a saved image file on the fly during transfer?
Thanks a bunch,
Lee
-----Original Message----- From: udpcast-bounces@udpcast.linux.lu [mailto:udpcast-bounces@udpcast.linux.lu] On Behalf Of Felix Rauch Sent: Wednesday, July 21, 2004 1:58 AM To: Donald Teed Cc: udpcast@udpcast.linux.lu Subject: Re: [Udpcast] Cloning Live Machine
On Tue, 20 Jul 2004, Donald Teed wrote:
On Tue, 20 Jul 2004, Christopher Curtis wrote:
Actually, in practice, you'll probably find that most of the time it "just works". However, if you have LVM installed, you could make a snapshot of the image before you udpcast it. Most of the data that's not written to disk is "inoccuous" stuff such as logfiles. However, I wouldn't try this to make a copy of a running database, for instance.
Yes, this is what I am assuming. If the person states that the system must be live, it must be a production system of some type, providing an essential service that is accessed at all times of the day. There is a good possibility of files being in a state of being half written, and so on.
[...]
A database is a easy example to understand, but the same could be true of any file that is being written on the system, including the filesystem table! Unless you are just fooling around, don't do it!
I had a few occasions where I had to clone a life machine, e.g. because there was no place to store the image to and I needed to have a bootable system on a second machine, or just because I was too lazy to create an image first. As has been said before, most of the time it works fine, but the cloned machine will go through a file-system check during its first boot, because the file system has not been unmounted properly.
Of course, the system to be cloned should be as idle as possible. Certainly I would not recommend to clone a machine with active users on it or to use the cloned machine as a critical system.
- Felix _______________________________________________ Udpcast mailing list Udpcast@udpcast.linux.lu http://udpcast.linux.lu/mailman/listinfo/udpcast
On Wednesday 21 July 2004 12:52, Lee Kransen wrote:
The machine I need to copy will be totally idle, except for normal system activity, with no active users, connections or intentionally running programs. The reason I thought to do it "live" is that the machine has no CD or diskette drive and not even a USB port. I need to update the machine every few days with the newest version of our system and then copy it to several 10's of new machines.
If I understand correctly, it seems that the best option is for me to take out the hard drive, put it in the DHCP/TFTP server and run the udp-sender from there. Is that correct?
That's one possibility.
Other possibilities include:
1. Booting udpsender from network (DHCP) on that machine 2. or, Putting udpcast's kernel and initrd on the machine, and set up a lilo (or grub) entry to boot that. 3. or, Do the udpcast "live" anyways, but from a low runlevel (1), with root fs unmounted. As the FS is unmounted, the system is not really live as far as udpcast is concerned...
If I decide to copy the disk to an image file and want to zero-ize the empty part of the drive, I understand that I can do this with dd. Could you tell me the command?
On the live system, just create a huge file:
dd if=/dev/zero of=filler-file
After a while, the dd will error out with a "disk full" error.
Then, do a couple of syncs, and remove the file.
sync sync rm filler-file
If I want to use lzop or gzip to UDPCast a saved image, do I have to save the image in the appropriate compressed format before the transfer or does UDPCast compress a saved image file on the fly during transfer?
Thanks a bunch,
Lee
Udpcast is able to compress the file on the fly (using the -p flag), after all, that's how the bootable udpsender does it.
However, if you keep images on the server, it's best to precrompress them on the server for two reasons: 1. Space saving (obvious...) 2. CPU time & disk read time saving (may lead to a faster transfer, esp for those long stretches of zeroes)
Alain
On Wed, 21 Jul 2004, Lee Kransen wrote: [...]
If I understand correctly, it seems that the best option is for me to take out the hard drive, put it in the DHCP/TFTP server and run the udp-sender from there. Is that correct?
You could also setup an environment that boots into a RAM disk, with the help of DHCP, TFTP and PXE. We did that for our own 128-node cluster. The RAM disk contains the most essential tools and is otherwise kept at the minimum. In our case, the RAM disk uses roughly 80 MByte of our nodes' memory. When all the nodes booted into the RAM-disk-based system, we clone their disks (not with UDPcast, but the principle is the same).
- Felix
On Thu, 22 Jul 2004, Felix Rauch wrote:
On Wed, 21 Jul 2004, Lee Kransen wrote: [...]
If I understand correctly, it seems that the best option is for me to take out the hard drive, put it in the DHCP/TFTP server and run the udp-sender from there. Is that correct?
You could also setup an environment that boots into a RAM disk, with the help of DHCP, TFTP and PXE. We did that for our own 128-node cluster. The RAM disk contains the most essential tools and is otherwise kept at the minimum. In our case, the RAM disk uses roughly 80 MByte of our nodes' memory. When all the nodes booted into the RAM-disk-based system, we clone their disks (not with UDPcast, but the principle is the same).
- Felix
We're using PXE boot for udpcast. What some people don't realize is that the boot option support comes from the network card's boot ROM, not the mainboard BIOS (although it probably helps to have something semi-recent for the mainboard). Most inexpensive retail network cards do not support PXE boot. This is one of the distinguishing differences between the equipment supplied in a "business PC/notebook" and a similar model for "home users" - the ones marketed as business models have network boot support in case they need to be operated as client workstations.
Booting from a CDROM is normally something that is cheap to add to a system with a $20 CDROM drive (cheap, relative to someone's time in moving the hard drive back and forth each time the image is freshened). The only situation I can imagine where that isn't possible is with an older laptop or a very old 486 and previous generation machine.
Any approach of cloning a system with a mounted (writable) disk is doomed.
If you want to clone a system it cannot be operating from the device you want to clone. If that were possible, we wouldn't need to go to all of these lengths to boot from cdrom/floppy/network.
If you want to back up data on a live system, don't look at a cloning tool, but a backup tool.
--Donald Teed
On Tue, 20 Jul 2004, Lee Kransen wrote:
In my scenario I would like to run DHCP and TFTP on one machine and use a second running machine as the sender. Is this possible to do with UDPCast? This would mean copying a live (online) machine, is that a problem? How would I set this up?
Thanks
Udpcast mailing list Udpcast@udpcast.linux.lu http://udpcast.linux.lu/mailman/listinfo/udpcast
Hi,
When using FEC, the receiver might not terminate because it might enter a blocking read when it shouldn't.
Line 874 of receivedata.c (release date 31052004) is: ret=recvmsg(fd, &clst->data_hdr, 0);
Ideally, this line shouldn't be reached when clst->endReached is 2. But since the value is set by a different thread, it might reach the recvmsg() call if clst->endReached was set to 2 after the while loop (on line 845) is executed. In that case, the udp-receiver does not terminate since the thread is blocked by the recvmsg() call, although the file has been successfully received.
One way to work around this is to replace the recvmsg() call with a non-blocking version, as follows: while (1) { ret=recvmsg(fd, &clst->data_hdr, MSG_DONTWAIT); if (ret < 0) { if (errno == EAGAIN) { // execute the same code as in the while loop discussed } else break; } else break; }
Regards, Vivek
__________________________________ Do you Yahoo!? Yahoo! Mail Address AutoComplete - You start. We finish. http://promotions.yahoo.com/new_mail
-
Alain Knaff -
Alain Knaff -
Christopher Curtis -
Donald Teed -
Felix Rauch -
Lee Kransen -
vivek vedagiri