Red Hat Knowledgebase

There have have been some reports of anaconda using all available memory during an install on some systems with Red Hat Enterprise Linux 4 Update 3 on the x86_64 version of the OS.

The symptom of this failure is that during the install, partition formatting will begin and after formatting about 90% of /boot, it appears to be hung. If you switch to a virtual terminal and run top, you'll see anaconda growing, and eventually it will terminate with a "cannot allocate memory" error.

In order to work around this issue for network installs, download http://kbase.redhat.com/faq/downloads/RHupdates.tgz and extract it in the top of your fully exploded tree. This should go in the same directory as the images, isolinux, RedHat and SRPMS directories.
# tar xvfz RHupdates.tgz
The tar archive will create a RHUpdates directory with the files needed to resolve this problem.

For a network install from a directory of ISOs, you need to download http://kbase.redhat.com/faq/downloads/RHupdates-img.tgz and extract RHupdates-img.tgz in the same directory that holds the ISOs.
# tar xvfz RHupdates-img.tgz
This archive will create a RedHat/base directory with updates.img file needed to resolve this problem.

If you are doing a CD install you can workaround this problem by using mem=512M on the kernel boot line. If this method fails to work around the problem, you'll need to dd updates.img to a USB key or floppy and from the kernel boot line, boot with "linux updates". If you use "linux updates", you will not need to use "mem=512m".