Red Hat Knowledgebase

On Red Hat Enterprise Linux 4 (RHEL4) GA through RHEL4.5, it is possible for the memory management data of a process to be modified while the process is still active on another processor, causing the system to hang. This problem only occurs when running 64-bit (x86_64) RHEL4 on an Intel processor that supports greater than 36-bit physical memory addressing or on an AMD processor that supports greater than 40-bit physical addressing. To determine if your processors are affected, issue the command cat /proc/cpuinfo and look for the physical address size reported on the line that begins with 'address sizes'. Here are two examples from affected systems:

Intel Processor

address sizes   : 38 bits physical, 48 bits virtual

AMD Processor

address sizes   : 48 bits physical, 48 bits virtual 

Since these processors are not supported in RHEL3, 32-bit RHEL4 is unaffected and the patch for this problem was included in RHEL5 at the time it was launched, the issue will only be seen on 64-bit RHEL4. RHEL4.6 and RHEL4.5 with the errata kernel-2.6.9-55-0.12.EL contain the necessary fix for this problem. Red Hat strongly recommends that you update to 4.6 or to the 4.5 errata kernel as soon as possible. Failure to implement these minimum Red Hat Enterprise Linux versions may lead to serious system errors.

Details:
The issue described above can potentially lead to stale TLB entries. Processor prefetching using these stale TLB entries can lead to different errors like Machine Check Exception (MCE) or application failures. The fix is to always reload Control Register 3 (CR3), which contains the base address of the page table hierarchy, when the memory manager updates a process' memory management data. This reloads the TLB list and flushes out any stale entries.