A highly dangerous privilege escalation vulnerability, which can allow an attacker to execute arbitrary code as root from any GUI application, has been patched in the Linux kernel.

The flaw was discovered by Rafal Wojtczuk, principal researcher at Invisible Things Lab (ITL), a security research company based in Poland.

According to Joanna Rutkowska, founder of ITL, the bug was discovered while Mr. Wojtczuk was working on GUI virtualization in Qubes OS, an operating system developed by the company, in which every application runs in a separate virtual machine.

"The attack allows a (unpriviliged) user process that has access to the X server (so, any GUI application) to unconditionally escalate to root (but again, it doesn't take advantage of any bug in the X server!).

"In other words: any GUI application (think e.g. sandboxed PDF viewer), if compromised (e.g. via malicious PDF document) can bypass all the Linux fancy security mechanisms, and escalate to root, and compromise the whole system," Ms. Rutkowska explains in a post on the company's blog.

The attack and the vulnerability are described in more detail in a paper (PDF) entitled "Exploiting large memory management vulnerabilities in Xorg server running on Linux," authored by Rafal Wojtczuk and published yesterday.

The flaw affects both x86_32 and x86_64 platforms and was reported to the X.org security team on 17 June 2010.

It was eventually agreed that the issue needs to be addressed in the Linux kernel, which was apparently vulnerable to the attack since version 2.6 was originally released.

On 13 August, Linus Torvalds committed an initial fix, but several patches were added afterward for various reasons. The problem has been addressed in versions 2.6.27.52, 2.6.32.19, 2.6.34.4 and 2.6.35.2 of the kernel.

This is described in a Red Hat security advisory giving the bug a "high" severity rating. Judging by Joanna Rutkowska's malicious PDF example, there is a remote attack vector associated with this vulnerability, which has been assigned the CVE-2010-2240 ID.

If anyone is wondering servers(like this one) shouldn't be effected by this because they shouldn't be running a GUI in the first place.