I used to have problems with the X11 server of Mac OS X 10.4 (X11.app): After a while, applications could no longer authenticate to the X server – programs that were running still worked, but I could not start any new X11 applications. If I restarted the X server, it worked for a while, then authentication would fail again.
The best explanation I found for this behaviour was this: The X server uses MIT magic cookies for authentication, and the cookies become invalid after the IP address changes. This is on a laptop, so I move between different networks all the time, and everytime I did, the X authentication would fail.
In previous versions of X11.app, there was a handy checkbox in the preferences to turn off authentication; unfortunately, this is gone in 10.4. The only way of turning off authentication is via the commandline:
$ defaults write com.apple.x11 no_auth 1 $ defaults write com.apple.x11 nolisten_tcp 1
The first command turns off authentication, the second one disables connections to the X server via TCP (programs can only use /tmp/.X11-unix/X0 unix domain socket to speak with the X server, which is faster and which they do anyway.) This limits the possibilities for mischief somewhat.
The downside of this is that you cannot use X via the network anymore – although you could get around this limitation by allowing TCP connections and firewalling the port off instead. And it is marginally unsafe, since any local program can connect to the X server, even if they were started by other users. But since this is practically a single user machine anyway, I do not care about the last part.