Broken Cursors in NixOS

When I migrated all my systems from Ubuntu to NixOS a few years ago, I kept a list of all the discrepancies I found and tried to fix them one by one. The last one on the list was wrong cursor sizes in some apps. I tried everything I could find but couldn’t fix it, and eventually gave up on it. I tried again six months later and gave up again. Recently I tried again and didn’t stop until I (mostly) fixed it. Here’s the full story, in case it helps someone else.

Symptoms ¶

The mouse cursor in certain X11 apps, like xterm, notion (my window manager), and Zoom, was not respecting my increased cursor size. This is especially a problem on high DPI screens where it’s easy to lose a 16 pixel cursor.

What’s happening ¶

There’s a thing called libXcursor. It loads cursor images from files and understands cursor themes and sizes.

The code lives with the other X11 libraries but is packaged separately. I gather (from the man page) the idea was that it was kind of experimental and not “core” enough to include in libX11 itself, even though it’s very common and useful functionality.

Here’s the odd part: many apps use libXcursor without linking to directly. How does that work? Well, libXcursor may not be core enough to be included in libX11, but at some point libX11 started using it to implement various functionality. Trying to use it: for some calls like XCreateGlyphCursor libX11 will actually dlopen libXcursor.so.1 and call functions in it! There’s a fallback implementation, but that one doesn’t handle themes and sizes and those nice things.

The whole thing is 80KB (including man pages) so I’m not sure why they don’t just fold it into libX11, but presumably that’s not changing at this point.

runpaths ¶

So… why does this work on other distros but not NixOS? Note that the dlopen is done with just a soname, not an absolute path. The dynamic linker searches for a soname along a search path. On FHS distros this would include something like /usr/lib and it would find it there. On NixOS, that’s not gonna work.

It will search in a few places, notably the runpath (or rpath… it’s complicated) that’s embedded in libX11, which Nix will have set to include its dependencies. But that doesn’t include libXcursor, so it doesn’t find it.

Fixes? ¶

Knowing all this, it should be easy to fix, right?

Let’s try to make the runpath of libX11 include the location of libXcursor. We just have to declare a dependency of libX11 on libXcursor… oh wait, that’s a circular dependency. Of course, libXcursor depends on libX11 😡

For the same reason, we can’t patch libX11 to call dlopen on an absolute path directly.

Well, to break a cycle we just need a third thing that points to both of them, right? Let’s take xterm: it already depends on libX11, let’s make it depend on libXcursor also. We can modify the build to put libXcursor in the runpath, so the dlopen will find it. Unfortunately, this doesn’t work either: when libX11 calls dlopen, it’s the runpath of libX11 that’s used, xterm’s runpath doesn’t matter at that point.

Still, there is something we can do here, and it’s really ugly: If we force xterm to explicitly load libXcursor by absolute path, then the later dlopen will find it already loaded and use it.

How can we force xterm to load a library? The simplest way is LD_PRELOAD: set that to point to /nix/store/…-libXcursor-1.2.1/lib/libXcursor.so.1 and that fixes the cursors. But wait, now I have LD_PRELOAD set in all my shells, so everything will try to load libXcursor, even terminal apps. This seems bad.

Patch ¶

The other way to get xterm to load a library is to have it call dlopen near the top of main. Yup, just patch the code. NixOS makes this really easy to do with an overlay. This approach is targeted exactly: only xterm will load the library.

final: prev: {

xterm = prev.xterm.overrideAttrs (old: {
  patches = old.patches ++ [
    (final.writeText "xterm-load-xcursor.patch" ''
      --- a/main.c
      +++ b/main.c
      @@ -94,4 +94,5 @@
       #include <graphics.h>

      +#include <dlfcn.h>
       #if OPT_TOOLBAR

      @@ -2510,4 +2512,6 @@ main(int argc, char *argv[]ENVP_ARG)
       #endif /* } TERMIO_STRUCT */

      +    // dlopen once here so it's cached when libX11 tries to dlopen it
      +    dlopen("${final.xorg.libXcursor}/lib/libXcursor.so.1", RTLD_LAZY);
           /* Init the Toolkit. */
           {
      '')
  ];
});

}

This does require rebuilding xterm but it’s pretty quick.

Too many rebuilds ¶

There’s still a problem: If you add just that overlay, you’ll probably notice Nix rebuilding lots more than just xterm. It turns out there’s a funny dependency from libadwaita → xvfb-run → xterm, and libadwaita is used by a bunch of apps, which would all need to be rebuilt if you change xterm.

Luckily we can cut this off by just leaving xvfb-run pointed to the old one:

# in the same overlay:
xvfb-run = prev.xvfb-run.override { xterm = prev.xterm; };

Other apps ¶

That’s xterm, what about, say, notion? Well, the big downside of this patch approach is that you have to adapt it for each app. For notion I did a similar overlay:

notion = prev.notion.overrideAttrs (old: {
  patches = [
    (final.writeText "notion-load-xcursor.patch" ''
      --- a/notion/notion.c
      +++ b/notion/notion.c
      @@ -17,4 +17,5 @@
       #include <sys/stat.h>
       #include <fcntl.h>
      +#include <dlfcn.h>

       #include <libtu/util.h>
      @@ -155,4 +156,7 @@ int main(int argc, char*argv[])
           bool nodefaultconfig=FALSE;

      +    // dlopen once here so it's cached when libX11 tries to dlopen it
      +    dlopen("${final.xorg.libXcursor}/lib/libXcursor.so.1", RTLD_LAZY);
      +
           libtu_init(argv[0]);

      '')
  ];
});

The last one that was bothering me, Zoom, is closed-source, so I can’t do the same. At this point I figured LD_PRELOAD is the best option there. Unlike xterm and notion, Zoom doesn’t launch other apps, so there’s no concern about “leaking” LD_PRELOAD.

Zoom’s derivation uses a postFixup to create some wrappers. I want to get the LD_PRELOAD in the wrappers, so I “patched” the script:

zoom-us = prev.zoom-us.overrideAttrs (old: {
  postFixup = builtins.replaceStrings
    ["--prefix LD_LIBRARY_PATH"]
    ["--set LD_PRELOAD ${final.xorg.libXcursor}/lib/libXcursor.so.1 --prefix LD_LIBRARY_PATH"]
    old.postFixup;
});

Here’s the final overlay: xcursor.nix

Finishing things up ¶

To actually get the cursors you want, you’ll need a few more things in your environment:

export XCURSOR_SIZE=48
export XCURSOR_THEME=Vanilla-DMZ
export XCURSOR_PATH=$(echo $XDG_DATA_DIRS | sed -E 's,(:|$),/icons&,g')

You can also set these in X resources, or let home-manager or something else handle them. See the Xcursor docs.

The size is self-explanatory.

That theme is from vanilla-dmz but you can use whatever you like.

The path thing takes advantage of the fact that NixOS already sets up XDG_DATA_DIRS to point to the right places, but it needs the /icons added for libXcursor.

The cursors in vanilla-dmz only go up to 48 pixels. On one of my machines I wanted bigger. Luckily, someone has already rendered larger versions of that theme into xcursor format: https://github.com/ganwell/dmz-cursors/

Fixing this for real ¶

Obviously these patches are too hacky to be upstreamed to nixpkgs. So what’s the real fix? Unfortunately there’s no easy one that I can see. Here are some ideas:

  • Make a new library that’s intended to be LD_PRELOADed, that would dlopen libXcursor and then unsetenv LD_PRELOAD so that children wouldn’t inherit it. Packages like xterm could be wrapped to LD_PRELOAD this library. This is slightly nicer than patching code, but it’s still fragile and I would hesitate to call it a “real fix”.
  • nixpkgs can package libXcursor in the same package as libX11. That eliminates the circular dependency, so when libX11 dlopens libXcursor, it would find it. Actually I’m not sure this would work, since libX11’s lib directory can’t be in its own RUNPATH. But it might work.
  • libX11 can just fold in libXcursor and eliminate the circular dependency right at the root.

More ideas are welcome!

References ¶

A few links that helped me figure this out: