I’m quite new to Genode, but since it seems relatively easy to port Linux drivers to Genode, how hard would it be to port, say, the Nouveau drivers for NVIDIA (or even the official open-source kernel module)?
Something like NVK could be used for the Mesa bits (someone in the Haiku community was able to port NVK to the open-source kernel modules, in fact)
Are there any complications I’m missing?
It is easy as in it is feasible in a manageable amount of time. The Genode Platforms 23.05 book (starting with “Taking Linux out for a walk”) describes the process on how to port a Linux driver over (with focus on an ARM SoC but it is generally applicable).
It is hard to say how difficult it actually would be to port the Nouveau DRM driver without taking a look at the inner workings first.
That being said, there is the intel_fb framebuffer driver based on the i915 driver that illustrates how to make use of the Linux driver for Genode’s Capture session.
The lima_gpu rendering driver based on Linux’ lima driver, on the other hand, shows how to connect the Linux’ DRM rendering API to Genode’s Gpu session (together with our libdrm port and naturally mesa).
So there are examples you can look at that show the custom/glue code required to make everything work and where to make a cut.
There is also a port of the open-source blob driver found in Managarm and from what I was told (second hand) using the abstraction layer provided by the driver directly was a straightforward endeavor and there is now a working framebuffer driver.
in general
Of course this driver only supports newer generations of devices but that would probably the road I would explore first (with having Haiku’s as well as the Managarm’s driver as examples) as the API seems to be more lean.
Well, GPU drivers tend to be quite complex beasts - as usual, the devil is in the details and there are the various parts (DRM, libdrm, mesa) involved and how to make use of them. Depending on your experience it is a lot to take in at first.
I would suggest to take a look at the available driver components first and get a feel for how they interact.
For a start I would focus on a framebuffer driver, i.e. only the display engine related parts, first as it would expose you to the DRM subsystem but not burden you with dealing with mesa.