Instead of completely shutting down the display driver, shrink the subsurface when removing fullscreen so that when the parent window sends events when it receives damage or is resized.