Sorts out references that cause some modules to be kept around after
backend shutdown.
Should also solve the issue with errors being thrown due to the config
being loaded after device creation, leading to the incorrect device being
used in a multi-adapter system.