The whole ownership model was getting a bit of a mess, with a some of special cases to deal with. And I'm planning to make it even more complex in the future. So here is some upfront work to convert it over to reference counted pointers.