The deleter can just be set in the constructor and maintained throughout
the lifetime of the object.

If a contained pointer is null, then the deleter won't execute, so this
is safe to do. We don't need to swap it out with a version of a deleter
that does nothing.