xsegbd: Don't destroy the segment, if it exists.