You can raise an exception which you then catch. Rails wraps deletes in a transaction, which helps matters.
It seems to happen if ActiveSupport unloads and reloads a constant that you have a reference to.
console reloads or code edits during specs have triggered this for me.