RSpec 2 usually cleans up after itself if you set config.use_transactional_fixtures to true. However, if you use Machinist 2, this isn't the case. Apparently Machinist likes to keep around a cached version to speed things up. This is bad when trying to run independent tests.
To prevent this from happening, add this to config/environments/test.rb.
Machinist.configure do |config|
config.cache_objects = false