Building_ids should not deleted for each AugmentedDiff after they are added by using flag clear_cache=True
clear_cache=True
Also solves some pytest errors
\approve @tara @ds