If you're using version history then you could look at the old version of the page to see the ones which were removed. Regarding deleting the remove widgets, they are already deleted so there's no reason to delete them again.
I understand what you're explaining with adding another widget in place and it taking the properties of the previous one but I believe this is either a bug or some other custom code is causing problems because this shouldn't be "normal" behavior. This similar activity happens with editable text webparts on a page but the left over is due to not cleaning out the editable text area prior to removing the webpart from the template. The fix for this is to go to the General tab of the page and scroll all the way to the bottom, click the Edit regions and webparts button and in there you'll find the editable text and image webpart values associated with a page. I don't believe the widget values are available like this on a page though, you'd have to look through the database to find those.