I recently deleted a lot of old coupon entries and moved several reviews over to my back up blog. During this process I HAD thought when I removed the blog entry, I was also removing the corresponding WordPress tags. Sadly, this is not the case. You must go into the tags section and delete them there. (Unless of course the tags are being used by other blog entry you wrote.)
So a word of warning, when you remove a blog article you will need to be sure to check the tags section located under POSTS.
After you click on TAGS click on COUNT which is on the far right, top of the line up. This will then make the tags that have zero entries be at the top. Better than scrolling through page after page of tags hunting for the zeros.
If you find the first page has a bunch of zeros, then you can quickly delete them all by clicking the square up by the NAME icon. Then selecting the BULK ACTIONS at the top left scroll down to DELETE, then click APPLY. There will be no confirmation. Once done, it can not be undone without work of course. You must repeat this process all over again because WordPress resets the page. This can be fun. If you by chance have entries scheduled, their tags will appear as zero so be careful! You could be removing tags you just set up.