It's difficult to say for certain because the sample image is quite small but it does as look as though part of the background is showing through the hat. If you used the Tolerance slider you may have allowed some bleeding into the hat.

The best way to try and deal with this would be to open Background Mask Editing again and use the Eraser - Decrease tool (see below) to go around the inside of the hat in the problem area. This will make sure no transparency exists and should solve the problem.