not sure about the $2 shop, but over here in Melbourne I've bought batteries from a chain of shops called The Reject Shop. no problems so far. electrical plugs like extension boards and piggy backs, no problems so far as well.
Batteries have a shelf life. Just like how some rechargeables lose their full charge when it has not been used for awhile. So for the cheap Energizers, they're probably old stock and have been kept for a long time and starting to lose some/most of their charge, but if you are lucky you will get one which still has lots of charge. Else it will kaput on you very quickly. See your luck.
I had the experience of buying from those "$2 shops" a set of energizer (or was it duracell) batteries. Based on the expiry date on the pack, the batteries were good for at least a few more years but each and every one from the pack was flat. After that incident I stopped buying batteries from them. I believe they might have either gotten them cheap because they are either rejects or fakes.
the last time i bought battery from $2 shop, the battery leaking and damage my electronic stuff. From that onwards, I stop buying battery from that kind of shop. Rather spend extra on battery than losing more expensive equipment.