Spanish and standing too close

Throughout the day I was subjected to people who stand too close to me.  This was happening when I was in line for things.  People will stand so close to you that they are touching you.  It annoyed me.  I kept moving but it kept happening again and again.  I never said anything about it.  I also noticed how frequently you get bumped walking the streets in Madrid.  There is no mention of sorry or con permiso as I believe they should say.  Just a minor annoyance here.