The problem is, if you _don't_ act on the invitation, the app still insists on showing updates for the group, so an active group is constantly there (you just can't see *what* has been posted).
So, they get a constant reminder of whatever it is that means they don't want to join