Just brush off kid related invites for a while. they will either get the pic or stop asking because you always say no and their kid will find an alternate to play with anyway. At that point, either the relationship gets weird or it doesn't.
If it is not their first kid, just tell them that the two kids don't get along well and they prob will understand. If it's their first, they won't understand and it will be weird coworker/ awkward friends time.