That same black family was on the show in a later episode when George realized (I can't remember why) that he had no black friends so he was desperately trying to find one, including going back to that families house and asking to hang out with them.
You are referring to when George made an innocent comment to a black co-worker, saying that he looked a lot like boxer Sugar Ray Leonard. To which the co-worker responded, "I bet we all look alike to you, Constanza." From there, George gets overly sensitive and then just basically is walking up to blacks on the street trying to make a friend.
Jerry had a great retort to George's dilemma. "You don't have any black friends. Outside of us, you don't have any white friends either."
