A favor is just that...a favor. This person even referred to it as a favor no??? I don't know how expensive this favor was and/or if it would be worth it to "pay" in order to keep this person's business?????? Perhaps you could approach this person and tell him exactly how you feel (in a business like way of course :)). Maybe he'll understand. If not, then I suppose you need to decide is he worth it or not???? Personally, I don't think I'd do business with him again, but then, I don't know the specifics as you do.
Honesty is usually the best policy. Good luck!!!
TIS