A Dill of a Pickle

I seem to have gotten myself into a situation with the former owners of BoBo. I let them visit him once a month at my house because they are old and they miss him...and I am a sucker. I am at my wits end now though. They came up a few days ago to take him for a walk. I didn't let them in like I usually do and sit and talk for a while. This time I was all business. I gave them his leash and Sam and I took the "A" Team for a run. We got back and they were sitting outside of the fence with him. They instantly started complaining about BoBo being hot and not having access to water. I told them that if they would have just let him into HIS yard he would have gone through the dog door and drank from HIS water bowl. Blah, blah, blah...I got a message from them later saying they were concerned about him and if I couldn't keep him they would take him back. I have not responded yet because as we all know I have a hard time with balance. I don't want to say too little and have them continue coming up, but I want to make it clear that they chose to get rid of their dog and that they really are not welcome to come up any more. They have put me down in some way every time they come up. My neighbor was even making fun of it yest. He heard them asking if he ever goes to the vet. Please help yo.