Sadly, this is a common problem for those with “cute” service pokémon, as not many people understand the fact that touching service pokémon may distract them from the very real, very important job they need to be doing. Until able-bodied people fully understand what service pokémon do and what it takes to be one, this will continue to be an issue, I’m afraid.
The best that you can do is, if you have the energy to do so, educate people who try to approach by informing them that your sylveon needs to concentrate on his task and that touching him will interrupt his concentration. Some people might not even understand then, but if that’s the case, then it’s best to deescalate the situation as much as possible or remove yourself (and your sylveon) quickly.
It may also help to have a friend along on days when you don’t feel up to explaining your sylveon’s purpose if you can. That way, you may have someone who can offer the explanation for you—or at least protect you from the more persistent individuals.
I apologize that this isn’t quite as helpful as you may have hoped, anonymous. Unfortunately, sensitivity towards the disabled is a concept that isn’t quite as widespread as it should be yet, and there are a lot of people in the world who simply don’t know how dangerous it can be to touch or otherwise interrupt a service pokémon. For this reason, it can be a bit tricky because these same people may not understand what their actions may do, especially when pet pokémon or pokémon that are otherwise okay to touch are so commonplace most humans who see a sylveon automatically think it’s a pet and therefore fine to touch, rather than assuming they should ask for permission first (and then listen if and when their humans say no). By attempting to educate those who approach you and try to touch your sylveon, there’s a chance you can defuse the situation before it happens—or at least give yourself a reasonable excuse for removing yourself from the situation.
And yes, it is rather unreasonable to ask someone (or their companion) to educate when approached, but that’s quite possibly the safest or more peaceful means of dealing with someone who has actually stopped in front of you and is currently reaching for your sylveon. Simply confronting them will make the situation worse, and if your sylveon is wearing an obvious service pokémon vest, you’ve done the best you can to offer prior warning before they approach.
Best of luck, anonymous.