People refusing baked goods because they're vegan BLOW MY MIND. I think when they hear the word "vegan" their common sense (if they have any) shuts off and they instantly assume it must mean the stuff has something hidden and creepy in it -- fear of the unknown, really. When, in truth, it has no cholesterol, less/no fat, etc. I think, maybe, this is where culture plays a role -- I know my family back in Europe wouldn't bat an eye and dig in if I just told them it was "for lent" (even if it isn't lent), since they're used to avoiding animal products for that purpose. Another idea would be to just say "it's made without milk or eggs" -- which might confuse or put off some people anyway, but I think it's the word "vegan" that irrationally scares some people because they don't quite know what that entails. (Or they do and they're jerks. Their tastebuds' loss.)
As for me, I think my parents still believe it's a phase and it'll wear out (or at least that I'll settle into a vegetarian diet). My grandparents all take it a lot better, with only my grandpa occasionally asking about calcium and protein. It may partly be because they're much more used to the idea of lent, one time that religion works in my favor.