I feel ya! I used to be quite an addict to baked goods, could eat a whole pack of cookies in one sitting, love a pastry, and anything cake. The secret is, you can really easily make vegan versions of EVERYTHING that you love. It takes a little more time, but this is part of the magic. You know everything that goes into it, you can smell it baking in the oven, and you can eat them while still warm. Mmmm..
Dairy actually contains small amounts of a chemical similar to morphine, so it is actually addictive. You might find that once you break the cycle, it's really easy to avoid in the future. This is certainly my experience.