Many people, like myself, develop lactose intolerance when they get older or stop eating dairy/reduce their dairy intake. As I understand it, the bacteria in the colon that produce lactase (the enzyme that breaks down lactose) die off when they have no food, so when you add dairy back into your diet after eating very little to no dairy for a while, it can be more than your body can handle. If you really want to continue eating dairy, you could try Lactaid dairy products or have your doctor prescribe you a lactase enzyme pill. They have over-the-counter lactase enzyme pills too, but I've found they're not as effective as the prescriptions.
If you want to find out if it's really dairy causing you problems, you can do an elimination diet. Completely delete dairy from your diet for a couple of weeks and then add it back in. If you wanted to find out faster, you could have your doctor do a test.
OT--Reducing animal products in your diet is a great thing, but if you still include fish, even occasionally, calling yourself a vegetarian isn't really accurate. You'd be a pescatarian. Vegetarians, don't eat animals or meat and fish is still meat and an animal.
You could also say you're transitioning to be a vegetarian. Don't get me wrong, you're doing a great thing, but it's just something to consider. It confuses the omnis when those who eat fish or chicken still call themselves vegetarians because those of us who eschew all animal flesh get served or offered fish or chicken (it's happened to me more times than I can conveniently count).