To be effective, the deer should get an appropriate dose. I don't think one could be certain that each deer would eat the correct amount. I would offer that continous low dose of dewormer could select for a strain of worms that is resistant or immune to the medication. Might not seem to be a huge issue, but if the parasite can also infect domestic animals, then we could have a problem. Something akin to antibiotic resistant strains of bacteria.
Ivermectin is a wonder drug. It has been used in humans, I have used it to worm dogs and horses, and it can be used for cattle (not sure about hogs). If I remember correctly, it somehow interupts the life cycle of the worm, interferring with the production of eggs or small worms. I'd hate to create the super worm that was Ivermectin tolerant.
Also, there is an ongoing study at the University of Iowa where they have found that the digestive tract has coexisted for so long with parasites that it has almost developed a symbiotic relationship. The deer have been plagued by worms for so long that if we worm them, maybe they will develop the human version of "irritable bowel syndrome", the disease where patients have been shown to benefit from worms.