As stated above, it's all about what makes the hunter happy. If you would be proud of the animal then by all means take him and worry not what others would have done.
For me personally I am looking to pass all known 3.5 year old bucks. I guess I have just been incredibly lucky that last two bucks that fit that description, and were in bow range with clear shots, were last years 169" and Saturdays 170".
I will also agree with what others have said about the ability of these deer to survive even with pressure on smaller tracts of land. There isn't an animal that amazes me more than the elusive whitetail.
I really don't agree with the idea that, "if I pass him, then the neighbor will shoot him anyway". The only thing guaranteed if you shoot him is that he has a 0% chance of survival. I bring this up because this is my uncles mentality on our family farm in MN and it eats at me...but again, if the animal makes him happy than I will be happy. If it's just to kill so the neighbors don't kill, I hold my disappointment internally.