Many dogs are apparently fearful of men, even those dogs that seemingly have no reason to be. Perhaps the most common reason is that as a puppy he was not socialised with men so they are an unknown quantity. Men are generally bigger than women, have deeper voices and are often physically stronger, which may be threatening to a fearful dog. The pheromones given off by men are different to those of women, they also use different personal hygiene products and given that dogs have a very sensitive sense of smell, puppies are brought up by their mothers whose comforting smell of oestrogen remains hard-wired into their olfactory systems throughout life; their fathers tend to be absent during this period of nurturing.
Another possibility is that puppies tend to trigger maternal instincts in women; they naturally want to comfort puppies whereas men tend to be more assertive and want to be playmates.
On a positive note, if your dog is fearful of men, as a professionally qualified behaviourist Sue Gilmore will be able to help you resolve the problem so that your dog can be calm and relaxed around all visitors and when men approach outside the home.