In Calcutta I confront this question every time my cab stops. Put aside the usual debate about selfishness and altruism, assume you will give something away. To whom should it go? Why not give it to the gentleman sleeping under a piece of cardboard who is poor but not begging?
You might think that begging is an overcrowded occupation subject to congestion. Perhaps the presence of many beggars causes many potential donors to ignore beggars altogether. In that case giving to beggars will encourage too much entry into the activity. Better to give elsewhere. This conclusion is overturned, of course, if a larger network of beggars is somehow good for all the beggars concerned.
Alternatively, what if begging is a constant cost activity? In that case donations induce more entry into begging, bringing the return back down to the (constant) cost of begging. You might create a transitory gain for some beggars, but the "bidding back down" effect is weaker for the gentleman sleeping on the street. The elasticity of street-sleeping with regard to donations is currently very low, probably zero. No one sleeps on the street in the hope of receiving a tourist donation.
Along related lines, the more you give to beggars, the harder beggars will try. This leads to what economists call "rent exhaustion," which again limits the net gain to beggars.
In short, you wish to give to the relatively inelastic factors. This usually rules out beggars, who respond to incentives rather quickly. If you are going to give, pick the poor person who is expecting it least.