All things being equal, the crown of the road is going to give you slight drift to the right. Some, including myself, have experimented with varying tire pressure between the left and right rear. I personally have not found that to be effective and quickly realized the miniscule amount of drift wasn't a real problem and now just ignore it.
If you want to ride for a couple of miles with your hands off the bars, that answer won't satisfy you, but if like me you normally have at least on hand on the bars at all times, it's really not an issue.
Pulling upon braking is a seperate issue and as ted pointed out, that is most likely a brake hanging up or unequal pressure. Might want to do a bleed as well.
Good luck!