You should be as far as required to fit the head and shoulders into the frame. which is actually very far... since 135mm on 1.6x crop give you 216mm FOV.... For 1.6x crop bodies, it is wiser to use the 85mm.
This is from more formal sources:
35mm FF body:
a 105mm lens gives between 4-6ft working distance for a 15" headshot, and about 7ft working distance for a 24" head and shoulders shot.
A 135mm lens gives 5-9ft working distance for a 15" headshot and 9ft working distance for a 24" head and shoulder shot.
A 85mm lens is not recommended for anything less that half-body shots.