It has a full ring-type USM autofocusing motor, instead of the inferior micro-USM motor in the 50 1.4. Faster, quieter and more accurate autofocus in 85mm and sharper image quality. Main downside is heavier purple fringing, though anyone buying this lens should have lightroom or its analogue, which removes fringing with one click.
I think the more significant difference between the two is the focal length however. If you're primarily shooting inside and prefer to shoot full body portraits, the 50mm will be more suited to that. If you're shooting outside or inside in a large space, or prefer tight portraits, the 85mm is better in my opinion. This is assuming you're using crop-body cameras (e.g. Rebel T4i, 60D, 7D), and not full frame.
In fairness to the 50mm, if you have a hood on it all the time, and it falls, it won't strip the USM motor because the hood will take the blow. But if the focus is set to 4ft (let's say), there is a portion of the lens that sticks out (I do not know the technical term) and if that is hit, it will move the motor out of alignment. I constantly keep the hood on my 50mm, and honestly... it scares me everytime I use the thing... so I'm not a big fan of the build quality of the 50mm, but if you treat it right, (and always buy that lens new), then you shouldn't have any issues.
Having a great time and visiting family are mutually exclusive.