It depends on what is the main subject in the picture...
And it's up to individuals. You can crop off the old lady on the left and get a visual feel of either without the walls on the right (if I cropped from bottom) or if I cropped from the left, then I may get less details of the house of the left as well as missed the statue.
anyway to be blunt to the threadstarter, it isn't actually a choice of preference. unless it is an item that is dynamic in air (e.g. a plane) or something that is not traditionally visualised from ground, all pictures will require a straight horizon because our sense of aesthetics are formed from how we visualise and are taught to visualise. and that is why we have 2 major perculiarities in conventional photography where we do not break the rules and be creative - horizon and skin tone.
and a correct horizon would be the one in the farthest distant (not necessarily what you can see) and will be supported by vertical components (unless distorted by an ultra-wide angle). and it will be the left that is correct. zac did a very good job to tilt it and illustrate it, although the bank can be further straightened. the reason why the shore is tilted is because you are facing the shoreline at an oblique angle, that is one part of the seashore is very near you and the further end is far from you. if you tilt even more, just imagine directly at the shoreline, eventually the shoreline would be at right angle to the horizon. just imagine you are at a bustop next to the road, the horizon will never change, but the road looks at different angle as you turn from facing front all the way to looking at the road to see if your bus is coming.
for someone who just see the photo, in a telezoom framing, you may lost your sense of perspective as the interception point of the lines are lost outside the frame. however for the shooter who is aware of the surrounding, can actually make it out better when he look outside the frame
(the link is down, not at home, will reupload another day)
for example, i'm diagonal to the river and both banks are oblique to me, but the sense of horizon can still be achieved.