This perennial topic again. It occurs to be that one of the potential causes of this problem is the alingment of the"swinging" cross member at the rear of the trailer. The pivots on which this swings are held onto the box section longitudinal girders of the trailer by the inevitable U bolts. If the line between the centres of these pivots is not at right angles to the centre line of the trailer, this alone will make the boat track one way or the other as it is winched in.
So, before taking a hammer to the roller sets on this swinging cross member, check the alignment.
CLH are a bit agricultural - I'd be surprised if alignment jigs exist for these items (see my previous posts on the alignment of axles), Perhaps Matt should add this comment to his note on the matter.