I have carried with leather and kydex/plastic against my bare skin. My experience is that they will ALL start rubbing you the wrong way at the end of a long day. Brian is spot on. An undershirt is essential for the long term.
This is interesting to me, because I've carried (in kydex) for quite some time (including 18-hour days, long-distance car rides, situations with lots of walking, etc) without chafe issues at all. I've certainly had times when the holster dug into me, due to how I was sitting/crouching, but those were all temporary issues.
I can't recall a time when chafe was an issue. Huh.
(And no, in the summers I don't wear an undershirt under the holster.)