Ezekiel 44:18

The priests wore clothes round their bodies, underneath their main clothes. It would be wrong if people saw their bodies. But even the clothes that other people did not see were linen.

Of course, a person sweats (becomes damp) when he is hot. And the priests would become hot as they worked. Unlike wool, linen is a cool material. So this material would help the priest not to sweat much.