Anas (May Allah be pleased with him) reported: Messenger of Allah (PBUH) permitted Zubair and `Abdur-Rahman bin `Auf (May Allah be pleased with them) to wear silk because they were suffering from an itch.[Al-Bukhari and Muslim].
This Hadith tells us that, if needed, even men can wear clothes made from silk. For example, it may be suggested in case of a skin disease like itch. A man may also be allowed to wear a silk dress in order to escape extremely cold or hot weather provided he has no other clothes.