The rule tweaks could be ok. For some reason what came to mind when reading the title was some of Kazimierz Żegleń work around silk body armor from the early 1900's. Plenty of lore around silk stopping blades as well in earlier dates. A simple 1:1 change between medium and heavy armors with some variation of silk armor could be a rules light way to address it. If you have magic or alchemical processes in the setting that could be used to explain away the nature of the materials. For some reason after seeing this the image caught in my mind is a silk parasol being used as a shield. Practical and stylish!