Does a non-magical sheild count toward your armslot? i.e. Can you wear a pair of magic bracers and then pick up a mundane sheild for the AC and ref bonus?
Yes, you can have a pair of magic bracers and a mundane shield (or vice versa, if you really wanted). The items slots being discussed are magic item slots. They have little bearing on how much you can physically wear.
PHB 224 - "You can benefit from only one magic item that you wear in your arms slot even if, practically speaking, you can wear bracers and carry a shield at the same time."