Nah, some monsters can go shopping or know people, or horse trade. Yes, some monsters are reclusive hermits, but not all.
Low magic games, you limit your options on how to challenge the PC's. At high levels it will severely hurt the DM, depending on the PC's builds.
Average to High magic...