Ghosts are part of the ethereal plane not the shadowfell. While ghosts can be found in the Shadowfell, they can also be found anywhere the ethereal plane touches including the Material plane. There is no reason to assume the Material Plane Echos are unsubstantial and not made of matter. They are part of the inner area after all.
But I also disagree with you on the premise that if matter of some sort like say a rock gets plane shifted by magic to one of the outer planes it ceases to be a rock and becomes a thought construct of a rock. It just makes more sense that the rock is still just a rock and magic has allowed it to be in a place it normally would not be.
The astral "dominions" appear to be the same as various "outer planes". They are part of the astral plane.
The Players Handbook describes outer planes as follows. Importantly, there is no "distance" or space for a material rock to exist there. Rocks take up space, and matter takes up space, but there is no actual space. Things are as far or near as the conceptual, semiotic, similarity. The aster is a mind scape, not a physical scape.
"
... The Outer Planes provide direction , thought, and purpose for construction [of the multiverse]. ... When discussing anything ... , the language used must be highly
metaphorical. ... Homes are
not literally "places" at all, but exemplify the
idea that the Outer Planes are realms of
thought and spirit. ... One can
imagine the perceptible part of the Outer Planes as a sort of border region, while extensive spiritual regions lie
beyond ordinary sensory experience. Even in those perceptible regions,
appearances can be deceptive. Initially, many of the Outer Planes appear hospitable and familiar to natives of the Material Plane. But the landscape
can change at the whims of the powerful forces that live on the Outer Planes. The
desires of the mighty forces that dwell on these planes can
remake them completely, effectively erasing and
rebuilding existence itself to bettcr fulfill
their own needs.
Distance is a virtually meaningless concept on the Outer Planes. The perceptible regions of the planes often seem quite small, but they can also stretch on to what seems like infinity.
"
Among the astral dominions, there is no distance, there is no space, there is no "place". It is a "thought" scape. There cannot be matter. A rock of material matter takes up space thus cannot exist there. Only thoughts can exist there.