A dwarf wearing heavy armor and carrying a heavy load still suffers all of the other penalties, although his speed is unaffected.
PHB pg 161
Heavy load - +1 Dex bonus to AC, -6 armor check penalty
• Dwarf base land speed is 20 feet. However, dwarves can move at this speed even when wearing medium or heavy armor or when carrying a medium or heavy load (unlike other creatures, whose speed is reduced in such situations).
PHB pg 161
Heavy load - +1 Dex bonus to AC, -6 armor check penalty