That’s what I’d do.Thinking about it, I would be inclined to house rule that retrieving an item from the Haversack is your free object interaction rather than a full action. That would fit the "wanted item on top" flavor and would also be a solid reason to prefer the Haversack.
The Haversack is less useful than the Bag of Holding. According to the DMG, the rarity of an item is correlated to the making time, so the Haversack, by RAW, takes 10 times the resources necessary to make a Bag. If it takes more (or the same) effort to create a Haversack than a Bag of Holding, and is less useful than said Bag of Holding, item makers wisely and smartly made more Bags than Haversack. So rarity (the game statistics) correlates greatly with availability.Why is the Handy Haversack a rare magic item in 5e? It takes the same action to retrieve an item from the Bag of Holding, which is an uncommon magic item, as it does from the Haversack and the Bag holds a lot more.