They are almost always connected to the municipal water supply, and might have a filter and/or water softener between the device and the water meter to meet the standards of the beverage company (Coca-Cola is very particular about the water their vendors are allowed to use). It then gets mixed with CO2 gas in the feed line, then mixed with syrup at the nozzle right before it lands in the cup.
Given the number of beverages that a soda fountain dispenses in a single day, it would require a huge tank to have its own independent water supply. Like, hundreds or maybe even thousands of gallons