If your temperature gauge has been reading in the normal range, and there is no indication that the engine has been overheating; it should be OK to drive to the store to buy coolant. The container you can see is only a reservior; most of the coolant is in the radiator and the engine.
But please bear in mind that pure coolant should not be poured into the reservoir. It should first be mixed with equal amounts of distilled (not de-ionized) water. You can also buy coolant that is labeled "pre-mixed" which already has distilled water added; but it costs more than buying an equal quantity of distilled water and coolant, and mixing them yourself.
If the reservoir is empty, you should remove the radiator cap (when the engine is cold), and pour mixed coolant directly into the radiator opening until the level is all the way to the top. Then install the radiator cap, and pour mixed coolant into the reservoir, until the level is up to the full line.
The level in the reservoir may drop some, in the first few days after refilling the system; so keep refilling the reservoir and the radiator, until the level stabilizes. If the coolant level keeps going down after the first few days; there probably is an external or internal leak; which should be checked for by a competent mechanic. Unrepaired coolant leaks can lead to far more serious problems.