Yes, go back, return the car and give them an hour to refund your deposit. If they balk at refunding your deposit in full, don't threaten or bluff, but go directly to small claims court and fill out the papers for a small claims suit against them. If they don't refund your deposit quickly, then they'll pay up a day before the court date.
You should not be financing a 10 year old car. Save your cash and buy outright. The only lenders who will finance a car this old will have such poor terms for you that you should not consider such a deal.