Why not spend 2 nights? We get in on a Thursday night around 4pm typically. Get the rental car and have dinner at Coco Maya (going to try Chez Bamboo next time!). Then Friday we tour the island - hit the baths and hog heaven as well as any other place. Sometimes we get back early afternoon and go to saba rock. Then we have the BBQ and Jumbies at Leverick that night.
It's a great way to experience a lot more.