Yes,a car will be useful for Little Bay. Restaurants on site are OK for beach lunches but in the past have not been great for dinners. I understand they have made some changes but haven't got any personal experience yet to report. But you wouldn't ever want to eat there for all meals, given all the wonderful options on the island.
Thrifty has a rental car location right at Divi, so depending on your plans you could rent a car there. That could be a good idea for example if you are arriving later in the day since you're a first timer. We've used Thrifty there and the cars were fine and rate seemed comparable to some others we've used.
.
