The forecasts above should give you warning many hours before any swell coming from hundreds if not thousands of miles away. You also can give yourself greater buffer from real harm and risk of damage by never putting yourself as the inside row of boats. Deeper water provides a buffer, distance also provides you a buffer against a sudden change to a lee shore.
I have never been in CGB when a sudden set of unsafe swell or surf showed up. I was in Salt Whistle Bay , Mayreau when the swell and surf showed up instantly.
Most of the cats closest to the shore in shallow water ended up on the beach in single digit minutes. Middle of the afternoon! Several boats with the crew on the boats were incapable of saving themselves. The swell simply came in and put the boats on the beach. The boats in the middle of the bay where safe. I have been in White Bay shortly after a sudden storm where a boat was up on the rocks or beach.
If any anchorage is wide open. You should always asked yourself why? If you see a sandy beach. Surf threw that sand up there. Someday the wind and waves will come along a throw a boat up there just like the sand.