Few guys looking for some time away from every day life. Since we are going without families, probably gonna have to keep it brief. Is it worth a 3-4 day charter in BVI, or should we stick with USVI to avoid ferry hassles? Is it worth it at all for that short a period? Any good itinerary ideas?

For that short trip I would charter from CYOA and explore the anchorages around STJ.

You could avoid the ferry hassle and charter from Virgin Island Yacht Charters in Compass Point Marina (east end of St Thomas near Red Hook) and easily do Great Harbor and White Bay on JVD, Cane Garden Bay, Norman Island and stop on the way back in Christmas Cove for pizza at Pizza Pi

Most charter companies have a 5 day minimum. 5 days is not ideal but we have done it many times. The ferries are only a hassle if they don't mesh with your flights. The USVI and the BVI are different experiences. Both are beautiful. For a guy trip there would be more to do at night in the BVI even post hurricane.