Palm Beach is the northern beach in the Sydney area - almost 1h by bus from downtown. Therefore, it is a perfect destination for a day trip. There are activities that would suit all ages, and not all of them involve swimming on the ocean.
One of this activities is a walk that would lead you to the top of the hill where the Barranjoey Lightouse stands since 1881. There are different walks starting at different points of Palm Beach that would lead you to the lighthouse, but some of them can be as long as 4 hours.
If you want to do this excursion with children or aged persons, you might like to do a shorter route which would allow you to see nice landscapes as well. If you get to Palm Beach by public transport, you can start this walk from Ocean Road, the place where the bus will drop you. If not, you can park anywhere around this point and start to walk.
From here you will be doing a 45-minute easy walk, following the directions that would lead you to the top of the Barranjoey Lighthouse. Please remember that you would be walking in a National Park, so you must respect all the plants and wildlife.
The views from the top of the hills are extraordinary, so maybe you might want to bring a refreshment with you that you can enjoy once you get there. If you are lucky enough to do this excursion in the winter months, you might be able to see whales!