Set aside several hours to visit Toitu Otago Settlers Museum - this place is full of interesting information about the early days of Otago, and is a really fun way to spend your time in Dunedin. Be sure to start or finish at the licensed café, which serves light lunches, treats, and snacks.
Founded 1898, there are many historical exhibits following the city's heritage from the indigenous Maori people, to Scottish emigrants, and even the Chinese gold miners. My children especially enjoyed exploring the interactive exhibits, such as the replica grass hut, the mud brick house and dressing up below deck on the passenger ship (see photo below).
Toitu Otago Settlers Museum occupies the former New Zealand Railways Transport Building, and therefore houses a significant collection of transportation vehicles collected over the years. We had fun riding the Penny Farthing bicycle, seeing the horse and cart, and hopping aboard a real trolley bus and tram.
In the Edwardian Gallery, I recommend taking the time to read the amazing stories of the early settlers and the incredible hardships they endured. There are also interesting fashion and music exhibits within the museum, which may feel like a trip down memory lane for some.