There's a really pleasant atmosphere in the streets of the old town of Galle. The architecture has that really interesting mix of European colonialism and South Asia. What I liked was that, although there are quite a few hotels and restaurants, not all of the Old Town felt touristy. There are still quite a few people living here and it has an authentic community.
There are great views along the coastline and it's worth walking the old fortifications to see that side of things.