When I was living in France, I bought an annual train pass for the local lines, and on days when I didn't have other plans I would go to the nearest train station, get on whatever train was leaving soonest, and ride it to the end of the line.
This is how I discovered Provins.
I visited in November or December, when no tourists were around, and in the quiet and solitude I found it easy to imagine the medieval town that once was there. Unlike many busier attractions that limit visitors' movements, at Provins you can walk all over the ramparts and follow the underground tunnels once used by secret societies. It's a fascinating step back into the past.