I would have checked the set. But youre right, one more Diamond would have been scary indeed. Even a 3d would have beaten you if he held one. Chances are very very slim heads up that your opp has the flush. But it really depends on who you are playing against. Generally speaking, if your opp was a bully, definately check and trap, you want him to raise for you. If not, and he is a timid player, play it straight up, maybe bet slightly less in the hopes he caught something or wants to hang on to his ace/king. But you know that already
On that board I think you played it right, you were unlucky to get a flush board like that. Too risky to lose a set with, better to take the small pot and move on.