There are two important principles:
Firstly, Dummy must not take ANY part in the playing of the cards. Dummy may only touch a card AFTER the Declarer has asked them to ‘play’ it.
Whenever there is doubt, or disagreement, the Director should be called to adjudicate
Now to answer the specific question:
The defender was not correct in saying that the card Dummy played should be left in play. The Declarer should be able to decide, as meant in the rules.
However, if the Declarer leaves Dummy’s card on the table as is, then the Director MUST be called (there is always a possibility that Dummy is giving an indication to the Declarer as to how s/he thinks the cards ought to be played).
The Director may at his/her discretion then make an adjusted score at the end of the hand (usually with any benefit of doubt going to the ‘non-offending’ side – the defenders in this case.