To be fair to your GP, he isn’t a mental health specialist so he isn’t wrong in saying he can’t say your mentally fit, this is the reason why they refer people to psychiatrists etc for diagnosis.
They can do an assessment for depression/anxiety etc but I think it’s unfair for a GP to be asked to ‘sign off’ someone’s mental health. It’s a very vulnerable and precarious position for them to be in.