Carl Sargeant is facing a Welsh Labour investigation after a number of incidents with women came to light, First Minister Carwyn Jones has alleged.
The Alyn and Deeside AM lost his senior cabinet role last Friday.
Mr Sargeant has called for an “urgent” investigation to clear his name.
He said Friday that the allegations about his conduct had not been disclosed to him and had agreed with Mr Jones to step aside.
The former communities secretary said he looked forward to return to government. He has been asked for comment about Mr Jones’ comments, made on Monday.
Reacting to separate allegations made at the weekend about harassment in politics, Mr Jones said it was hugely important for people to come forward.
Referring to Mr Sargeant, the first minister said he had become “aware of a number of incidents at the beginning of last week”.
“I asked my office to speak to those women involved who had provided detail of those incidents. As a result of those conversations I felt I had no choice but to refer the matter to the party.”
The first minister said the women spoke to his office on the basis of confidentiality.
He said that, to his knowledge, Welsh Labour had “never” kept allegations secret to keep politicians in line.
“It is said to be something that happens in Westminster. It doesn’t happen with us,” Mr Jones said.
“If we have somebody who is in a position where they could embarrass the party or bring the party into disrepute, then action is taken. It is not held in the back pocket to threaten them if they decided they are not going to vote a certain way.
“It is said that that happens in Westminster. It’s never happened to my knowledge in the Bay, ever.”
Former Welsh government advisor Cathy Owens said on Sunday that harassment of women in Welsh politics had gone on for some years.
She told the BBC’s Sunday Politics Wales: “We are talking about a small number of men who have used their positions and are sexual predators – they have used their position in politics knowing that the parties will protect them.”
But the first minister said on Monday that nobody in Welsh Labour had been “in any way protected”.
“I’ve got no reason to doubt what Cathy has said,” Mr Jones said.
He said it was “difficult of course when you don’t know who these people are”, and added that it was “hugely important for people to come forward in order for these things to be investigated”.
The first minister said he had never been aware of the incidents Ms Owens referred to in her interview.
“What I can say is that I’m not aware of anybody in my own party who has in some way been protected. That really hasn’t happened,” he said.
Mr Jones said it was difficult for him to say if there was a cultural problem at the assembly because “as first minister, I’m detached from it”.
He said an independent structure for dealing with allegations was something that needed to be looked at by Senedd party leaders, who are meeting on Tuesday to discuss the issue.
“What we can’t do is be in a position where are structures are seen as somehow weaker than those in Westminster”, Mr Jones said.