Housing Minister Kit Malthouse has become the second MP to pull out of the Conservative leadership race.
He said it had become clear in the last few days there was an “appetite for this contest to be over quickly”.
Earlier, Brexit Minister James Cleverly became the first to withdraw from the contest, less than a week after declaring his intention to stand.
It leaves 11 Tory MPs competing for the top job. Theresa May stands down as leader on Friday.
She will remain PM until a successor is named by the week beginning 22 July.
The Conservative Party board has agreed to change the rules for the leadership election to speed up the process.
Mr Malthouse said he entered the contest “believing that I could make a real difference in delivering a Brexit that would command the support of the House of Commons”.
But he said his experience in politics had made him a “realist”, and there was a desire “for the nation to have a new leader in place as soon as possible”.
His name was given to the so-called Malthouse Compromise – a proposal drawn up by backbenchers from Leave and Remain wings of the Tory Party, which would have implemented Mrs May’s Brexit deal with the controversial Irish backstop replaced by “alternative arrangements”.
In a statement, Mr Cleverly said it had “become clear” it was “highly unlikely” he would progress to the final two candidates that will appear on the ballot paper.
“Unfortunately and with a heavy heart I’ve decided to withdraw from the race.”
He said he had hoped to be “the face and voice of” of a new conversation within the party and had asked colleagues “to make a leap of faith, skip a generation and vote for a relatively new MP”.
“It is clear that despite much support, particularly from our party’s grassroots, MPs weren’t comfortable with such a move and it has become clear that it is highly unlikely that I would progress to the final two candidates.”
Who will replace Theresa May?
The winner of the contest to lead the Conservative Party will become the next prime minister.
Under new rules for the contest, candidates must now gain the backing of eight other colleagues by the week commencing 10 June if they want to stand.
Candidates would have needed only two MPs supporting them to put themselves forward under the previous rules.
After nominations close, all 313 Conservative MPs will vote for their preferred candidate in a series of polls that will whittle down the contenders one by one until only two are left.
Due to another rule change, candidates will need to win 5% of votes (16 MPs) in the first ballot and 10% (32 MPs) in the second to proceed.
If all the candidates exceed this threshold, the person with the fewest votes will be eliminated, a process that will continue in subsequent rounds until only two remain.
The wider Tory membership of 124,000 will then choose between the final two candidates.