Rashid was on a 48-hour long sit-in against Jammu and Kashmir government's decision to give domicile certificates to the West Pakistan refugees living in Jammu since 1947.
The MLA and a few of his supporters had spent the night under the open sky in sub-zero temperature despite the government's persuasion to end his protest.
Police on Saturday morning removed Rashid and his supporters from the pavement opposite the Fair View residence of the Chief Minister.
The state government has made it clear that the West Pakistan refugees have been given identity cards and not domicile certificates.
Despite the government's assurance, some mainstream politicians and the separatists have been opposing the move.