According to the announcement made by Modi at the meeting of Council of Ministers here, the Task Force comprising experts will be set up over the next few days.
"The Task Force will be set up to prepare a comprehensive action plan for effective participation of Indian sports persons in the next three Olympic games 2020, 2024 and 2028," a PMO statement said.
"The Task Force will prepare overall strategy for sports facility, training, selection procedure and other related matters," it said.
It said the Task Force will comprise members who are in house experts as well as those from outside.
The next Olympic Games will be held in Tokyo in 2020.
The decision to set up the Task Force comes against the backdrop of India's dismal performance in Rio Olympics as the country won only one Silver and one Bronze even after sending its biggest contingent of 118 athletes.
India finished 67th in the medal tally