The 36-year-old, who won his previous gold in the 2004 Athens Games, bettered his own world record to finish on top in the men's F46 event.
Devendra, whose previous best was 62.15 metres (achieved in the 2004 Games), improved the mark with an attempt of 63.97 metres at the Olympic Stadium (Engenhao).
Another Indian Rinku Hooda, who also competed in the event, finished fifth with a personal best of 54.39 metres in six attempts, while Sundar Singh Gurjar didn't start the event.
The Rajasthan born athlete had lost his left hand when he was electrocuted while climbing a tree as an eight-year-old.
But he continued to follow his dream and went on to achieve the Arjuna award in 2004 and the Padma Shri in 2012, becoming the first Paralympian to receive the honour.
Devendra, who won the gold at the International Paralympic Committee's (IPC) Athletics World Championships in Lyon in 2013, last participated in the Paralympics 12 years ago as the F46 event did not feature in the 2008 and the 2012 editions.
Currently ranked third in the world, Devendra swelled India's medal tally at Rio to four medals -- two golds, one silver and a bronze.
Thangavelu Mariyappan had earlier struck gold in the men's high jump, while Varun Bhati had secured a bronze in the same event.
Later, Deepa Malik won a silver in the women's shot put event to add to the tally.
Accolades started to pour in for Devendra as from Prime Minister Narendra Modi to cricketers such as Virendra Sehwag and Mohammad Kaif congratulated the athlete for his stupendous feat at Rio.
"Congratulations to Devendra Jhajharia for the historic and well-deserved Gold at the #Paralympics. We are very proud of him. #Rio2016," Modi wrote in a tweet.
Sehwag wrote: "#Paralympics is pyaara-lympics. Bow down to #DevendraJhajharia fr d Gold&new World Record.He won Gold in2004as well."
"Congrats #DevendraJhajharia fr the Gold& new World Record.Truly inspiring,wishing fr more coverage than ur 2004 gold," Kaif wrote in his twitter handle.
2004 Athen Olympics silver medallist shooter, Rajyavardhan Rathore also wrote: "Congratulations #DevendraJhajharia Your #Gold @ #Paralympics is inspiring to many. Kudos to your efforts ?? #Rio2016."
India's Olympic gold medallist shooter, Abhinav Bindra also heaped praise on Devendra. "Many many congratulations to Devendra Jhajharia. You inspire us all."
Tennis star Rohan Bopanna tweeted: "Congrats Devendra Jhajharia for breaking your own world record &making the country proud.