He was a judge in the Supreme Court of India. He assumed charge of this post on 9 April 2008 and retired on 17 September 2011.

Sharma was enrolled as an advocate on 16 June 1970. He practised in the Gauhati High Court in constitutional, service, civil and taxation matters, etc. He also worked as panel and government lawyer for the states of Assam and Manipur and also railways. He was the standing counsel for state of Mizoram from 1981 to 1988 and advocate general of the same State from 28 April 1988 to 7 February 1989. He was designated as senior advocate on 1 January 1988. Worked as Advocate General for state of Nagaland from April 1992 to January 1994. He was a part-time lecturer in law in Gauhati University law college from 1979 to 1992.

He was appointed as permanent judge of the Gauhati High Court from 10 January 1994. Was transferred to Patna High Court and joined on 14 February 1994. Was then transferred to Delhi High Court on 12 December 1994. He was elevated as acting Chief Justice of Delhi High Court effective 28 November 2006.