Israel is to pay $1.1m to the family of an alleged Mossad spy known as "Prisoner X" who killed himself in prison in 2010.