It is part of the etiquette of those listening to the Khutbah of the Imaam on Jumu'ah to fix their gazes at the person of Imaam while he delivers the khutbah.
Al-Imaam Ibn Qayyim al-Jawziyyah - rahmatullaah alayhi - said in Zaad al-Mi'aad, 1/430:
'It was from the guidance of the Prophet - sallallaahu alahyi wa sallam - and his Companions regarding the Khutbah of Jumu'ah, that whenever he delivered the Khutbah standing, his Companions would turn their faces to him; his face would become their qiblah as the Khutbah went on.'
Some people would rest on the masjid walls thereby facing directions other than that of the Imaam; this is not correct.
The wisdom behind facing the Imaam is that it brings more concentration from the listener and it depicts him as a person of good etiquette.
Al-Imaam As-Sana'aanee - rahmatullaah alahyi - said in Subulus-Salaam, 2/82:
'That the listeners face the khateeb is an established practice (from the time of the Companions), in fact, we ca say there is a consensus as to its desirability...'
Adapted from: Al-Qaulul Mubeen of Ash-Shaykh Mash'hoor Hasan Aal Salmaan - hafidhahullaahu. Pp.354 - 356.
In the past edition: