Bismillaah-ir-Rahmaan-ir-Raheem
The Messenger of Allaah - sallallaahu alahyi wa sallam - described playing with pebbles and other similar things during the Khutbah of the Imaam as a foul act as he - sallallaahu alahyi wa sallam - described talking during the Khutbah.
Al-Imaam Muslim, Aboo Daawud, At-Tirmidhee and others recorded in their books that the Messenger of Allaah - sallallaahu alahyi wa sallam - said:
من مس الحصى فقد لغا
'Whoever plays with pebbles has fouled.'
Ash-Shaykh Mash'hoor Hasan Salmaan - hafidhahullaahu - said: This is because it takes the heart away from concentration. Playing with subhah (prayer rosary), keys and what is similar also follows the ruling.
Source: Al-Qaul Al-Mubeen p.356-357
In the past edition: