The situation is as follows:
The character is punished, for example, for 6 hours, but he can perfectly impose silence from the store on others. It would be logical to limit this possibility

i don't see the logic :

Exclusively game logic: if you are silent, you cannot read the scroll with the spell.