The attachment authorization control could be implemented with the help of authorization scope.
When you create an attachment via the Attachment button provided in attachment assignment block, you could also assign a given authorization scope for it.
Once the scope is maintained and the attachment is saved, it could never be changed any more.
