I have translated the plugin's additional field labels into German and found out that having an umlaut in a field's label makes the set permissions be ignored in the ticket changelog.
For example, in trac.ini I have set this to remove the hours field for users without TIME_VIEW permission:
[field settings]
...
hours.permission = TIME_VIEW:remove, TIME_RECORD:disable
[ticket-custom]
...
hours.label = Stunden hinzufügen
Now if somebody adds hours to a ticket, the label and value are displayed in the ticket changelog also for users without TIME_VIEW permission. If I replace the umlaut by ue ("Stunden hinzufuegen"), the field is correctly removed from the changelog.
The ticket form and reports are not affected as far as I can see, the field is correctly removed.
I don't know any Python, but I suppose that the reason lies in remove_changelog() in blackmagic.py.
Otherwise, permissions work great! Thank you.