[[PageOutline(2-4,Contents)]] = Announcer cryptographic capabilities == Why This page documents the effort to add optional encryption for AnnouncerPlugin email messages using GnuPG. See #6773 for the corresponding ticket for this enhancement and [wiki:CryptoPlugin] for the future of this development. Imagine, you're using Trac in a corporate environment, typically allowing external access to Trac, repositories etc. only after authorization or not at all. Still, you may wish to keep business partners, support customers, etc. informed about certain or all developments, and that involves sending potentially sensitive (security, privacy ...) information outside the tightly controlled corporate network. Co-workers in a home office setup may create a similar demand. Signing emails will help recipients to be sure that they got unaltered content and got it from you. Encrypting emails will help to distribute announcements to external recipients as freely as to recipients inside without concerns on discretion. == How (proposed implementation) It might help a lot to have a closer look at [t:wiki:TracDev/Announcer internal structure and event flow] of AnnouncerPlugin. Where to kick in is of course an essential decision. Since it's not only about mangle the message body (one of the earlier assumptions that all turned out to reach not far enough), there is not a single point but a bunch of changes needed to get cryptographic operations working. There are modifications required and code to be added to `announcer/distributors/mail.py`. A new file `announcer/util/mail_crypto` introduces a new class `CryptoTxt`, that provides the cryptographic operations. The approach is generic enough to be extended beyond OpenPGP, currently the only supported crypto standard and handled by GnuPG, to i.e. use X.509 certificates as well. ''Update:'' I've thought a lot about the aforementioned approach and found it far to narrow. So a major re-design is on the way now. Cryptographically operations will be completely moved into a dedicated stack of python scripts: CryptoPlugin, that provides universal, reusable methods for certificate and key management and beyond announcement email content. === !AnnouncerEmailEncryption To get !AnnouncerEmailEncryption up and working you'll have to take the following steps: 1. Install GnuPG on the same host along with Trac[[BR]] For Debian GNU/Linux a simple {{{#!sh $ apt-get install gnupg }}} will do. 2. Install `python-gnupg` Currently there is no Debian package available. Install from source of [https://bitbucket.org/vinay.sajip/python-gnupg project site] is preferred. So we get a mature and actively maintained Python interface to GnuPG. 3. Configure !AnnouncerEmailEncryption in the !`[announcer]` section of `trac.ini` for the given Trac environment ||'''available option''' ||'''value''' ||'''default''' ||'''note''' || ||rcpt_allow_regexp ||string ||`''` ||not strictly related to cryptography but a way to generally limit the possible scope for any announcement email ^1^|| ||rcpt_local_regexp ||string ||`''` ||whitelist email addresses excluded from crypto operations ^2^|| ||email_crypto ||'sign'|'encrypt'|'sign,encrypt' ||`''` ||crypto action selector || ||gpg_binary || ||`'gpg'` ||name, full path prepended optionally to allow for use of custom GnuPG installations in unusual locations or switch between several GnuPG versions installed || ||gpg_home || ||`''` ||something like `/gnupg` ^3^|| ||gpg_signing_key ||OpenPGP key by ID or fingerprint ||`None` ||key selector to set one of several available keys for signing operation || ^1^ e-mail address(es) or valid Python regex describing range of e-mail addresses[[BR]] ^2^ string or regex like before, to sort matching recipients, sending verbatim announcement in parallel with an encrypted version, if both resulting recipient lists are not empty[[BR]] ^3^ if not existent, directory will be created and populated with necessary (initially empty) files on next announcement with !AnnouncerEmailEncryption enabled 4. import public keys (to keyring file in specified `gpg_home` dir) and optionally associate with users * admin: * (mass-)upload from local pubkey(ring) files * upload from local or public keyserver * associate pubkey from previous upload with users User administration should allow for key import and association to users on behalf of every registered user. * user: * upload like admin, but associate only to himself/herself * select pubkey from previous upload (self or admin) Users may wish to add an OpenPGP key to their configuration. For simplicity I'd make existence of a pubkey equivalent to an "always encrypt msg for me with this key" option. For convenience it might still be possible to temporarily disable a key and re-enable it later without deletion and re-import as this is directly supported by GnuPG. 5. add an automatic signing key for the given Trac environment (optional) * upload from local secret key(ring) file or create it on-demand (in/into secret keyring file in specified `gpg_home` dir) * just use (only/last added) secret key, provide a drop-down or similar for selection, if more than one is available For convenience it should be possible to temporarily disable a key and re-enable it later without deleting it. Beware, that 4 and 5 is not fully covered by current development code. So this is subject to change a lot, before a public release. As with current code for AnnouncerPlugin there'll be DEBUG logging embedded into all operations mentioned above. === Q&A [FIXME: add more Q+A here to help with code design evaluation and code review] ?: Is it true that different users will have different keys? If so, we can add configuration to the user's preference page. We could have a big textbox for GPG key and if they have one entered, then use encryption. A: Yes, different users will (typically) have different keys. It might be desirable even to support multiple keys per user. Only in rare cases one key would be associated with different users/e-mail addresses, even if this might be technically perfectly valid and useful. But it indicates violation of the de-facto standard one-owner-per-key that abhor kind of group keys. Update: User preferences will be provided by CryptoPlugin. ?: Why not implement encryption as another IAnnouncementEmailDecorator A: Decorators are called without guaranteed order. Encryption needs control, that it'll be the last message body mangling action. ''We can change this pretty easily'' - '''doki_pen''' ?: Why not implement encryption as another IAnnouncementFormatter A: Encryption is not about encoding etc. ''Formatter is more about turning an event into a message, it shouldn't be done here.'' - '''doki_pen''' ? Doesn't smtplib or any other stock python library handle encryption? A: No. Pythons smtplib is dedicated to e-mail construction including MIME, but no PGP/MIME etc. (see http://docs.python.org/library/smtplib.html). Pythons nativ [http://docs.python.org/library/crypto.html crypto utils] currently consist of secure hash and checksum generators (md5, sha). ?: What are the explicitly handled exceptions? A: For readability let's try to put this into a table. ||exception ||cause ||action/behavior || ||missing pubkey ||fingerprint in user settings but no corresponding key in pubkeyring file ||delete recipient from recipient list of event in delivery, create new event with info "specified pubkey not in Tracs keyring" to be sent to this user and project admin || == Beyond current development scope There is a [t:wiki:TracDev/Proposals/Announcer proposal] to replace current Trac Notification system with AnnouncerPlugin. This will make the effort for a really clean solution even more urgent. Consider [https://subtrac.sara.nl/oss/email2trac/ticket/186 cryptography related features] for EmailtoTracScript ([https://gitlab.com/surfsara/email2trac current home]). It could be interesting for example to allow only e-mails with valid signature from known senders to pass, fighting spam at another level. === Development traces (history) This is kept for reference and personal attitude to preserve historical notes. 1. step: add some code to make encryption just work '''done'''[[BR]] * expect encryption/signing key ID hard-coded, some fixed values for variables I'd like to see as options in [annoucer] section of trac.ini and other ugliness 2. step: code evolution over time, i.e. a. put code into separate python script and import function into distributors/mail.py '''done'''[[BR]] * add new reference {{{ from announcer.util.mail_encrypt import encrypt_txt }}} * add ./announcerplugin_trunk/announcer/'''util/mail_encrypt.py''' containing new cryptographic functions b. consider invention of a new class, i.e. to allow for reusable code, gpg interface initialization before sign and encrypt actions would profit among others c. add a minimal set of new options to [annoucer] section of trac.ini and replace formerly fixed values to gain planned control about new cryptographic functions '''mandatory options''' (not needed for basic operation with common GnuPG installation ||option name ||default value ||note || ||email.encrypt ||disabled ||whole e-mail encryption disabled, to allow smooth upgrading of existing installations || ||email.sign ||disabled ||whole e-mail signing disabled || '''additional options''' ||option name ||default value ||note || ||gpg_binary ||gpg ||full path to binary, needed i.e. for custom GnuPG install or GnuPG v2 || ||gpg_home ||None ||keyring files dir, defaults to ~/.gpg (in home dir of user that is running trac) || ||email.encrypt_except ||None ||one or more e-mail-addresses or domain as comma-separated list, matching recipients that still get cleartext msg || ||email.sign_except ||None ||valid values as before, but matching recipients here still get msg without cryptographically signature || 3. step: extend web_ui of AnnouncerPlugin to remote-control new options from user and/or administration settings - ''canceled, as this is largely unrelated to this plugins core business'' == Resources (for ideas and code) * Python e-mail test server http://docs.djangoproject.com/en/1.1/topics/email/#topics-email * How Django, another Python based system handles e-mail-encryption with [http://code.google.com/p/django-email-extras/ django-email-extras] ([http://code.google.com/p/django-email-extras/source/browse/trunk/email_extras/ browse the code]) * MIME-capable Python mailer script is published at http://www.physics.drexel.edu/~wking/code/python/send_pgp_mime, this look very interesting, may help with a lot of things, but may not serve as a code base (incompatible license GPL2 vs. BSD-alike), not portable (consider Linux/Unix as well as Windows) - hasienda