* Get rid of the django_markwhat dependency, and implement our own
classes to get more control. In passing also remove django-markdown,
because we never used that.
* Instead of trying to clean markdown with regexps, use the bleach
library (NEW DEPENDENCY) with special whitelisting of allowed tags
based off standard markdown. This means that one can input links or
formatting in HTML if one prefers, as long as it renders to the same
subset of tags that markdown allows.
* Replace javascript based client side preview with an actual call to a
preview URL that renders the exact result using the same function,
since the use of showdown on the client was increasingly starting to
differ from the server, and since that cannot be kept secure the same
way. Rewrite the client side javascript to work better with the now
longer interval between updates of the preview.
Long in planning, but never got around to it.
Suggestion to use bleach for escaping from David Fetter.
When a news article is approved, it gets delivered as an email to the
pgsql-announce mailinglist. It will render the markdown of the news
article into a HTML part of the email, and include the markdown raw as
the text part (for those unable or unwilling to read html mail).
For each organisation, a mail template can be specified. Initially only
two templates are supported, one "default" and one "pgproject" which is
for official project news. The intention is *not* to provide generic
templates, but we may want to extend this to certain related projects in
the future *maybe* (such as regional NPOs).
These templates are stored in templates/news/mail/*.html, and for each
template *all* images found in templates/news/mail/img.<template>/ will
be attached to the email. "Conditional image inclusion" currently not
supported.
To do CSS inlining on top of the markdown output, module pynliner is now
required (available in the python3-pynliner package on Debian).
A testing script is added as news_send_email.py in order to easier test
out templates. This is *not* intended for production sending, so it will
for example send unmoderated news. By sending, it adds it to the
outgoing mailqueue in the system, so unless the cronjob is set up to
send, nothing will happen until that is run manually.
Support is included for tagged delivery using pglister, by directly
mapping NewsTags to pglister tags.
While at it, update the moderation preview forms to preview news items
using the HTML template for the email (while leaving other types of
items previewing without a particular stylesheet).
This includes a number of new features:
* Move some moderation functionality into shared places, so we don't
keep re-inventing the wheel.
* Implement three-state moderation, where the submitter can edit their
item and then explicitly say "i'm done, please moderate this now".
This is currently only implemented for News, but done in a reusable
way.
* Move moderation workflow to it's own set of URLs instead of
overloading it on the general admin interface. Admin interface remains
for editing things, but these are now separated out into separate
things.
* Do proper stylesheet clearing for moderation of markdown fields, using
a dynamic sandboxed iframe, so it's not ruined by the /admin/ css.
* Move moderation email notification into dedicated moderation code,
thereby simplifying the admin subclassing we did which was in some
places quite fragile.
* Reset date of news postings to the date of their approval, when
approved. This avoids some annoying ordering issues.
