Domain of messages in pgweb's search is currently hardcoded as postgresql.org
in templates/search/listsearch.html. This patch allows to use another domain
for links to messages in archives in the search function.
Since pgarchives' domain can differ from pgweb's one, a new variable is defined
for that purpose instead of using SITE_ROOT.
* Get rid of the django_markwhat dependency, and implement our own
classes to get more control. In passing also remove django-markdown,
because we never used that.
* Instead of trying to clean markdown with regexps, use the bleach
library (NEW DEPENDENCY) with special whitelisting of allowed tags
based off standard markdown. This means that one can input links or
formatting in HTML if one prefers, as long as it renders to the same
subset of tags that markdown allows.
* Replace javascript based client side preview with an actual call to a
preview URL that renders the exact result using the same function,
since the use of showdown on the client was increasingly starting to
differ from the server, and since that cannot be kept secure the same
way. Rewrite the client side javascript to work better with the now
longer interval between updates of the preview.
Long in planning, but never got around to it.
Suggestion to use bleach for escaping from David Fetter.
When a news article is approved, it gets delivered as an email to the
pgsql-announce mailinglist. It will render the markdown of the news
article into a HTML part of the email, and include the markdown raw as
the text part (for those unable or unwilling to read html mail).
For each organisation, a mail template can be specified. Initially only
two templates are supported, one "default" and one "pgproject" which is
for official project news. The intention is *not* to provide generic
templates, but we may want to extend this to certain related projects in
the future *maybe* (such as regional NPOs).
These templates are stored in templates/news/mail/*.html, and for each
template *all* images found in templates/news/mail/img.<template>/ will
be attached to the email. "Conditional image inclusion" currently not
supported.
To do CSS inlining on top of the markdown output, module pynliner is now
required (available in the python3-pynliner package on Debian).
A testing script is added as news_send_email.py in order to easier test
out templates. This is *not* intended for production sending, so it will
for example send unmoderated news. By sending, it adds it to the
outgoing mailqueue in the system, so unless the cronjob is set up to
send, nothing will happen until that is run manually.
Support is included for tagged delivery using pglister, by directly
mapping NewsTags to pglister tags.
While at it, update the moderation preview forms to preview news items
using the HTML template for the email (while leaving other types of
items previewing without a particular stylesheet).
We already set DATETIME_FORMAT, but it makes sense to also set
DATE_FORMAT. In all our frontend templates we were already using an
explicit format, but this will help the /admin/ side.
This is the place to load them to make sure they always load. This means
we can also now remove a hardcoded special varnish purging in the CVE
crawler script, because all saves including those from cron jobs will
now fire the signals and thus the automatic varnish purges.
This adds an xkey header to all outgoing requests with the hash of the
name of all templates loaded. In the future we will be able to use this
to purge "all pages that included a specific template", regardless of
where in the hierarchy it was loaded.
Do this by faking a template loader that never finds anything -- but it
will record the path of the template that it tried to load, and then
leave it to another template loader to actually load it. Store this in
thread local storage (it's a bit ugly, but it's the only thing Django
supports for storing things at the request level from a template
loader), and fetch it from the middleware.
This is today only used for things coming out of the accounts system,
like new accounts and password resets. To make sure we don't
accidentally start using it for something else, change the name of the
parameter.
Seems django 1.11 automatically enables caching template loader, which
of course breaks the ability to make any changes to the pages of a
website without restarting it. And there is no way to turn it off other
than to explicitly configure individual loders (the logic to turn it on
in non-debug configurations is hardcoded and cannot be changed).
This finally moves the patches into the db, which makes it a lot easier
to filter patches in the views.
It also adds the new way of categorising patches, which is assigning
them a CVSSv3 score.
For now, there are no public views to this, and the old static pages
remain. This is so we can backfill all existing security patches before
we make it public.
Sending from the submitters address runs afoul to DMARC and other
restrictions. Instead, send the email from a defined noreply address.
Instead, add the original submitter to both the Cc and the Reply-To
header, to make sure they receive followups.
Patch by Stephen, minor changes by Magnus
This workaround enabled a weaker password hasher to work with old
versions of the community authentication plugin. This was already
overridden in prod, but remove the old workaround completely from the
code so we don't accidentally re-enable it.
We want to use the noreply@ email address when sending out notifications
to addresses that have yet to be verified, since we can just ignore
bounces to those.
This creates Google, Github, Microsoft and Facebook login integrations.
Other providers can also be added if needed. Accounts still need to be
created in the community auth system, and will be automatically created
on first login, when the user also gets to pick a username. Once an
account exists, it will be matched on email address from the external
systems.
No methods are enabled by default, as they all require encryption keys
and identities configured in local_settings.py.
Review by Stephen Frost, Jonathan Katz and Daniel Gustafsson.
The ftp server can then submit a list (and structure) of which platforms
are supported for yum downloads, which can then later (in a separate
commit) be used to generate a nicer download for yum repo rpms.
We have so many users now that loading these forms take forever.
Instead, implement a textbox with autocomplete using django-selectable,
which will not load the whole list of users at once.
Per discussion from a long time ago, interactive docs aren't really
working out. The majority of submissions are either support questions
(which then get rejected because they cannot be answered in this
context) or pointing out docs incorrectnesses (which should be submitted
as a docs bug instead, so they can actually be fixed in the main
documentation).
Old references to /interactive/ will get redirected to /static/
automatically, and we expect to keep doing that for a long time (since
there are many links to them around the net).
The pwn module has never been used, as the pwn are simply sent to
-announce and nothing else. We've kept the code and model around for
doing it on the site for years now and it's unused, so let's remove it
to cut down on maintenance cost.
Move manage.py up one step, set up a new wsgi.py file, and adjust
relative paths that refer to other directories for the new project
layout that we need to use in django 1.8.
The queue used for varnish purges has so few entries that it's really
not worth paying the management overhead for skytools/pgq. Instead we
can use a very simple local deamon using LISTEN/NOTIFY to fire them off.
Now include a proper nagios plugin in this package, so we can get rid
of the not-very-nice munin plugin currently used in the deployment.
The idea is a git repository hook will send a POST to this URL, which
will drop a trigger file somewhere. A cronjob (or inotify listener if
we want to be really fancy sometiem in the future) will pick up that
cronjob and run the update script. The goal being to shorten the time
required to process an update.
Basically, user generated email (bug report form) will be sent to the mail
frontends for antispam. Any errors generated there will be ignored and
the mails "dropped on the floor". Other emails keep entering the system
through localhost and delivered there.
Import the code from the PostgreSQL Europe website to handle this, since it's
well proven by now.
Any points that send email now just write them to the database using the
functions in queuedmail.util. This means we can now submit notification
emails and such things within transactions and have them properly roll bcak
if something goes wrong (so no more incorrect notifications when there is
a database error).
These emails are picked up by a cronjob that runs frequently (typically
once per minute or once every 2 minutes) that submits them to the local
mailserver. By doing it out of line, this gives us a much better way of
dealing with cases where mail delivery is really slow.
The submission from the cronjob is now done with smtp to localhost instead
of opening a pipe to the sendmail command - though this should have no
major effects on anything.
This also removes the setting SUPPRESS_NOTIFICATIONS, as no notifications
are actually ever sent unless the cronjob is run. On development systems
they will just go into the queuedmail table, and can be deleted from there.
