RSS feeds (especially security) take an age to update

Apologies if this belongs in the webmaster queue, it’s tricky to know where this sits.

The RSS feeds for security can be woefully out of date.

At the moment, if I request the feed for contrib security I get:

$ http -h
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 5389
Cache-Control: public, max-age=86400
Connection: keep-alive
Content-Encoding: gzip
Content-Language: en
Content-Length: 21819
Content-Security-Policy: frame-ancestors 'self'
Content-Type: application/rss+xml; charset=utf-8
Date: Thu, 03 Mar 2016 11:48:40 GMT
Etag: "1457000330-1"
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Fastly-Debug-Digest: 73ba47cb255994fda18c345ce4895963252ca82462d32c44458f493ffafd0705
Last-Modified: Thu, 03 Mar 2016 10:18:50 GMT
Server: Apache
Strict-Transport-Security: max-age=10886400; includeSubDomains; preload
Vary: Cookie,Accept-Encoding
Via: 1.1 varnish
Via: 1.1 varnish
X-Cache: HIT, HIT
X-Cache-Hits: 1, 2
X-Content-Type-Options: nosniff
X-Drupal-Cache: MISS
X-Frame-Options: SAMEORIGIN
X-Served-By: cache-sea1924-SEA, cache-lcy1122-LCY
x-url: /security/contrib/rss.xml

So the feed data can be cached for 24 hours. That seems reasonable enough, to avoid overloading the d.o servers, but it takes AGES for the RSS feed to be updated with new data, I guess, up to 24 hours.
That’s probably not a problem for most feeds, but when we’re talking about security updates, it’s a bit annoying.

We have an automated tool that is reading the RSS feed and creating issues in our internal issue tracker for security issues. At the moment it’s not finding the security releases that came out last night for hours and hours.

Ideally, as soon as security release comes out, the downstream caches would get flushed, I’m assuming an internal Varnish and Fastly Varnish are the downstream caching servers here. Is this possible?

Presumably the Drupal security Twitter account: is also consuimg the RSS feed, which is why the tweets appeared there over 12 hours after the initial release.


Leave a Reply