Fallback to GET on top of HEAD in redirection_test

[donbowman]

Some websites don't implement HEAD, returning 500.
Here is an example: https://duquesnelight.com/

In redirection_test(), which is used for liveness check from trafiltura, this causes a 500 error to be returned.

I guess an alternative would be to allow 500 in the ACCEPTABLE_CODES

[adbar]

Hi @donbowman, I think it would be better to implement a fallback strategy for pages which don't support HEAD requests.

[donbowman]

since i can't a priori know those pages, and i can't disambiguate the error returned from courlan, it would mean checking in advance of calling courlan, doubling the number of requests.
i think here courlan could see the unsupported method returned and fallback to get, rather than making the caller of the library replicate the logic.

[adbar]

Yes, that's what I meant. Would you be available to draft a pull request?