How do I get response headers?

[todordev]

Hi,
How can I get the response headers of each crawled page?

Thank you for this wonderful project. :)

[B4nan]

The response object is available in the context, so you should be able to read it from there

https://crawlee.dev/api/http-crawler/interface/HttpCrawlingContext#response

[todordev]

Thank you!
I found a solution.

router.addHandler('detail', async ({ request, response, page, log, parseWithCheerio }) => {
   const redirectFromUrl = response.request().redirectedFrom().url();
   const redirectFromHeaders = response.request().redirectedFrom().headers();

   const redirectFromResponse = await response.request().redirectedFrom().response();
   const redirectFromStatus = redirectFromResponse.status();
   const redirectFromStatusText = redirectFromResponse.statusText();
});

[tonnyavery]

@B4nan Is it only available for Playwrite and Puppeteer, but not for Cheerio crawler? If so, what is the reason why? Cheeriocrawler also visits pages and should be able to get response and headers

[B4nan]

It is available in all crawlers, the doc link above is valid for cheerio, the HTTP crawler is a base of the cheerio one.

[tonnyavery]

@B4nan I will try it out. Confusion comes from here https://crawlee.dev/api/cheerio-crawler/interface/CheerioCrawlerOptions#requestHandler.
It says that The function receives the [BasicCrawlingContext](https://crawlee.dev/api/basic-crawler/interface/BasicCrawlingContext) as an argument, where the [request](https://crawlee.dev/api/basic-crawler/interface/BasicCrawlingContext#request) represents the URL to crawl.
Also when destructuing parameters out of options, response doesn't show up

[B4nan]

cheerio crawler has cheerio context:

https://crawlee.dev/api/cheerio-crawler/interface/CheerioCrawlingContext