Web Scraping with Screenshot

The scraping agent allows you to capture the webpage screenshot automatically while crawling a website, as it is very common in web scraping to capture a screenshot of a website for testing or as a proof of data being scraped is exactly same as it’s shown on the website.

There are 3 types of webpage screenshot capture options available in Agenty

  1. Small screenshot : Visible portion on screen or viewport set
  2. Full screenshot: Full web page screenshot
  3. HD screenshot: High definition screenshot of full web page

Step by step guide: Let’s make them easy to understand how to capture automated website screenshot with this tool:)

Small screenshot

To capture the visible webpage screenshot only, follow these steps:

  • Go to your agent page
  • Click on the Configuration tab and scroll down to Fields section
  • Add a new field with configuration like this
  • Field Name : Any name of your choice
  • Field Type:DEFAULT`
  • From : screenshot

  • Save and re-run your agent
  • The scraping agent will capture the visible webpage screen.

This option allow you to capture the above the fold or visible part of website on computer screen without scrolling. The webpage will be captured as per the viewport width and height given in the web scraping engine. So, if you are looking to capture the visible page screenshot with web scraping, use this option. Here is an example :-

https://s3.amazonaws.com/data.agenty.com/Screenshots/2022/7/21/books.toscrape.com-i547w4ehnl.png

Full page screenshot

The full page screenshot option allow us to capture automated website’s full page screen, which make the task easier to diagnose to view full content of website.

To make screenshot of an entire portion of a webpage - We need to follow the same steps as followed above, except change the from type to full_screenshot:

  • Agent page > Configuration Tab
  • Add new Field
  • Select Default as Type
  • Select full screenshot option in system fields

  • Save this and Click on Run button

Now, when the scraping job is executed, Agenty will capture the full page screenshot.

Here is an example of full webpage screenshot captured in this agent with viewport : full web page screenshot with web scraping

HD Screenshot

The High Definition screenshot produce larger, wider and clearer picture of a website with increased pixels and wider scaled viewport. So, the HD screenshot option allows you to get the better experience to diagnose the web page during web scraping.

To capture the HD screenshot of web page, follow these steps:

Now you can see the high definition quality screenshot of all the pages of the website will be captured by Agenty automated scraping agent.

Signup now to get 100 pages credit free

14 days free trial, no credit card required!