Even though web scraping is commonly used across most industries, most websites do not appreciate it and new anti-scraping methods are being developed regularly. The main reason is that aggressive web scraping can slow down the website for regular users, … Read More
Scrape an entire HTML page without getting blocked
When you want to collect and analyze data, let it be for price comparison, statistics or to see a general evolution, scraping is a great and essential time saver.
However, many websites do not appreciate to be heavily scraped, some of them don’t allow it at all, especially in the retail industry sector.
There are some generic rules and tricks to respect/follow if you do not want to be blocked from scraping a website, temporarily or permanently. Thanks to our API you will be able scrape the content of a page without getting blocked.
The Raw HTML API collects the data while preserving its structure.
This module is especially made for scraping HTML from Google results for example.
On our last real estate case study, we scraped useful data from a property listing on Funda, one of the main housing websites in France, for both private and professional ads. This time, we’re going to the UK, to scrape … Read More