36 Articles

Science of Web Scraping

Deep technical dives into the science behind crawling, parsing, anti-bot systems, and the engineering challenges of web-scale data extraction.

Science of Web Scraping
Scrapy vs. Selenium: A Comprehensive Guide to Choosing the Best Web Scraping Tool
Science of Web Scraping

Scrapy vs. Selenium: A Comprehensive Guide to Choosing the Best Web Scraping Tool

Explore the in-depth comparison between Scrapy and Selenium for web scraping. From large-scale data acquisition to handling dynamic content, discover the pros, cons, and unique features of each. Learn how to choose the best framework based on your project's needs and scale.

Suciu Dan14 min read
Aug 10, 2023
Web Scraping vs Data Mining -  Are We Solving The Confusion?
Science of Web Scraping

Web Scraping vs Data Mining - Are We Solving The Confusion?

Data mining is frequently mistaken with web scraping or data extraction. However, they are two distinct processes that utilize vastly different techniques to achieve their objectives.

ȘTefan RăCila5 min read
Apr 10, 2023
What Are ISP Proxies And How To Use It For Web Scraping
Science of Web Scraping

What Are ISP Proxies And How To Use It For Web Scraping

Discover what ISP proxies are and how they can improve your web scraping efforts. Learn the advantages of using ISP proxies, their potential drawbacks, and how to use them effectively.

Mihnea-Octavian Manolache8 min read
Feb 22, 2023
HTTP Headers 101: How to Use Them for Effective Web Scraping
Science of Web Scraping

HTTP Headers 101: How to Use Them for Effective Web Scraping

Discover the basics of HTTP headers for effective web scraping. Learn common headers, grabbing cookies, and custom headers to bypass security. Master HTTP headers and level up your scraping game.

Raluca Penciuc11 min read
Feb 3, 2023
Node Unblocker for Web Scraping
Science of Web Scraping

Node Unblocker for Web Scraping

Take your web scraping to the next level by creating and deploying a custom proxy using Node Unblocker and never get blocked again.

Suciu Dan8 min read
Jan 16, 2023
The Top 9 Residential Proxy Service Providers
Science of Web Scraping

The Top 9 Residential Proxy Service Providers

Bypassing geo-restrictions and IP blocking can be a true hurdle when web scraping. This list of the best residential proxy providers will help!

Anda MiuțEscu12 min read
Jan 16, 2023
Rotating Proxies: Everything You Need To Know
Science of Web Scraping

Rotating Proxies: Everything You Need To Know

A reliable proxy pool is just the first step towards web scraping greatness. The next one is rotating those proxies. Here's what you need to know!

Raluca Penciuc20 min read
Jan 8, 2023
The Ultimate Web Scraping Tips & Tricks List
Science of Web Scraping

The Ultimate Web Scraping Tips & Tricks List

Having trouble extracting web data? There are plenty of ways to improve your scraper, here are 12 tips that will definitely help!

Anda MiuțEscu13 min read
Jun 15, 2021
Top 5 JavaScript Libraries For Web Scraping
Science of Web Scraping

Top 5 JavaScript Libraries For Web Scraping

Discover the 5 most popular JavaScript web scraping libraries: Axios, Nightmare, Cheerio, Puppeteer, Selenium. Which one is the best?

Robert Sfichi8 min read
Apr 3, 2021
The Ultimate List of Web Scraping Tools
Science of Web Scraping

The Ultimate List of Web Scraping Tools

Just like the many use cases for web scrapers, there are a whole array of tools to choose from. We take a look at 20 tools and see how they help you!

Gabriel Cioci22 min read
Mar 29, 2021
Explore Other Topics