How to Web Scrape Google Shopping Nearby Sellers with Node.js

Andrei Ogiolan on Feb 28 2023

This article will guide you through the process of using our API with Node.js to extract nearby sellers from Google Shopping. The steps involved will include setting up the development environment, scraping relevant data, and discussing issues you may face. By the end of this tutorial, you will have the necessary knowledge and tools to scrape nearby sellers from Google Shopping on your own. Before delving into the technical details, it's important to understand what Google Shopping is.

Google Shopping is a platform that allows consumers to search for and compare products from various online retailers. The platform displays a wide range of products along with their prices, making it a user-friendly tool for finding the best deals. With web scraping techniques, we can extract valuable data such as nearby sellers from Google Shopping to gain insights and make informed decisions.

Why should you use a professional web scraper?

Building your own web scraper can be a time-consuming and complex task. Not only do you need to have a strong understanding of programming, but you also need to have a deep understanding of web scraping techniques and technologies. This can be a daunting task for someone without a background in web scraping, and the time and resources required to build a functioning scraper may not be worth the investment.

On the other hand, using a professional scraper like ours offers several advantages. For one, professional scrapers are built and maintained by experts in the field, ensuring that they are up-to-date with the latest technologies and techniques. This means that they are more likely to be able to handle the complexities of the modern web, such as CAPTCHAs and dynamic web pages. Additionally, professional scrapers often come with built-in features such as automatic IP rotation, which can help to prevent your IP from being blocked by the website you are scraping.

Another advantage of using a professional scraper is that it can save you a significant amount of time and resources. Instead of spending weeks or even months building your own scraper, feel free to subscribe to our scraper by creating an account here and start scraping right away. This is especially beneficial for businesses that need to extract data quickly in order to make informed decisions, or for individuals who want to scrape data for personal projects but don't have the time or resources to build their own scraper.

What are Google Shopping Nearby Sellers?

Google Shopping Product Nearby Sellers is a feature that allows users to find nearby retail stores that carry a specific product. This feature is available on the Google Shopping website and mobile app, and it allows users to search for products and see a list of nearby sellers that carry that product. This information includes store name, distance, and address of the store, and also if the store is currently open or closed.

Web scraping Google Shopping Product Nearby Sellers can be beneficial for businesses and individuals looking to gain insights into local retail markets. By scraping this data, businesses can gain a better understanding of the competition in their area, and adjust their pricing and inventory accordingly. Additionally, businesses can use this data to identify new sales opportunities, such as approaching stores that don't currently carry their products. For individuals, web scraping Google Shopping Product Nearby Sellers can be a useful tool for finding the best deals on products, and also identify the store that is closest to them.

What does our target look like?

Setting up

To scrape nearby sellers from Google Shopping using our API, you will need to have the proper tools in place. First and foremost, you must have Node.js installed. This is a JavaScript runtime that allows you to run JavaScript on the server-side and can be downloaded from the official Node.js website.

Additionally, you will need an API KEY to use our service which you can get by creating an account here and activating the SERP service.

After you have Node.js and an API KEY, the next step is to create a Node.js script file by running the following command:

$ touch scraper.js 

And now paste the following line in your file:

console.log("Hello World!")

And the run the following command:

$ node scraper.js

If you see the message "Hello World!" displayed on the terminal, it means that Node.js has been successfully installed and you are ready now to proceed to the actual scraping section.

Let’s start scraping Google Shopping Nearby Sellers

Having the environment set up, you can now start scraping Google Shopping Nearby Sellers using our API. This is a straightforward process and apart from what was discussed above, all you need to do is to get the product ID of the product you are interested in.

Tip: This is how you can get the product ID of a product from Google Shopping:

You are now prepared to begin scraping after setting up Node.js, obtaining an API key, and acquiring a product ID. To proceed, either create a new JavaScript file or utilize the one created earlier and import Node.js's built-in `https` module, which allows you to send requests to the API, by using the following code:

​​const https = require("https");

Secondly, you need to specify your API key and the product_id property of the product you are interested:

const API_KEY = "YOUR-API-KEY-HERE" // You can obtain one by registering here

const PRODUCT_ID = "4887235756540435899"

Next, you need to pass this information in an options object in order to let our API know what is the product you are looking to scrape:

const options = {

 "method": "GET",

 "hostname": "serpapi.webscrapingapi.com",

 "port": null,

 "path": `/v1?engine=google_product&api_key=${API_KEY}&product_id=${PRODUCT_ID}`,

 "headers": {}

};

And lastly you need to set up a call to our API with all this information:

const req = http.request(options, function (res) {

  const chunks = [];

  res.on("data", function (chunk) {

    chunks.push(chunk);

  });

  res.on("end", function () {

    const body = Buffer.concat(chunks);

    const results = JSON.parse(body.toString());

    const nearbySellers = results.sellers_results.online_sellers;

    console.log(nearbySellers)

  });

});

req.end();

All is left for you to do now is to execute the script you have created and wait for the results:

​​$ node scraper.js

And you should now receive your results:

[

  {

    position: 1,

    name: 'Gamestop',

    link: 'https://www.google.com/url?q=https://www.gamestop.com/consoles-hardware/playstation-5/consoles/products/sony-playstation-5-digital-edition-console/225171.html%3Futm_source%3Dgoogle%26utm_medium%3Dfeeds%26utm_campaign%3Dunpaid_listings&sa=U&ved=0ahUKEwi27suDvtP8AhVkk2oFHXzfDeMQ2ykIZg&usg=AOvVaw3ZevYyiKByTyo_THSF1qUJ',

    additional_details: '460.5 mi · In stock at EulessToday: 10:00 AM - 9:00 PM',

    base_price: '$399.99',

    additional_price: { shipping: 'See website' },

    total_price: '$399.99',

    trusted: true

  },

  {

    position: 2,

    name: 'Gamestop',

    link: 'https://www.google.com/url?q=https://www.gamestop.com/consoles-hardware/playstation-5/consoles/products/sony-playstation-5-digital-edition-console/225171.html%3Futm_source%3Dgoogle%26utm_medium%3Dfeeds%26utm_campaign%3Dunpaid_listings&sa=U&ved=0ahUKEwi27suDvtP8AhVkk2oFHXzfDeMQ2ykIcw&usg=AOvVaw1QiXFtHB6-CApj-HDvbNxl',

    additional_details: '462.6 mi · In stock at ArlingtonToday: 11:00 AM - 8:00 PM',

    base_price: '',

    trusted: false

  },

  {

    position: 3,

    name: 'Gamestop',

    link: 'https://www.google.com/url?q=https://www.gamestop.com/consoles-hardware/playstation-5/consoles/products/sony-playstation-5-digital-edition-console/225171.html%3Futm_source%3Dgoogle%26utm_medium%3Dfeeds%26utm_campaign%3Dunpaid_listings&sa=U&ved=0ahUKEwi27suDvtP8AhVkk2oFHXzfDeMQ2ykIdg&usg=AOvVaw0CxoqlJzdEZ93B-6U-Jmuf',

    additional_details: '557.6 mi · In stock at HoustonToday: 12:00 PM - 9:00 PM',

    base_price: '',

    trusted: false

  },

  {

    position: 4,

    name: 'Gamestop',

    link: 'https://www.google.com/url?q=https://www.gamestop.com/consoles-hardware/playstation-5/consoles/products/sony-playstation-5-digital-edition-console/225171.html%3Futm_source%3Dgoogle%26utm_medium%3Dfeeds%26utm_campaign%3Dunpaid_listings&sa=U&ved=0ahUKEwi27suDvtP8AhVkk2oFHXzfDeMQ2ykIeQ&usg=AOvVaw2BAlgtL85g1mvOonMQK14U',

    additional_details: '568.6 mi · In stock at PasadenaToday: 12:00 PM - 9:00 PM',

    base_price: '',

    trusted: false

  },

  {

    position: 5,

    name: 'Gamestop',

    link: 'https://www.google.com/url?q=https://www.gamestop.com/consoles-hardware/playstation-5/consoles/products/sony-playstation-5-digital-edition-console/225171.html%3Futm_source%3Dgoogle%26utm_medium%3Dfeeds%26utm_campaign%3Dunpaid_listings&sa=U&ved=0ahUKEwi27suDvtP8AhVkk2oFHXzfDeMQ2ykIfA&usg=AOvVaw3mi7jMt3aMSJde0sQb9yjR',

    additional_details: '591.2 mi · In stock at San AntonioToday: 12:00 PM - 9:00 PM',

    base_price: '',

    trusted: false

  },

... 

]

You have now successfully scraped Google Product Nearby Sellers using our API. You can use the obtained data for various purposes such as price comparison, market research, SEO optimization and more. For more information and code samples in other six programming languages, feel free to refer to our Google Product API documentation.

Limitations of Google Shopping Nearby Sellers

Scraping Google Shopping Nearby Sellers to collect data can be a useful way to gather information on local retailers who carry a specific product. However, it also has some limitations. One limitation is that the feature may not always return accurate or up-to-date results. For example, a store listed as selling a product may have sold out or stopped carrying it. Additionally, the feature may not include all local retailers that carry the product, leading to a limited selection of options for the user. Finally, Google Shopping Nearby Sellers section is only available for select products and categories, so users may not be able to find local sellers for all the products they're interested in.

Conclusion

I hope you found this article a helpful resource to get you started in scraping Google Shopping Nearby Sellers with our API using Node.js. With just Node.js set up, an API key for our service and a product ID you can gather information on prices and availability in no time. This can be particularly useful for businesses looking to stay competitive in pricing, as well as for consumers looking to find the best deals.