Shopify Integration : Scrape supplier website and upload product to Shopify store automatically

Agenty Shopify integration allow you to automatically crawl the products from your manufacturer or supplier websites(even password protected) and upload to your Shopify store, update prices, images and more.

Shopify is most popular eCommerce platform and used by 600,000+ businesses to create their online stores, without writing any code to make your website live.
All you need is — Signup and upload your products to create an eCommerce website in few minutes. And their built-in templates and apps marketplace makes it even more easier for businesses to launch their eCommerce store online and start selling products globally in no time.

Our Shopify integration allows you to :

  • Crawl manufacturer, supplier and brands website to download the products
  • Transform the products to match with your Shopify store format, change price to add profit etc. and finally upload the products to your store
  • Schedule the crawler to find new products daily, weekly and upload to your store
  • Schedule the crawler to update price, out-of-stock, inventory quantity to sync with the source website for better inventory management.

With all the custom rule and logic’s before uploading to your Shopify store, for example:

  • Add a custom script to increase the price by 10% to add your margin.
  • Replace “USD” with “$” in price field or something else using the built-in post-processing functions
  • Remove “By manufacturer name” form product title
  • And any other custom rule…


  1. Good knowledge of Agenty web scraping tool to scrape data from supplier websites
  2. Credentials of supplier website, if password protected. (see docs here to learn how to crawl password protected site
  3. A Shopify store
  4. Agenty professional or higher plan to get access of Shopify plugin

Shopify Connector

  • Go to Agenty workflows page
  • Create a workflow
  • Select your web scraping agent

  • Select Shopify connector and click on Connect to Shopify
  • Enter your Store URL and click on Connect button.

  • Click on “Connect to Shopify” button and you’ll be redirected to Shopify for authorization.

  • Authorize Agenty to by clicking on the “Install app” button.
  • Give the permissions to read_products, write_products and write_inventory

  • Once connected successfully, Shopify will redirect you back to Agenty with Oauth token to access your store for product and stock sync.

Setup your scraping agent

The web scraping agents MUST have following fields with exact same name to map them with Shopify API to sync inventory between your Shopify store and the supplier website.

Field Name Data Type Description
url string (mandatory field) Product URL of vendor website, this will be used as reference to sync the products (update price, quantity etc.) and avoid duplicates
title string (mandatory field) Product title
vendor string Brand or vendor name
body_html string HTML body of the product
product_type string The type of products, e.g mobiles, books, laptops etc.
tags string Comma separated tags
images string Comma separated images URLs
price number Product price in decimal format. (Agenty will automatically remove any non number or period character(if any) while converting the price into valid decimal format.)
compare_at_price number Product compare at price in decimal format to show discount. (Agenty will automatically remove any non number or period character(if any) while converting the price into valid decimal format.)
inventory_quantity number In stock quantity of products in number format. Agenty will automatically remove any non number character(if any) while converting the quantity into valid number format. By default : 1

Variants importing

If your product has variants like color or sizes. Just add option1 field to attach the variants with product while importing with your Shopify store.

Field Name Data Type Description
sku string Product SKU
option1 string Name of the variant
price number Price of the variant
inventory_quantity number Inventory quantity

Here is an example agent with size variants scraping

If you do not know how to create a scraping agent? I’d recommend to watch this video here or read this post first.

Advance Shopify settings

  • Go back to workflows page
  • Edit your Shopify workflow
  • A new page will open where you can select the web scraping agent, to attach the Shopify product sync workflow and can select your store, if multiple we just connected on step #1

  • Profit percentage : Use the profit percentage text box to add the margin percentage. Agenty will increase the extracted price by a given percentage number to add your profit margin automatically.

  • Rounds up : Use the rounds up switch to enable/disable the rounding off. When enabled, the price will be rounded by next 10, for example 198.99 will become 200; and 92.5 will become 100.

Start the scraper

  • Go to your agent page
  • Click on the Start button. (Remember : Don’t enter all URLs in input, if you are testing the scraper because that will run longer and consume your pages
    credit while testing

crawl shopify website

Alternatively, you can also use our API to start a scraping job

curl -X POST -H "Content-type: application/json" -d '{
    "agent_id":"{AGENT ID HERE}"
    }' '{API KEY HERE}'

The job will take few minutes, depending on the number of web pages you are crawling and also your plan limitation.

Once the job has been completed, check the trace logs and you’ll find messages like :

Shopify sync started...
Product created successfully- Shopify Id: 1644195479605
No updated available - Shopify Id: 1644196003893
Product created successfully- Shopify Id: 1644196003887
Product updated successfully- Shopify Id: 1644196003862

Check your store

Now the agent has been completed successfully, check your Shopify store and you’ll find that all the scraped products have been uploaded to your store.

shopify product catalog import automatically

Signup now to get 100 pages credit free

14 days free trial, no credit card required!