The web is an invaluable source of data when it comes to competitive intelligence, research and creating a go-to-market strategy. A simple Google search will reveal incredible amounts of public information about your main competitors, and connect you to mounds of existing research, financial analysis and other information that could be pertinent to your next big move.
However, unless you’re competing against a tribe of nomadic cave-dwellers, it’s fair to assume your competitors are also getting the types of information that’s available through Google. To really take things to do the next level, you should think about extracting data from websites that could be relevant to your industry – which can allow you to supplement your qualitative research with deeper patterns, trends and insights gleaned by using analytic tools and techniques.
Within this sphere, a major data source could be eCommerce websites. If approached correctly, these sprawling online stores can serve as a virtual treasure trove of insights into your potential buyers, competitors and the overall trends in the market. This article will present 3 types of analyses you can perform with structured data crawled from eCommerce websites.
If you want to try it yourself on some actual products across a wide variety of leading retail websites, you can do so by signing up for the free Webz.io eCommerce API.
1. Monitoring Pricing Trends over Time
One of the most important things you want to be looking at is product pricing, both to inform your own strategy and to gain insight into market sentiment and the state of the competition.
For example, let’s say you’re planning to launch a new brand of toasters, you can see the price range in which new and used toasters are being sold online, which can help you decide on the best pricing strategy for your own product.
However, if you have access to historical data regarding those same products, you can go one step further and check the pricing trend of a competing product or line of products. This can give you some idea as to consumer sentiment – are people paying less for the same toaster today than they were a few months ago? If so, is this because of market saturation, competition, or some new technology that is making toasters obsolete?
In other words, this is a crucial data point as it allows you to ask more intelligent questions and further inform your commercial strategy. You can read more about access to historical web data in our guide to comparing data extraction tools.
2. Keeping an Eye on your Competitors
The most basic aspect of competitive intelligence is, well… actually seeing what your competitors are doing. To do this, you should definitely take the obvious steps of visiting their website, reading their company blog, subscribing to their press releases, etc. But once again you can use crawled eCommerce data to go one step deeper and see beyond the bright public image every company inevitably tries to paint.
If your competitor sells products in online marketplaces, you can monitor these websites to gain real-time visibility into new products being launched, and how they are being received in the market (by looking at reviews or sales figures, if these are available).
You can see which channels seem to be working best for your main competitors, and make an educated guess regarding the ways they are allocating marketing and research budgets, based on what products they are releasing and where. You might also be able to discover the geographical markets they are focusing on.
The most important thing here is breadth of coverage: you’re looking for information that you might not know exists before you find it. This means you need a web data platform that automatically scans and indexes many eCommerce product pages, which you can then scan for your competitor’s name – rather than having to manually “point” at product pages to scrape.
3. Demand Analysis
Let’s say you’re juggling a few different ideas or verticals for your next product launch. You’ll usually want to target the “blue ocean” where demand is high and competition is low. Web data can be a powerful tool to find these relatively-empty spaces where your own product can address a gap in the market.
By querying several major online retailers for keywords related to the type of product you’re interested in launching, you can ask and answer questions such as:
- How many brands are already selling similar products?
Note that if you’re not familiar with every player and product in the market, you would have to be able to broadly scan websites based on a set of keywords to answer this question. - How popular are the existing products with consumers?To gauge this you can use numerical ratings, sentiment analysis or the number of comments / reviews tied to the product listing
- Does demand vary according to region?
One type of product could be extremely popular in India but yet to gain mainstream acceptance in France. This could suggest that clever sales and marketing will allow you to grab some still-untapped markets.
Extracted eCommerce product data won’t necessarily answer all of these questions. It might give you a general notion for some, and serve as a starting point for others; it might actually raise more questions than it answers. But that’s actually a good thing because relevant questions are the building blocks of any analytical project!
The Sky’s the Limit
These are just jumping off points – creative data scientists and market researchers are constantly finding new and innovative ways to use web data, as we’re seeing with many of our existing clients who are using the eCommerce data feed. You’re welcome to try it out yourself and see what insights you can find – it’s as simple as filling out this form.
Want to learn more about using web data for competitive research? Check out our guide to using rated reviews for sentiment analysis, or get in touch with one of our experts to see a live demo of Webz.io.
