Web Scraping at Scale for Market Insights in the Automotive Industry

Learn how GroupBWT helped a client in the automotive industry collect data at scale and gain valuable market insights using web scraping.

single cases background

The Client Story

Our client is multinational automotive manufacturing company and the world’s largest automaker by worldwide sales in 2020 with more than 10 million vehicles sold worldwide.

The client produces a wide range of vehicles, including cars, vans, and trucks, under various brands and has a global presence, with operations in more than 150 countries and production facilities in more than 20 countries.

Industry: Automotive
Cooperation: Since 2022
Location: Worlwide

Improved assortment planning system with the possibility to track real-time data on the market

Improved the client’s market share and boost their sales.


Collecting Data on Used Car Market in 10 European Countries

Web scraping in automotive industry can be a powerful tool for businesses looking to gather valuable market insights and stay ahead of the competition.

Our client, a company in the automotive industry was facing a problem with collecting data on the used car market in 10 European countries. They wanted to know which brands of their competitors were selling the most cars in each country, but didn’t have the expertise or resources to collect this data internally.

single cases content
The Solution

Scraping OEM and Popular Online Marketplaces, Developing a Robust Architecture

GroupBWT was selected as a web scraping expert to help with this challenge. We proposed a solution to scrape two types of websites in each country: OEM (original equipment manufacturer) and popular online marketplaces. Our team manually researched and collected a list of OEM websites for each brand of interest and 2-3 popular marketplaces in each country.

We then set up customized scrapers capable of collecting large volumes of data from up to 10 websites daily, ensuring that we obtained the desired information, including prices, search positions, and inventory levels. We implemented cutting-edge technologies to ensure that our scrapers were efficient, reliable, and accurate.

The data was stored in a database and an API was developed for the client’s data analysts to access and analyze the data further.

single cases content

We recognized the importance of not overloading the target websites during the scraping process. To address this, we implemented anti-ban solutions and captcha solving mechanisms, ensuring that we scraped data ethically and legally.

Gleb Yama
Team Lead WEB
The Results

Gaining Valuable Insights and Improving Market Share with 1.5 Million Pieces of Car Information

The project was a success and the client was able to gain valuable insights into the used car market in 10 European countries. They were able to see which brands of their competitors were selling the most cars and make data-driven decisions to improve their market share.

In total, we scraped 418 websites and collected over 1.5 million pieces of car information. The project took 5-6 months to develop the solution and 4-6 weeks to collect all the data. The client was able to use this data to improve their market share and make more informed business decisions.

GroupBWT’s expertise in data collection at scale, robust data architecture, and solid QA processes were key to the success of this project. We were able to provide the client with a solution that met their needs and helped them to gain a competitive advantage in the market.

single cases content

Ready to discuss your idea?

Our team of experts will find and implement the best eCommerce solution for your business. Drop us a line, and we will be back to you within 12 hours.

Contact Us