Crawling the TOR network – Challenge Accepted!

Crawling the TOR network – Challenge Accepted!

The following short story portrays the surprising technological and logical challenges we faced while developing our dark web monitoring technology. Back in 2017 when I initially had the idea of adding content […]

Webz.io Image Recognition Helps Identify Illicit Content

Webz.io Image Recognition Helps Identify Illicit Content

How Webz.io Uses Image Analysis and Recognition to Identify Illicit Content on the Dark Web Collecting data from the Dark Web is immensely more complex than it is in the open web. […]

How Does a Web Crawler Work?

How Does a Web Crawler Work?

Learn how a web crawler works, the challenges that arise when building one, and the advantages of building a web crawler using the python language.

The Danger of Fake Reviews

The Danger of Fake Reviews

How to Spot Fake Reviews in Time for the Holidays Black Friday is here, and as the biggest shopping day of the year, it means a lot of people will be on […]

Survey Results: What Matters to Web Data Collection Buyers

Survey Results: What Matters to Web Data Collection Buyers

While structured web data presents exciting possibilities in many fields of endeavor – including finance, cyber-security, artificial intelligence and more – the market for data extraction platforms is still fairly young. Only […]

What is the Omgili Bot, and why is it Crawling Your Website?

What is the Omgili Bot, and why is it Crawling Your Website?

Hi there. If you’re reading this, it’s probably because you’ve run into Omgilibot – perhaps in your web analytics or server logs (user agent: omgili/0.5 +https://omgili.com) – and turned to Google to […]

How to Extract Data from Websites: Scraping Tools, DIY or DaaS

How to Extract Data from Websites: Scraping Tools, DIY or DaaS

This is part 2 of our guide to web data extraction. Read part 1 to learn about the questions to ask before you start, or download the complete Web Data Extraction Playbook […]

Web Data Extraction Guide: 11 Questions to Ask

Web Data Extraction Guide: 11 Questions to Ask

The following is an excerpt from our new Web Data Extraction Playbook. We’ll be publishing the second part next week, or you can grab the full guide here. The internet has become […]

A Judge Just Ordered LinkedIn to Allow Scraping – Here's Why

A Judge Just Ordered LinkedIn to Allow Scraping – Here's Why

When is it okay to grab data from someone else’s website, without their explicit permission? A new ruling by a federal judge in California might have dramatic implications on this question, and […]

The Hackathon Award for Best API Mashup Goes to…

The Hackathon Award for Best API Mashup Goes to…

Competitive programming competitions, commonly referred to as Hackathons, offer a great opportunity for new talent to show what they can do. Much like professional sports, industry leaders send recruiters to scout out […]

Webz.io API Featured in New Guide to Web Development with Django

Webz.io API Featured in New Guide to Web Development with Django

Last February, co-authors Leiff Azopardi and James Maxwell completed the latest edition of their book Tango with Django. It presents an excellent step-by-step approach to learning Python on the popular Django framework […]

How to Use Online Review Ratings to Crush the Market

How to Use Online Review Ratings to Crush the Market

Sifting through millions of posts on review sites presents both a massive undertaking and an incredible opportunity for influencer marketing. Some of the most successful app makers are capitalizing on that oppotunity. Use […]