+1 (718) 878-4993 [email protected]
Data Crawling: Let’s Get To The Legal Side

Data Crawling: Let’s Get To The Legal Side

With the recent Facebook data scandals and the start of the GDPR law, you might be wondering that: “is web crawling legal or not?” Because after all, you have a business. And every business, no matter whether a SaaS startup, an eCommerce Store or any other service provider, needs access to valuable data. Of course, accessing restricted, private and unauthorized data is wrong and will always be illegal. But what about the public data on the internet that’s freely available for anyone to access? Data like:    1. The product inventory list of your competitors on their website.    2. Prices of inventory items on display across various eCommerce stores.    3. Public reviews that mention your own business that are meant to be seen. If that’s the kind of data you want to collect, is it still legal? The answer, unfortunately, is not a simple yes or no. To understand the whole picture regarding what makes data crawling services legal or illegal, we first need to take a quick look at what data scraping actually is. Intro To Data Scraping: A Highly Common Activity That Takes Place On The Internet Data scraping is the act of downloading a web page’s data and taking specific information from it. For example, suppose you wanted to start your own movie streaming service. For that, you’d need data such as movie bio, cast list, the year it was released, it’s rating etc. But there are millions of movies released since they came into existence. What are you going to do? Manually write the bio, cast list and year of release by typing...
Huge Fines in Germany Due To “Illegal Content” On Social Media and How it Can Affect Data Scraping

Huge Fines in Germany Due To “Illegal Content” On Social Media and How it Can Affect Data Scraping

On January 1st, Germany passed a law that obligates social networks to remove content that is considered to be illegal and offensive in 24 hours, or they will be forced to pay fines going as high as $60 million. The bill is designed to cut down any hate speech, fake news, or any criminal content on social media. However, there are people opposing this law, saying that it might put freedom of speech in the online community in jeopardy. Although this law was approved in 2017, the German government waited for the beginning of 2018 to enforce it in order to give social media companies time to prepare. All social media companies will receive complaints or notifications whenever certain content is considered offensive, informing them that they have to remove it within a week or pay a fine. If the content is considered to be “clearly” illegal, it must be banned or removed within a day or the punishment will be severe. These new changes can affect a lot of internet services and businesses looking to protect themselves online. Information on social media is a valuable resource for many companies and service providers. This new law could change the way information on social media is gathered and used.   Web scraping and social media scraping services, in particular, will be greatly affected in light of these events.  Social media scraping, and data scraping services overall offer great value to companies as they can track their competition, learn about the latest news in their industry, learn more about their customers and see what people are talking about them online. This...
4 Simple Steps to Secure Your Valuable Data from Scraping

4 Simple Steps to Secure Your Valuable Data from Scraping

We have all copied and pasted pieces of information or images in our lives and it never occurred to us that we are doing something wrong or breaching any copyright law. But copying on a larger scale, which we have already identified as web scraping takes it all to the next level making it possible to acquire virtually any information out there. As long as it is online – it is reachable and scrapeable and depending on the receiving end of the information it can be a problem for the content owner and their main purpose of work at the same time. It is important to mention that scraping in general if done sustainably, is not evil at all: before we go ahead and point fingers, let’s note that Google is the largest scraper out there and I have never heard anyone complaining about it indexing their content. As many things in life, practice becomes a problem when it’s done in excessive amounts and by the wrong people. In addition, the problem is that web data scrapers take what many companies spent enormous amounts of man-hour and funds to accumulate, for free. Consequently, it gives rise to such problems like customer confidence with a brand, uniqueness of online brands and spreading of sensitive information. As a matter of fact, any behavior that a browser makes can be copied by a knowledgeable web scraper given the intention: that’s why many content creators and site owners get understandably anxious about the thought of a web harvester copying all their data from the website. If the question here is “how to completely...