What is Web Scraping?Web Scraping is a technique to extract large amount of data from websites. It is also know as Screen Scraping, Web Harvesting, Web Data Extraction etc.
Generally, to view data from third party websites, we use web browsers, for example we use our web browser to view data from yellow pages directories, social networking sites, real estate sites, contact databases, online shopping sites etc. But if you want to save the data from those websites in your hard disk, you find it very difficult and time killing because you can't download their databases and you have to do the data collection manually and it may takes hours and even days to complete.
So, to automate this process, Web Scraping is used. Instead of doing manual work, you may do your data collection work using the Web Scraping software within a fraction of the time.
How Web Scraping Software Works?Just like a human, Web Scraping software treats a website. It interacts with the websites in the same way as your web browser does. But instead of displaying the data served by the website on the screen, the Web Scraping software saves the required data from the webpage to a local database or file.
"Web scraping is closely related to web indexing, which indexes information on the web using a bot or web crawler and is a universal technique adopted by most search engines. In contrast, web scraping focuses more on the transformation of unstructured data on the web, typically in HTML format, into structured data that can be stored and analyzed in a central local database or spreadsheet. Web scraping is also related to web automation, which simulates human browsing using computer software. Uses of web scraping include online price comparison, contact scraping, weather data monitoring, website change detection, research, web mashup and web data integration."