Web scraping

What is web scraping?

• update data of websites

• extract data from websites

• stole data from websites

scraping intellectual property is legal

• True

• False

Two main languages to develop web scraping projects

• Nodejs and Python

• Typescript and Python

• Javascript and Python

• Javascript and Selenium

The main steps are:

• 1- Pick a target URL

  2 - Fetch the web page

  3 - Extract the data

  4 - Save it in a useful format

• 1- Pick a target URL

  2 - Extract the data

  3 - Fetch the web page

  4- Process it

  5 - Save it in a useful format

• 1 - Extract the data

  2 - Fetch the web page

  3- Process it

  4 - Save it in a useful format

• 1- Pick a target URL

  2 - Fetch the web page

  3 - Extract the data

  4- Process it

  5 - Save it in a useful format

What sould not do:

• Use a cron to automate task

• Scrap data on every request

• Decide the usable format in the scraping stage

• Create a previous UML diagram