Contribute on GitHub

Part 0: Setup

Initial setup of your scraper environment.

Setup

IMPORTANT: Please be sure to work through the machine setup before proceeding.

You’ll neet PostgreSQL installed:

Within your terminal:

  • Change into the Web Scraping project.
1
$ cd new-coder/scrape
  • Make sure you’ve installed virtualenvwrapper and followed the steps above from Initial Requirements above to set up your terminal correctly. More information can be find at virtualenvwrapper’s docs.
  • Make a virtual environment specific to your Scrape project:
1
$ mkvirtualenv ScrapeProj
  • You should see (ScrapeProj) before your prompt. Now install package requirements with the following command for this project.
1
(ScrapeProj) $ pip install -r requirements.txt
  • Your virtual environment will store the required packages in a self-contained area to not mess up with other Python projects.

Continue on to Part 1: Scraper Setup →