Commit 3dbb67f9 authored by Alexander Ross Melnick's avatar Alexander Ross Melnick
Browse files

Update UpdateEdits.md

parent e0cb1def
Loading
Loading
Loading
Loading
+12 −7
Original line number Diff line number Diff line
# Alex
- Accepted Feedback:
    - Removed stack traces from error messages for the scrawler
    - Added displaying number of pages crawled thus far regardless of --stats argument
    - Blacklisted assets.tumblr.com
@@ -6,7 +7,11 @@
    - Added to INSTALL.md:
        - Added some common websites to try crawling
        - Added some instructions for seed URLs
- Rejected Feedback:
    - n/a
- Other Changes:
    - Emphasized the reduced amount of URLs available when running without --xl
    - Crawler now fetches a new URL without incrementing the crawl count if the URL contains a file extension for a image, video, audio, or PDF


# Reza