Commit bd26f9a9 authored Mar 31, 2024 by Alexander Ross Melnick

Basic Requirements

parent 30f5b738

Crawler/src/main/java/Debug.class

0 → 100644

+312 B

File added.

No diff preview for this file type.

Crawler/src/main/java/Debug.java

+1 −1

Original line number	Diff line number	Diff line
		@@ -2,6 +2,6 @@

		public class Debug {
		// Set to true to enable debug messages. Set to false to disable debug messages
		public static boolean DEBUG = true;
		public static boolean DEBUG = false;
		public static boolean DEBUG_RobotsTXT = false;
		}

Crawler/src/main/java/RegexParser.class

0 → 100644

+891 B

File added.

No diff preview for this file type.

Crawler/src/main/java/RobotsTXT.class

0 → 100644

+1.19 KiB

File added.

No diff preview for this file type.

Crawler/src/main/java/ScratchCrawler.class

0 → 100644

+8.71 KiB

File added.

No diff preview for this file type.