Merge branch '21-fix-corpus-read-error' into 'master'

Integrated separate checker/corrector and crawler modules into one package

Implemented Serialization and Deserialization of TrieNode for crawler (limited compression to 1KB per site by tracking incremental compressed size)

Implemented Serialization and Deserialization of TrieNode for crawler

Implemented streaming for GUI so that output appears as it crawls each site

Implemented TrieNode structure for crawler to use ngrams and track conditional probabilities

Limited compression to 1KB per site by tracking incremental compressed size in crawler

Implemented logic for making correction and ranking based on differences

Implemented GUI text highlighter for checker

with open('/Users/manuelsegimonplana/Documents/Current Courses/Not Completed Homework/DS - Project/group7/src/main/java/resources/english.txt', 'r') as file:

    lines = file.readlines()

# Remove lines with less than 4 words

lines = [line for line in lines if len(line.split()) >= 4]

# Shuffle the lines

random.shuffle(lines)

# Optionally, you can write the cleaned lines back to a file

with open('src/main/java/resources/brown.txt', 'a') as file:

    file.writelines(lines[:450000]) # Last working number: 500000

    file.writelines(lines[:10000])

                }

            }

        } else {

            extractWordUsage(web_data.text(), wordUsage);

            extractWordUsage(web_data.text().replaceAll("\\p{Punct}", ""), wordUsage);

            // System.out.println("Ngrams built successfully. for size:"+MAXNGRAM);

            uncompressedData = wordUsage.serialize();

            compressedData = compress(uncompressedData);