Commit 30f5b738 authored by Seyed Reza Sajjadinasab's avatar Seyed Reza Sajjadinasab
Browse files

Merge branch 'Reza' into 'master' 

updateReadMe

See merge request ec504/ec504_projects/group6!5
parents 862a4680 454f5a78

CheckerCorrector/samples/confidence_scores.json

deleted100644 → 0
+0 −115
Original line number Diff line number Diff line 
{

  "sentences": {

    "it a very good book, but it is small small book": 18.181818181818183,

    "it is very bad": 25.0,

    "I am sopistcated": 33.333333333333336,

    "This is so strange a word choice": 0.0,

    "if you good you will do it": 28.571428571428573,

    "this is a reminder that if you fail to complete you will pass the test": 6.666666666666667,

    "": 0.0

  },

  "phrases": {

    "it a": 50.0,

    "a very": 50.0,

    "very good": 50.0,

    "good book,": 50.0,

    "book, but": 50.0,

    "but it": 50.0,

    "it is": 50.0,

    "is small": 50.0,

    "small small": 50.0,

    "small book": 50.0,

    "it a very": 33.333333333333336,

    "a very good": 33.333333333333336,

    "very good book,": 33.333333333333336,

    "good book, but": 33.333333333333336,

    "book, but it": 33.333333333333336,

    "but it is": 33.333333333333336,

    "it is small": 33.333333333333336,

    "is small small": 66.66666666666667,

    "small small book": 33.333333333333336,

    "it a very good": 25.0,

    "a very good book,": 25.0,

    "very good book, but": 25.0,

    "good book, but it": 25.0,

    "book, but it is": 25.0,

    "but it is small": 25.0,

    "it is small small": 50.0,

    "is small small book": 50.0,

    "is very": 50.0,

    "very bad": 50.0,

    "it is very": 33.333333333333336,

    "is very bad": 33.333333333333336,

    "it is very bad": 25.0,

    "I am": 0.0,

    "am sopistcated": 100.0,

    "I am sopistcated": 33.333333333333336,

    "This is": 0.0,

    "is so": 50.0,

    "so strange": 50.0,

    "strange a": 50.0,

    "a word": 50.0,

    "word choice": 50.0,

    "This is so": 0.0,

    "is so strange": 33.333333333333336,

    "so strange a": 33.333333333333336,

    "strange a word": 33.333333333333336,

    "a word choice": 33.333333333333336,

    "This is so strange": 0.0,

    "is so strange a": 25.0,

    "so strange a word": 25.0,

    "strange a word choice": 25.0,

    "if you": 50.0,

    "you good": 50.0,

    "good you": 50.0,

    "you will": 50.0,

    "will do": 50.0,

    "do it": 50.0,

    "if you good": 66.66666666666667,

    "you good you": 33.333333333333336,

    "good you will": 33.333333333333336,

    "you will do": 33.333333333333336,

    "will do it": 33.333333333333336,

    "if you good you": 25.0,

    "you good you will": 25.0,

    "good you will do": 25.0,

    "you will do it": 25.0,

    "this is": 50.0,

    "is a": 50.0,

    "a reminder": 50.0,

    "reminder that": 50.0,

    "that if": 50.0,

    "you fail": 50.0,

    "fail to": 50.0,

    "to complete": 50.0,

    "complete you": 50.0,

    "will pass": 50.0,

    "pass the": 50.0,

    "the test": 50.0,

    "this is a": 33.333333333333336,

    "is a reminder": 33.333333333333336,

    "a reminder that": 33.333333333333336,

    "reminder that if": 33.333333333333336,

    "that if you": 33.333333333333336,

    "if you fail": 33.333333333333336,

    "you fail to": 33.333333333333336,

    "fail to complete": 33.333333333333336,

    "to complete you": 33.333333333333336,

    "complete you will": 33.333333333333336,

    "you will pass": 33.333333333333336,

    "will pass the": 33.333333333333336,

    "pass the test": 33.333333333333336,

    "this is a reminder": 25.0,

    "is a reminder that": 25.0,

    "a reminder that if": 25.0,

    "reminder that if you": 25.0,

    "that if you fail": 25.0,

    "if you fail to": 25.0,

    "you fail to complete": 25.0,

    "fail to complete you": 25.0,

    "to complete you will": 25.0,

    "complete you will pass": 25.0,

    "you will pass the": 25.0,

    "will pass the test": 25.0

  }

}
 
 No newline at end of file

README.md

+2 −0
Original line number Diff line number Diff line
@@ -36,6 +36,8 @@ In order to have a good working State machine we need to first update the gramma 
#### n-Grams checker

This checker used the crawled data and gave a score by summing up all the n_grams probabilities of phrases in a sentence.



### Proof of Effictiveness

In order to show the effictiveness of our tool we used ChatGBT to write a script that is using a third-party Python library to make the exact same score for each sentence. These can be found in CheckerCorrector/samples/ directory. Both Json generated by our tool and the third party are available.



### Corrector

The current corrector uses the typo corrector which was used in the checker and also the state machine to suggest possible corrections. Corrections of the typo corrector are based on the most similar path through the state machine. In order to find a similar path we are first doing a DFS on the graph starting from the first token and storing all the possible paths. Then, based on the most similar path we decide whether we should change/delete/add a token and suggest a token with a similar role.