Spaces:

ML-unipi
/

TermsOfServiceSummarization

Runtime error

tommasobaldi commited on Aug 28, 2022

Commit

530ce31

1 Parent(s): 9c2c31c

working on text splitting

Files changed (1) hide show

app.py CHANGED Viewed

@@ -104,7 +104,7 @@ def main() -> None:
         result = []
         for sentence in sentences:
             tokens = word_tokenize(sentence)
-            if token_count + len(tokens) < 1024:
                 token_count += len(tokens)
                 text_block += " ".join(sentence)
             else:

         result = []
         for sentence in sentences:
             tokens = word_tokenize(sentence)
+            if token_count + len(tokens) < 500:
                 token_count += len(tokens)
                 text_block += " ".join(sentence)
             else: