Spaces:

peter2000
/

policy_test

Runtime error

peter2000 commited on Aug 3, 2022

Commit

d08905f

•

1 Parent(s): acafedc

Update scripts/clean.py

Files changed (1) hide show

scripts/clean.py CHANGED Viewed

@@ -84,7 +84,7 @@ def preprocessing(document):
         clean_header_footer=True,
         split_by="sentence",
         split_length=3,
-        split_respect_sentence_boundary=True,
         split_overlap=1
     )
     for i in document:

         clean_header_footer=True,
         split_by="sentence",
         split_length=3,
+        split_respect_sentence_boundary=False,
         split_overlap=1
     )
     for i in document: