2024-03-29 14:20:49.463961: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:20:50.507795: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 5.86 GiB. GPU 0 has a total capacity of 14.58 GiB of which 1.76 GiB is free. Including non-PyTorch memory, this process has 12.81 GiB memory in use. Of the allocated memory 11.71 GiB is allocated by PyTorch, and 1008.80 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:31:17.459384: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:31:18.518981: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 5.86 GiB. GPU 0 has a total capacity of 14.58 GiB of which 2.47 GiB is free. Including non-PyTorch memory, this process has 12.10 GiB memory in use. Of the allocated memory 11.71 GiB is allocated by PyTorch, and 278.80 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:36:15.933048: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:36:16.966744: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 5.86 GiB. GPU 0 has a total capacity of 14.58 GiB of which 5.63 GiB is free. Including non-PyTorch memory, this process has 8.95 GiB memory in use. Of the allocated memory 8.59 GiB is allocated by PyTorch, and 234.51 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:38:49.739939: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:38:50.770137: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 365, in forward
    context_layer = torch.matmul(attention_probs, value_layer)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 500.00 MiB. GPU 0 has a total capacity of 14.58 GiB of which 285.56 MiB is free. Including non-PyTorch memory, this process has 14.30 GiB memory in use. Of the allocated memory 13.96 GiB is allocated by PyTorch, and 222.09 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:42:21.299532: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:42:22.362964: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 365, in forward
    context_layer = torch.matmul(attention_probs, value_layer)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 500.00 MiB. GPU 0 has a total capacity of 14.58 GiB of which 285.56 MiB is free. Including non-PyTorch memory, this process has 14.30 GiB memory in use. Of the allocated memory 13.96 GiB is allocated by PyTorch, and 222.09 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:48:12.298004: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:48:13.329416: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Running on local URL:  http://0.0.0.0:80
Running on public URL: https://008ca76c2bb7f8d8a3.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.007239958671520294, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.005940472653387939, 'She was the first woman to earn a geology degree from Stanford.': 0.0016036026130179831, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.001537302361237576, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.007227867941129461, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -0.0003420683510965876, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.009836457467468768, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.005398886066759868} bc
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.007024535420434749, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.005433933632620999, 'She was the first woman to earn a geology degree from Stanford.': 0.0033503657592824465, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.0012667157053936522, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.007406581188202247, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -0.0006685564234160865, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.009190228364350466, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.004699842541408435} bc
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.641953608456155, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': 0.020200923452086798, 'She was the first woman to earn a geology degree from Stanford.': 0.008136189058261252, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.12504063362482074, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': 0.14466029601373961, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': 0.045496763632525375, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': 0.11435786746768793, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.3560611292221768} quillbot
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.049232424744256965, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.0808599351295588, 'She was the first woman to earn a geology degree from Stanford.': -0.028306312264799082, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.018576473883078034, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.0658758038308371, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': 0.00520141594810037, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.06700218547318215, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.11886694361432464} bc
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.07048027659860119, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.07512228868644406, 'She was the first woman to earn a geology degree from Stanford.': -0.04560898943130033, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.01102573043004705, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.06753051178176432, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -0.0016847880819046478, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.06913938144762188, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.13576338155813136} bc
2024-03-29 15:01:50.768841: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 15:01:51.796519: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/optimum/bettertransformer/models/encoder_models.py:301: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ../aten/src/ATen/NestedTensorImpl.cpp:177.)
  hidden_states = torch._nested_tensor_from_mask(hidden_states, ~attention_mask)
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Running on local URL:  http://0.0.0.0:80
Running on public URL: https://e095d1a53e42b16b1b.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.007083724433403481, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.005900632715473411, 'She was the first woman to earn a geology degree from Stanford.': 0.00288471219406703, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.0012162868179568342, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.006270546763081995, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -6.844510148763104e-05, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.008883191796269094, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.005504050009961782} bc
Original BC scores: AI: 6.408023001114316e-09, HUMAN: 1.0
Calibration BC scores: AI: 0.0, HUMAN: 1.0
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 6.408023001114316e-09, HUMAN: 1.0
Calibration BC scores: AI: 0.0, HUMAN: 1.0
Starting MC
MC Score: {'OpenAI GPT': 0.0, 'Mistral': 0.0, 'CLAUDE': 0.0, 'Gemini': 0.0, 'LLAMA 2': 0.0}
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.599086635981887, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': 0.08136319631271138, 'She was the first woman to earn a geology degree from Stanford.': 0.02834857510284846, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.061459884832511476, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': 0.16672173091342543, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': 0.0820923392682848, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': 0.13399838230662856, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.3821691921261263} quillbot
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
2024-03-29 19:06:50.019873: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 19:06:51.074912: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/optimum/bettertransformer/models/encoder_models.py:301: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ../aten/src/ATen/NestedTensorImpl.cpp:177.)
  hidden_states = torch._nested_tensor_from_mask(hidden_states, ~attention_mask)
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
Token indices sequence length is longer than the specified maximum sequence length for this model (881 > 512). Running this sequence through the model will result in indexing errors
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Running on local URL:  http://0.0.0.0:80
Running on public URL: https://94e72aa3904122b29c.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
Original BC scores: AI: 1.0, HUMAN: 1.7993962986295742e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 1.0, HUMAN: 1.7993962986295742e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 0.9994855300308254, 'Mistral': 5.1601761922609244e-11, 'CLAUDE': 8.48403323344426e-11, 'Gemini': 2.8437433518348655e-10, 'LLAMA 2': 9.498188443606356e-11}
Original BC scores: AI: 1.0, HUMAN: 1.7997051626750249e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Original BC scores: AI: 0.6732428669929504, HUMAN: 0.3267570436000824
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 0.5024993419647217, HUMAN: 0.49750062823295593
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 0.7561723589897156, HUMAN: 0.24382765591144562
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 1.0, HUMAN: 1.8036925286679661e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Original BC scores: AI: 0.7560267448425293, HUMAN: 0.24397319555282593
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 0.989621639251709, HUMAN: 0.010378347709774971
Calibration BC scores: AI: 0.5178571428571429, HUMAN: 0.4821428571428571
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 0.9994855298515718, 'Mistral': 4.535480345181983e-11, 'CLAUDE': 2.261075985034601e-10, 'Gemini': 3.1878497183516737e-10, 'LLAMA 2': 1.0480460580159845e-10}
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 0.9994855298515718, 'Mistral': 4.535480345181983e-11, 'CLAUDE': 2.261075985034601e-10, 'Gemini': 3.1878497183516737e-10, 'LLAMA 2': 1.0480460580159845e-10}
{'Add-on features now encompass AI and Source Identification, leveraging forensic linguistic analysis to ascertain the origin, reliability, and authenticity of content.': -0.15216478135731262, 'These advanced tools can distinguish between human and AI-generated material, pinpointing the specific AI models employed in creation.': -0.05895885252560595, 'This enhancement bolsters the ability to assess content trustworthiness effectively.': 0.03353039204460538} bc
Original BC scores: AI: 0.998177170753479, HUMAN: 0.0018228011904284358
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 0.998177170753479, HUMAN: 0.0018228011904284358
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Starting MC
MC Score: {'OpenAI GPT': 0.6614420057714218, 'Mistral': 2.7132188074993352e-11, 'CLAUDE': 1.2335682936047867e-10, 'Gemini': 1.7620911369483686e-10, 'LLAMA 2': 1.714725314469418e-10}
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.006323229799663152, 'They can even identify the precise AI models that were used to create the content.': 0.017586576131630234} bc
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.43261755952898956, 'They can even identify the precise AI models that were used to create the content.': 0.10732631520197373} quillbot
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.4322117278076279, 'They can even identify the precise AI models that were used to create the content.': 0.10778412185868685} quillbot
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.43300422387049115, 'They can even identify the precise AI models that were used to create the content.': 0.10687924275434384} quillbot
{'Add-on feat ures now encompass AI and Source Identifi cation, leveraging for ensic linguistic analysis to ascertain the origin, reliability, and authen ticity of content.': -0.16172325612226013, 'These advanc ed tools can distinguish between human and AI-generated material, pin pointing the specific AI models employed in creation.': -0.06511130357854991, 'This enhance ment bolsters the ability to assess content trust worthiness effectively.': 0.05332794099561823} bc
{'Add-on feat ures now encompass AI and Source Identifi cation, leveraging for ensic linguistic analysis to ascertain the origin, reliability, and authen ticity of content.': -0.16378145994849636, 'These advanc ed tools can distinguish between human and AI-generated material, pin pointing the specific AI models employed in creation.': -0.06739973523793355, 'This enhance ment bolsters the ability to assess content trust worthiness effectively.': 0.05366690466131973} bc
Original BC scores: AI: 0.995067834854126, HUMAN: 0.004932152573019266
Calibration BC scores: AI: 0.5957446808510638, HUMAN: 0.4042553191489362
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 0.995067834854126, HUMAN: 0.004932152573019266
Calibration BC scores: AI: 0.5957446808510638, HUMAN: 0.4042553191489362
Starting MC
MC Score: {'OpenAI GPT': 0.5957441340683721, 'Mistral': 2.0416833660118585e-10, 'CLAUDE': 5.001776967436859e-07, 'Gemini': 2.5271727453711155e-08, 'LLAMA 2': 2.1129099166428725e-08}
Original BC scores: AI: 0.00025900782202370465, HUMAN: 0.9997410178184509
Calibration BC scores: AI: 0.04296875, HUMAN: 0.95703125
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 0.00025900782202370465, HUMAN: 0.9997410178184509
Calibration BC scores: AI: 0.04296875, HUMAN: 0.95703125
Starting MC
MC Score: {'OpenAI GPT': 0.025428532807609403, 'Mistral': 1.6376084024317497e-09, 'CLAUDE': 1.6831211047289287e-06, 'Gemini': 1.8230926181583228e-06, 'LLAMA 2': 0.017536709341059307}
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 680, in __call__
    await self.app(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 78, in wrapped_app
    await response(scope, receive, sender)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/responses.py", line 151, in __call__
    await send(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 50, in sender
    await send(message)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 161, in _send
    await send(message)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 489, in send
    output = self.conn.send(event=response)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_connection.py", line 512, in send
    data_list = self.send_with_data_passthrough(event)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_connection.py", line 537, in send_with_data_passthrough
    self._process_event(self.our_role, event)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_connection.py", line 272, in _process_event
    self._cstate.process_event(role, type(event), server_switch_event)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_state.py", line 293, in process_event
    self._fire_event_triggered_transitions(role, _event_type)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_state.py", line 311, in _fire_event_triggered_transitions
    raise LocalProtocolError(
h11._util.LocalProtocolError: can't handle event type Response when role=SERVER and state=MUST_CLOSE
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.