DeDeckerThomas
commited on
Commit
Β·
93edba3
1
Parent(s):
fd2369f
Update README.md
Browse files
README.md
CHANGED
@@ -14,7 +14,7 @@ metric:
|
|
14 |
# π Keyphrase Extraction model: KBIR-inspec
|
15 |
Keyphrase extraction is a technique in text analysis where you extract the important keyphrases from a text. Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
16 |
Currently, classical machine learning methods, that use statistics and linguistics, are widely used for the extraction process. The fact that these methods have been widely used in the community has the advantage that there are many easy-to-use libraries.
|
17 |
-
Now with the recent innovations in deep learning methods (such as recurrent neural networks and transformers, GANS, β¦),
|
18 |
|
19 |
|
20 |
## π Model Description
|
@@ -26,9 +26,9 @@ Paper: https://arxiv.org/abs/2112.08547
|
|
26 |
### β How to use
|
27 |
```python
|
28 |
# Define post_process functions
|
29 |
-
def concat_tokens_by_tag(
|
30 |
keyphrase_tokens = []
|
31 |
-
for id, label in
|
32 |
if label == "B":
|
33 |
keyphrase_tokens.append([id])
|
34 |
elif label == "I":
|
@@ -65,13 +65,13 @@ model = AutoModelForTokenClassification.from_pretrained(model_name)
|
|
65 |
```python
|
66 |
# Inference
|
67 |
text = """
|
68 |
-
|
69 |
from a text. Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
70 |
Currently, classical machine learning methods, that use statistics and linguistics, are widely used
|
71 |
for the extraction process. The fact that these methods have been widely used in the community has
|
72 |
the advantage that there are many easy-to-use libraries. Now with the recent innovations in
|
73 |
deep learning methods (such as recurrent neural networks and transformers, GANS, β¦),
|
74 |
-
|
75 |
and context of a document, which is quite an improvement.
|
76 |
""".replace("\n", "")
|
77 |
|
@@ -98,19 +98,19 @@ print(extracted_kps)
|
|
98 |
|
99 |
```
|
100 |
***** Input Document *****
|
101 |
-
|
102 |
from a text. Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
103 |
Currently, classical machine learning methods, that use statistics and linguistics, are widely used
|
104 |
for the extraction process. The fact that these methods have been widely used in the community has
|
105 |
the advantage that there are many easy-to-use libraries. Now with the recent innovations in
|
106 |
deep learning methods (such as recurrent neural networks and transformers, GANS, β¦),
|
107 |
-
|
108 |
and context of a document, which is quite an improvement.
|
109 |
|
110 |
***** Prediction *****
|
111 |
-
['Artificial Intelligence' 'GANS' '
|
112 |
-
'classical machine learning
|
113 |
-
'
|
114 |
'semantics' 'statistics' 'text analysis' 'transformers']
|
115 |
```
|
116 |
|
|
|
14 |
# π Keyphrase Extraction model: KBIR-inspec
|
15 |
Keyphrase extraction is a technique in text analysis where you extract the important keyphrases from a text. Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
16 |
Currently, classical machine learning methods, that use statistics and linguistics, are widely used for the extraction process. The fact that these methods have been widely used in the community has the advantage that there are many easy-to-use libraries.
|
17 |
+
Now with the recent innovations in deep learning methods (such as recurrent neural networks and transformers, GANS, β¦), keyphrase extraction can be improved. These new methods also focus on the semantics and context of a document, which is quite an improvement.
|
18 |
|
19 |
|
20 |
## π Model Description
|
|
|
26 |
### β How to use
|
27 |
```python
|
28 |
# Define post_process functions
|
29 |
+
def concat_tokens_by_tag(keyphrases):
|
30 |
keyphrase_tokens = []
|
31 |
+
for id, label in keyphrases:
|
32 |
if label == "B":
|
33 |
keyphrase_tokens.append([id])
|
34 |
elif label == "I":
|
|
|
65 |
```python
|
66 |
# Inference
|
67 |
text = """
|
68 |
+
Keyphrase extraction is a technique in text analysis where you extract the important keyphrases
|
69 |
from a text. Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
70 |
Currently, classical machine learning methods, that use statistics and linguistics, are widely used
|
71 |
for the extraction process. The fact that these methods have been widely used in the community has
|
72 |
the advantage that there are many easy-to-use libraries. Now with the recent innovations in
|
73 |
deep learning methods (such as recurrent neural networks and transformers, GANS, β¦),
|
74 |
+
keyphrase extraction can be improved. These new methods also focus on the semantics
|
75 |
and context of a document, which is quite an improvement.
|
76 |
""".replace("\n", "")
|
77 |
|
|
|
98 |
|
99 |
```
|
100 |
***** Input Document *****
|
101 |
+
Keyphrase extraction is a technique in text analysis where you extract the important keyphrases
|
102 |
from a text. Since this is a time-consuming process, Artificial Intelligence is used to automate it.
|
103 |
Currently, classical machine learning methods, that use statistics and linguistics, are widely used
|
104 |
for the extraction process. The fact that these methods have been widely used in the community has
|
105 |
the advantage that there are many easy-to-use libraries. Now with the recent innovations in
|
106 |
deep learning methods (such as recurrent neural networks and transformers, GANS, β¦),
|
107 |
+
keyphrase extraction can be improved. These new methods also focus on the semantics
|
108 |
and context of a document, which is quite an improvement.
|
109 |
|
110 |
***** Prediction *****
|
111 |
+
['Artificial Intelligence' 'GANS' 'Keyphrase extraction'
|
112 |
+
'classical machine learning' 'deep learning methods'
|
113 |
+
'keyphrase extraction' 'linguistics' 'recurrent neural networks'
|
114 |
'semantics' 'statistics' 'text analysis' 'transformers']
|
115 |
```
|
116 |
|