|
|
--- |
|
|
license: mit |
|
|
--- |
|
|
|
|
|
# SRTK Scorer |
|
|
|
|
|
This model is a trained scorer for [SRTK](https://github.com/happen2me/subgraph-retrieval-toolkit). It is used to compare the similarity between a query and the expansion path at the time of subgraph retrieval. |
|
|
|
|
|
## Training Information |
|
|
|
|
|
It is initialized with `roberta-base`. It is trained jointly on the following datasets: |
|
|
|
|
|
- [WebQSP for Freebase](https://www.microsoft.com/en-us/download/details.aspx?id=52763) |
|
|
- [SimpleQuestionsWikidata for Wikidata](https://github.com/askplatypus/wikidata-simplequestions) |
|
|
- [SimpleDBpediaQA](https://github.com/castorini/SimpleDBpediaQA) |
|
|
|
|
|
It achieves an answer coverage rate of 0.9728 on SimpleQuestionsWikidata (depth 1) 0.8501 on WebQSP test set (depth 2) with a beam width of only 2! |
|
|
|
|
|
## Usage Example |
|
|
|
|
|
First install the package: |
|
|
|
|
|
```bash |
|
|
pip install srtk |
|
|
``` |
|
|
|
|
|
Then you can retrieve subgraphs with the help of this scorer: |
|
|
|
|
|
```bash |
|
|
srtk retrieve -i data/wikidata-simplequestions/intermediate/scores_test.jsonl \ |
|
|
-o artifacts/subgraphs/wikidata-simple-contrast \ |
|
|
-e http://localhost:1234/api/endpoint/sparql \ |
|
|
--scorer-model-path drt/srtk-scorer \ |
|
|
--scorer --beam-width 2 --max-depth 1 --evaluate |
|
|
``` |
|
|
|
|
|
## Limitations |
|
|
|
|
|
As both SimpleQuestionsWikidata and SimpleDBpediaQA contain only one-hop relations, the model tends to stop at one-hop when you retrieve subgraphs on Wikidata and DBpedia. We will release a updated version of the model that is trained on a more diverse dataset in the future. |
|
|
|
|
|
## License |
|
|
|
|
|
MIT |
|
|
|
