data/databases/README.md

• This folder contains all the SQL databases for the different processed data along with their raw data.

• The databases are named after the arXiv category and the format of the generated data.

Each file in this folder is a database containing 2 tables:

• papers

  The papers data from the raw folder that was fed to the model.

  SCHEMA:

  • paper_id TEXT PRIMARY KEY,
  • abstract TEXT,
  • authors TEXT,
  • primary_category TEXT,
  • url TEXT,
  • updated_on TEXT,
  • sentence_count INTEGER

• predictions

  The corresponding model generations stored in the results folder.

  SCHEMA:

  • id INTEGER PRIMARY KEY AUTOINCREMENT,
  • paper_id TEXT,
  • sentence_index INTEGER,
  • tag_type TEXT,
  • concept TEXT,
  • FOREIGN KEY (paper_id) REFERENCES papers(paper_id)

To query any database, open SQLite in your terminal and specify the database name.