Create README.md
Browse filesTo classifier a code source use
import torch
from transformers import RobertaTokenizer, RobertaConfig, RobertaModel
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = RobertaTokenizer.from_pretrained("microsoft/codebert-base")
model = RobertaModel.from_pretrained("microsoft/codebert-base")
model.to(device)
If you want more informations you can go to https://github.com/microsoft/CodeBERT
It tokenizes the source code as well