microsoft

/

table-transformer-structure-recognition

like 164

Follow

Microsoft 4,503

Hmmm, it shouldn't! Would you mind sharing the error you're getting?

Hi,
Sorry I can't paste the error here.
I believe the error arises from a model that is not available offline. Correct me if I'm wrong but it seems like it is requiring to use https://huggingface.co/timm/resnet18.a1_in1k/tree/main as backbone (according to config.json). I have downloaded that model and put it offline, but I'm not sure how I can link it to the TableTransformerForObjectDetection.from_pretrained()

Hi @guidel ,

You can obtain this by doing:

from transformers import TableTransformerForObjectDetection

model =  TableTransformerForObjectDetection.from_pretrained("microsoft/table-transformer-structure-recognition", use_pretrained_backbone=False)

next, you can save the model locally:

model.save_pretrained("path_to_local_directory")

This will allow you to load it again fully offline:

from transformers import TableTransformerForObjectDetection

model =  TableTransformerForObjectDetection.from_pretrained("path_to_local_directory")

In addition, I've been working on converting the TableTransformer checkpoints with a Transformers-based backbone instead of a timm one (using the new AutoBackbone API). It can be instantiated as follows:

from transformers import TableTransformerConfig, ResNetConfig, TableTransformerForObjectDetection

backbone_config = ResNetConfig.from_pretrained("microsoft/resnet-18", out_features["stage1", "stage2", "stage3", "stage4"])
config = TableTransformerConfig(backbone_config=backbone_config, use_timm_backbone=False)
model =  TableTransformerForObjectDetection(config)

I'll push the weights for it shortly to a "no_timm" branch of this repository, which will allow you to do:

from transformers import TableTransformerForObjectDetection

model =  TableTransformerForObjectDetection("microsoft/table-transformer-structure-recognition", revision="no_timm")

This enables you to use the model without requiring the timm library.

Hi @nielsr
could you please push weights to branch no_timm, so it is possible to load model completly offline?

Hi @BonDevil ,

just pushed the weights, you can now use from_pretrained("microsoft/table-transformer-structure-recognition", revision="no_timm") :)

I have downloaded all the files in a folder, but
when I try "model = TableTransformerForObjectDetection.from_pretrained(path_to_the_folder )"，
the error is "raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on."
when I try "model = TableTransformerForObjectDetection.from_pretrained(path, use_pretrained_backbone=False)",
though it doesn't raise a error, I can't get the params this way.
Could you please tell me how to load model offline?

Hi,

I tried this out by first loading the weights from the web, and saving them locally:

from transformers import TableTransformerForObjectDetection

model = TableTransformerForObjectDetection.from_pretrained("microsoft/table-transformer-structure-recognition", revision="no_timm")
model.save_pretrained(".")

Next I turned off my wifi and checked whether the following worked:

from transformers import TableTransformerForObjectDetection

model = TableTransformerForObjectDetection.from_pretrained(".")

and it works for me without issues. There's no need to pass use_pretrained_backbone=False when you pass the no_timm revision.

Hi, I wanted to ask you the following:

1. I did the pretraining of the structure recognition following the instructions in the repository, I tried to update the parameters of the model doing this:

model = TableTransformerForObjectDetection.from_pretrained("microsoft/table-transformer-structure-recognition")
model_keys = list(model.state_dict().keys())
checkpoint = torch.load("model.pth", map_location=torch.device('cpu'))

but there was a warning, none of the parameters name matched, am I doing it wrong ? what is the right way ?

2. To extract the information of the tables, Do I have to use the inference script in the repo ?

Thanks a lot !