File size: 3,743 Bytes
70f2ddd 9d793d0 a040d47 9d793d0 fb591e6 9d793d0 71a4132 a040d47 9d793d0 79d1413 9d793d0 79d1413 9d793d0 8ebaba1 9d793d0 a040d47 9d793d0 a040d47 57c2b26 9d793d0 a040d47 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
---
title: Pdf Translator For Human
emoji: 🦀
colorFrom: indigo
colorTo: green
sdk: streamlit
sdk_version: 1.42.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: PDF Translator powered by local llm, side by side reading
---
# PDF Translator for Human: A PDF Reader/Translator with Local LLM/ChatGPT or Google
## Use Case
There is tons of PDF reader/translator with AI supported. However none of them meets my need. I hope it could run totally on local with local LLMs.
I hope to read both the original PDF and the translated pages side by side.
Also I don't like to translate a 1000 pages long PDF file all at once, it costs lots of time and tokens. And most of the time, I never complete reading through all contents of a long paper.
## Features in PDF Translator for Human
You can read both the original PDF file and the translated content side by side.
The local/remote translation API is invoked on a per-page basis as needed, triggered by page turns during reading.
## Snapshot
## Huggingface Space
https://huggingface.co/spaces/davideuler/pdf-translator-for-human
## Supported translators and LLMs:
* Google Translator (NO need api-key, it it totally free)
* Local deployed LLMs (ollama, llama.cpp, mlx_lm ... etc.)
* ChatGPT
* DeepSeek (Use the OpenAI Compatible endpoint at https://api.deepseek.com/v1)
* Qwen (Use the OpenAI Compatible endpoint)
* Other OpenAI Compatible LLMs like GLM/Moonshot etc.
## Start the Web Application for PDF Translator for Human
``` bash
./run_translator_web.sh
# or just start the streamlit application if you have run the previous script:
streamlit run app.py
```
## Notes on deployment and starting a local llm inference service
### Option 1.Start local llm By mlx_lm (works on Mac Sillicon.)
Here I download aya-expanse-8b 4bit as an example.
``` Bash
# download mlx models from huggingface to local folder
git clone https://huggingface.co/mlx-community/aya-expanse-8b-4bit
# install mlx_lm
pip install mlx_lm
# start the server
mlx_lm.server --model ./aya-expanse-8b-4bit --port 8080
```
### Option 2. By llama.cpp (Works on CPU/GPU/Mac Machines)
Llama.cpp works on CPU machines and Mac Intel/Sillicon machines, you need 48GB memories for aya-expanse-32b-q4_k_m.gguf.
``` Bash
# download gguf models from huggingface to local folder
wget https://hf-mirror.co/bartowski/aya-expanse-32b-GGUF/resolve/main/aya-expanse-32b-Q4_K_M.gguf -O aya-expanse-32b-Q4_K_M.gguf
# download llama.cpp and install llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir -p build && cmake -B build
cmake --build build --config Release -j 12
# start llama.cpp server
./llama-server -m ~/models/aya-expanse-32b-Q4_K_M.gguf --port 8080
```
### Options 3. Local inference service by ollama/vLLM and other application such as LMStudio
Please read the official guide for you LLM inferencing tool.
### Option 4. Note on using OpenAI Compatible LLM service provider
For example, run the following command before start the streamlit application to enable translation by deepseek :
``` bash
export OPENAI_MODEL=deepseek-chat
export OPENAI_API_BASE=https://api.deepseek.com/v1
export OPENAI_API_KEY=sk-xxxx
```
Run the following command before start the streamlit application to enable translation by moonshot :
``` bash
export OPENAI_MODEL=moonshot-v1-8k
export OPENAI_API_BASE=https://api.moonshot.cn/v1
export OPENAI_API_KEY=sk-xxxx
```
## Acknowlegement
https://github.com/nidhaloff/deep-translator
The project is based on the awesome deep-translator. Thanks to the excellent work in the original project, I can integrate it to the pdf translator tool.
Pull Requests are welcome. |