Running Llama on Android Device
Are there any instructions or documentation on how to run Llama model on android device ?
We're actively working to provide more information on this topic.
@shreyajn
Any update?
One more question. How do you process with tokinzer? I want to use tokenizer to get text from output_ids. Could you give me a reference?
@AICoding91 , please take a look at our instructions on running Llama models on device via AI Hub: https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/llama_v2_7b_chat_quantized/README.md
@meghan3
Actually, I want to use Llama models in Android app via Android API or C++ API with JNI. But currently I didn't see anything about this. Could you share some information about this item in Qualcomm?
Hi @AICoding91 , we're working on releasing an Android app in the next few weeks, there are certain ecosystem complexities delaying us, these will be outlined in the ReadMe. If you haven't already, we encourage you to join our Slack Community to be notified when the latest models, features, apps etc are released.
@meghan3
Thank you for sharing the status. I will join Slack community.
Great! Also, just a heads up our ChatApp was released; https://github.com/quic/ai-hub-apps/tree/main/apps/android/ChatApp