Pulkit Mehta's picture

Pulkit Mehta

pulkitmehtawork
Ā·

AI & ML interests

None yet

Recent Activity

upvoted a collection 1 day ago
DeepSeek-R1
upvoted a paper 7 days ago
More Agents Is All You Need
upvoted a collection 7 days ago
šŸ¤– Agents
View all activity

Organizations

None yet

pulkitmehtawork's activity

upvoted an article 10 days ago
view article
Article

Train 400x faster Static Embedding Models with Sentence Transformers

ā€¢ 125
upvoted an article 24 days ago
New activity in answerdotai/ModernBERT-large about 1 month ago
reacted to santiviquez's post with šŸ‘ 12 months ago
view post
Post
Hey GPT, check yourself...

Here is a black-box method for hallucination detection that shows strong correlation with human annotations. šŸ”„

šŸ’” The idea is the following: ask GPT, or any other powerful LLM, to sample multiple answers for the same prompt, and then ask it if these answers align with the statements in the original output. Make it say yes/no and measure the frequency with which the generated samples support the original statements.

This method is called SelfCheckGPT with Prompt and shows very nice results. šŸ‘€

The downside, we have to do many LLM calls just to evaluate a single generated paragraph... šŸ™ƒ

More details and variations of this method are in the paper: SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning (2308.00436)