sparse-generative-ai

community

AI & ML interests

None defined yet.

Recent Activity

sparse-generative-ai's activity

zhiminy 
posted an update 18 days ago
zhiminy 
posted an update 21 days ago
view post
Post
1163
We're thrilled to introduce our latest project: SE Arena! 🎉

SE Arena is an interactive platform designed to evaluate and compare software engineering chatbots powered by foundation models. With a transparent, open-source leaderboard, support for multi-round conversations, and head-to-head model comparisons, SE Arena is here to bring clarity to the evaluation process for FMs in software engineering tasks.

Check it out here: SE-Arena/Software-Engineering-Arena

We’d love your feedback and contributions! 🚀
zhiminy 
posted an update 6 months ago
zhiminy 
posted an update 6 months ago
view post
Post
1996
Hey everyone!

Our team just dropped something cool! 🎉 We've published a new paper on arxiv diving into the foundation model leaderboards across different platforms. We've analyzed the content, operational workflows, and common issues of these leaderboards. From this, we came up with two new concepts: Leaderboard Operations (LBOps) and leaderboard smells.

We also put together an awesome list with nearly 300 of the latest leaderboards, development tools, and publishing organizations. You can check it out here: https://github.com/SAILResearch/awesome-foundation-model-leaderboards

If you find it useful or interesting, give us a follow or drop a comment. We'd love to hear your thoughts and get your support! ✨

Link to the paper: https://arxiv.org/abs/2407.04065