🌟 𝐋𝐋𝐌 𝐏𝐫𝐨𝐯𝐢𝐝𝐞𝐫'𝐬 𝐑𝐞𝐥𝐞𝐚𝐬𝐞 (2024 1H) & 📝 2024 𝐋𝐋𝐌 𝐒𝐮𝐫𝐯𝐞𝐲 (on Training / Data / RAG / Serving / Agent)

🌟 𝐋𝐋𝐌 𝐏𝐫𝐨𝐯𝐢𝐝𝐞𝐫’𝐬 𝐑𝐞𝐥𝐞𝐚𝐬𝐞 (2024 1H)

I have selected 21 LLMs from various providers for 2024 1H. [📣Release news] and [📋Tech report] are in this post. [📘API docs] and [🤗HF models] are in the comments.

1. 𝐐𝐰𝐞𝐧-2 (​Alibaba Group​, 2024.06.07)
   • • 📣News: https://qwenlm.github.io/blog/qwen2/
2. 𝐒𝐨𝐥𝐚𝐫-𝐌𝐢𝐧𝐢-𝐣𝐚 (​Upstage​, 2024.05.22)
   • • 📣News: https://www.upstage.ai/feed/tech/solar-mini-chat-ja
3. 𝐘𝐢-𝐋𝐚𝐫𝐠𝐞 (​01.AI​, 2024.05.13)
   • • 📣News: https://x.com/01AI_Yi/status/1789929378467426794
4. 𝐘𝐢-1.5 (​01.AI​, 2024.05.13)
   • • 📣News: https://x.com/01AI_Yi/status/1789869537317540016
   • • 📋arXiv: https://arxiv.org/abs/2403.04652
5. 𝐆𝐏𝐓-4𝐨 (​OpenAI​, 2024.05.13)
   • • 📣News: https://openai.com/index/hello-gpt-4o/
6. 𝐐𝐰𝐞𝐧-𝐌𝐚𝐱 (​Alibaba Group​, 2024.05.11)
   • • 📣News: https://qwenlm.github.io/blog/qwen-max-0428/
7. 𝐃𝐞𝐞𝐩𝐒𝐞𝐞𝐤-𝐕2 (DeepSeek, 2024.05.07)
   • • 📣News: https://x.com/deepseek_ai/status/1787478986731429933
   • • 📋arXiv: https://arxiv.org/abs/2405.04434
8. 𝐒𝐧𝐨𝐰𝐟𝐥𝐚𝐤𝐞-𝐀𝐫𝐜𝐭𝐢𝐜 (​Snowflake​, 2024.04.24)
   • • 📣News: https://www.snowflake.com/blog/arctic-open-efficient-foundation-language-models-snowflake/
9. 𝐏𝐡𝐢-3 (​Microsoft​, 2024.04.22)
   • • 📣News: https://azure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms/
   • • 📋arXiv: https://arxiv.org/abs/2404.14219
10. 𝐋𝐥𝐚𝐦𝐚-3 (​Meta Facebook​, 2024.04.18)
    • • 📣News: https://ai.meta.com/blog/meta-llama-3/
11. 𝐌𝐢𝐱𝐭𝐫𝐚𝐥-8𝐱22𝐁 (​Mistral AI​, 2024.04.17)
    • • 📣News: https://mistral.ai/news/mixtral-8x22b/
12. 𝐑𝐞𝐤𝐚-𝐂𝐨𝐫𝐞 (​Reka AI​​, 2024.04.15)
    • • 📣News: https://www.reka.ai/news/reka-core-our-frontier-class-multimodal-language-model
    • • 📋arXiv: https://arxiv.org/abs/2404.12387
13. 𝐂𝐨𝐦𝐦𝐚𝐧𝐝-𝐑-𝐏𝐥𝐮𝐬 (​Cohere​, 2024.04.04)
    • • 📣News: https://cohere.com/blog/command-r-plus-microsoft-azure
14. 𝐃𝐁𝐑𝐗 (​Databricks​, 2024.03.27)
    • • 📣News: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
15. 𝐆𝐞𝐦𝐢𝐧𝐢-1.5 (​Google​, 2024.03.08)
    • • 📣News: https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/
    • • 📋arXiv: https://arxiv.org/abs/2403.05530
16. 𝐂𝐥𝐚𝐮𝐝𝐞-3 (​Anthropic​, 2024.03.04)
    • • 📣News: https://www.anthropic.com/news/claude-3-family
17. 𝐌𝐢𝐬𝐭𝐫𝐚𝐥-𝐋𝐚𝐫𝐠𝐞 (​Mistral AI​, 2024.02.26)
    • • 📣News: https://mistral.ai/news/mistral-large/
18. 𝐆𝐞𝐦𝐦𝐚 (​Google​, 2024.02.21)
    • • 📣News: https://blog.google/technology/developers/gemma-open-models/
    • • 📋arXiv: https://arxiv.org/abs/2403.08295
19. 𝐐𝐰𝐞𝐧-1.5 (​Alibaba Group​, 2024.02.04)
    • • 📣News: https://qwenlm.github.io/blog/qwen1.5/
20. 𝐒𝐨𝐥𝐚𝐫-𝐌𝐢𝐧𝐢 (​Upstage​, 2024.01.25)
    • • 📣News: https://www.upstage.ai/feed/product/solarmini-performance-report
21. 𝐒𝐨𝐥𝐚𝐫-10.7𝐁 (​Upstage​, 2023.12.23)
    • • 📣News: https://www.upstage.ai/feed/press/solar-10-7b-emerges-as-worlds-top-pre-trained-llm
    • • 📋arXiv: https://arxiv.org/abs/2312.15166

---

📝 2024 𝐋𝐋𝐌 𝐒𝐮𝐫𝐯𝐞𝐲 (on Training / Data / RAG / Serving / Agent)

I’ll read all below !!

𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠

📌 A Survey on Self-Evolution of Large Language Models (2024.04.22)

• • arXiv: https://arxiv.org/abs/2404.14387
• • Github: https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/Awesome-Self-Evolution-of-LLM

📌 Continual Learning of Large Language Models: A Comprehensive Survey (2024.04.25)

• • arXiv: https://arxiv.org/abs/2404.16789
• • Github: https://github.com/Wang-ML-Lab/llm-continual-learning-survey

📌 Continual Learning with Pre-Trained Models: A Survey (2024.01.29)

• • arXiv: https://arxiv.org/abs/2401.16386
• • Github: https://github.com/sun-hailong/LAMDA-PILOT

D𝐚𝐭𝐚

📌 Datasets for Large Language Models: A Comprehensive Survey (2024.02.28)

• • arXiv: https://arxiv.org/pdf/2402.18041
• • Github: https://github.com/lmmlzn/Awesome-LLMs-Datasets

📌 A Survey on Data Selection for Language Models (2024.02.26)

• • arXiv: https://arxiv.org/abs/2402.16827
• • Github: https://github.com/alon-albalak/data-selection-survey

📌 A Survey on Data Selection for LLM Instruction Tuning (2024.02.04)

• • arXiv: https://arxiv.org/abs/2402.05123
• • Github: https://github.com/Bolin97/awesome-instruction-selector

𝐑𝐀𝐆

📌 RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing (2024.04.30)

• • arXiv: https://arxiv.org/abs/2404.19543
• • Github: https://github.com/2471023025/RALM_Survey

📌 Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024.02.29)

• • arXiv: https://arxiv.org/abs/2402.19473
• • Github: https://github.com/hymie122/RAG-Survey

📌 Retrieval-Augmented Generation for Large Language Models: A Survey (2023.12.18)

• • arXiv: https://arxiv.org/abs/2312.10997
• • Github: https://github.com/Tongji-KGLLM/RAG-Survey

𝐒𝐞𝐫𝐯𝐢𝐧𝐠

📌 LLM Inference Unveiled: Survey and Roofline Model Insights (2024.02.26)

• • arXiv: https://arxiv.org/abs/2402.16363
• • Github: https://github.com/hahnyuan/LLM-Viewer

📌 A Survey on Effective Invocation Methods of Massive LLM Services (2024.02.05)

• • arXiv: https://arxiv.org/abs/2402.03408
• • Github: https://github.com/W-caner/Effective-strategy-for-LMaas

📌 Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models (2024.01.01)

• • arXiv: https://arxiv.org/abs/2401.00625
• • Github: https://github.com/tiingweii-shii/Awesome-Resource-Efficient-LLM-Papers

𝐀𝐠𝐞𝐧𝐭

📌 Large Multimodal Agents: A Survey (2024.02.23)

• • arXiv: https://arxiv.org/abs/2402.15116
• • Github: https://github.com/jun0wanan/awesome-large-multimodal-agents

📌 Large Language Model based Multi-Agents: A Survey of Progress and Challenges (2024.01.21)

• • arXiv: https://arxiv.org/abs/2402.01680
• • Github: https://github.com/taichengguo/LLM_MultiAgents_Survey_Papers

📌 Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security (2024.01.10)

• • arXiv: https://arxiv.org/abs/2401.05459
• • Github: https://github.com/MobileLLM/Personal_LLM_Agents_Survey