GenAi, Deep Learning and Computer Vision

How big do LLMs need to be able to reason?🤔 Microsoft released Orca 2 this week, a 13B Llama-based LLM trained on complex tasks and reasoning. 🧐 Orca's performance comes from its use of synthetically generated data from bigger LLMs. I took a deeper look at paper and extracted the implementation and other insights.

𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻:
1️⃣ Constructed a new dataset (Orca 2) with ~817K samples using prompts from FLAN, and GPT-4 to generate reasoning responses with the help of detailed system prompts.
2️⃣ Grouped prompts into categories based on similarity to assign tailored system prompt that demonstrate different reasoning techniques.
3️⃣ Replaced the original system prompt with a more generic one, to have the model learn the underlying reasoning strategy (Prompt erasing).
4️⃣ Used progressive learning, starting with finetune Llama on FLAN-v2 (1 ep) , retrain on 5M ChatGPT data from Orca 1 (3 ep), combine 1M GPT-4 data from Orca 1 & 800k new Orca 2 data for final training (4 ep).

𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀:
📊 Imitation learning can improve capabilities with enough data.
🔬 Reasoning and longer generations to get the correct answer help smaller models to compete with bigger LLMs.
💫 Prompt Erasing helped Orca to “learn” reasoning
🎯 Lowest hallucination rates of comparable models on summarization
⚙️ Used packing for training, concatenating multiple examples into one sequence.
👨‍🦯 Masked user & system inputs (prompt) and only used generation for loss
🖥 Trained on 32 A100 for 80h

Paper: https://huggingface.co/papers/2311.11045
Model: https://huggingface.co/microsoft/Orca-2-13b

huggingface.co

Paper page - Orca 2: Teaching Small Language Models How to Reason

Join the discussion on this paper page

❤5👍1

www.tgoop.com/awesomedeeplearning/230

5.22K viewsArtificial Intelligence, edited  Nov 27, 2023 at 05:15

How big do LLMs need to be able to reason?🤔 Microsoft released Orca 2 this week, a 13B Llama-based LLM trained on complex tasks and reasoning. 🧐 Orca's performance comes from its use of synthetically generated data from bigger LLMs. I took a deeper look at paper and extracted the implementation and other insights.

𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻:
1️⃣ Constructed a new dataset (Orca 2) with ~817K samples using prompts from FLAN, and GPT-4 to generate reasoning responses with the help of detailed system prompts.
2️⃣ Grouped prompts into categories based on similarity to assign tailored system prompt that demonstrate different reasoning techniques.
3️⃣ Replaced the original system prompt with a more generic one, to have the model learn the underlying reasoning strategy (Prompt erasing).
4️⃣ Used progressive learning, starting with finetune Llama on FLAN-v2 (1 ep) , retrain on 5M ChatGPT data from Orca 1 (3 ep), combine 1M GPT-4 data from Orca 1 & 800k new Orca 2 data for final training (4 ep).

𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀:
📊 Imitation learning can improve capabilities with enough data.
🔬 Reasoning and longer generations to get the correct answer help smaller models to compete with bigger LLMs.
💫 Prompt Erasing helped Orca to “learn” reasoning
🎯 Lowest hallucination rates of comparable models on summarization
⚙️ Used packing for training, concatenating multiple examples into one sequence.
👨‍🦯 Masked user & system inputs (prompt) and only used generation for loss
🖥 Trained on 32 A100 for 80h

Paper: https://huggingface.co/papers/2311.11045
Model: https://huggingface.co/microsoft/Orca-2-13b

BY GenAi, Deep Learning and Computer Vision