GenAi, Deep Learning and Computer Vision

After spending almost month with new hype of GenAI (text, LLM not image/video) these are my observations.
Not in particular order and these are 'MY' observations on 'MY' tasks. Your conclusions will differ.

1. We need minimum 7B parameter models. Less than that performance of natural language understanding goes down drastically. More than this you need >24GB gpu.
2. Benchmarks are tricky ... some LLMs are good with some tasks while bad in others. Try to find model which works in your case the best. MPT-7B is still best for my usecases .. even better than Falcon-7B.
3. Prompts change with almost each model. You have to rework many times (There are some solutions around it .. trying to see if they work)
4. For finetuning you need at-least 1 gpu with >24 Gb vram .. 32 or 40 GB one good enough.
5. Finetuning just last few layers to speed up training/finetuning of LLM might not work out well (I tried!)
6. 8-bit, 4-bit model loading for VRAM saving works. For 7B model instead of 16gb, it takes ~10gb and <6gb respectively. BUT .. inference speed goes down drastically. (At-least I faced this issue). Performance also goes down in text understanding tasks.
7. Those like me who are trying to figure out LLM applications for your companies .. be aware for Licensing part. One model trained with other as reference and in case of llama you need original weights ... not a good idea to work in commerical setting.
8. There are 3 types of major LLMs types - basic(like gpt2/3), chat enabled, instruction enabled. Most of the time basic is not usable as it is .. unless you finetune it. Chat versions are the best versions. But most of the time they are not open-source.
9. Not everything needs to be solved with LLMs. Just do not force-fit any solution around LLM .. I have seen the same happening with Deep reinforcement learning some years back. Check this out -> https://lnkd.in/d2mxqhH9
10. I tried out but did not use langchains & vector-dbs. Never needed to ... simple python, embddings and efficient dot product worked for me.
11. LLMs need not have whole world knowledge .. we humans also do not have complete knowledge and still we survive bcz of adaptibility. They just need to know how to use knowledge. I think we can go super smaller in model size if we separate knowledge part somehow.
12. Simulating "thoughts" before answering and NOT just predicting one word after another might be the next wave of innovation.

lnkd.in

LinkedIn

This link will take you to a page that’s not on LinkedIn

❤3👍3

www.tgoop.com/awesomedeeplearning/204

1.85K viewsJun 5, 2023 at 05:52

After spending almost month with new hype of GenAI (text, LLM not image/video) these are my observations.
Not in particular order and these are 'MY' observations on 'MY' tasks. Your conclusions will differ.

1. We need minimum 7B parameter models. Less than that performance of natural language understanding goes down drastically. More than this you need >24GB gpu.
2. Benchmarks are tricky ... some LLMs are good with some tasks while bad in others. Try to find model which works in your case the best. MPT-7B is still best for my usecases .. even better than Falcon-7B.
3. Prompts change with almost each model. You have to rework many times (There are some solutions around it .. trying to see if they work)
4. For finetuning you need at-least 1 gpu with >24 Gb vram .. 32 or 40 GB one good enough.
5. Finetuning just last few layers to speed up training/finetuning of LLM might not work out well (I tried!)
6. 8-bit, 4-bit model loading for VRAM saving works. For 7B model instead of 16gb, it takes ~10gb and <6gb respectively. BUT .. inference speed goes down drastically. (At-least I faced this issue). Performance also goes down in text understanding tasks.
7. Those like me who are trying to figure out LLM applications for your companies .. be aware for Licensing part. One model trained with other as reference and in case of llama you need original weights ... not a good idea to work in commerical setting.
8. There are 3 types of major LLMs types - basic(like gpt2/3), chat enabled, instruction enabled. Most of the time basic is not usable as it is .. unless you finetune it. Chat versions are the best versions. But most of the time they are not open-source.
9. Not everything needs to be solved with LLMs. Just do not force-fit any solution around LLM .. I have seen the same happening with Deep reinforcement learning some years back. Check this out -> https://lnkd.in/d2mxqhH9
10. I tried out but did not use langchains & vector-dbs. Never needed to ... simple python, embddings and efficient dot product worked for me.
11. LLMs need not have whole world knowledge .. we humans also do not have complete knowledge and still we survive bcz of adaptibility. They just need to know how to use knowledge. I think we can go super smaller in model size if we separate knowledge part somehow.
12. Simulating "thoughts" before answering and NOT just predicting one word after another might be the next wave of innovation.

BY GenAi, Deep Learning and Computer Vision