Comments (4)
I don't think LLaMA is anything special. It's just something Facebook felt like they had to release at this point in order to keep the share price up.
We will see in the next few months that Large Language models are ten a penny
from llama.
Looks like it's not inconsistent? It betters Chinchilla in every test but it is beaten by PaLM 540B on a couple.
Table 3: Zero shot performance on common-sense reasoning tasks.
Model | Parameters | BoolQ | PIQA | SIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA |
---|---|---|---|---|---|---|---|---|---|
GPT-3 | 175B | 60.5 | 81.0 | - | 78.9 | 70.2 | 68.8 | 51.4 | 57.6 |
Gopher | 280B | 79.3 | 81.8 | 50.6 | 79.2 | 70.1 | - | - | - |
Chinchilla | 70B | 83.7 | 81.8 | 51.3 | 80.8 | 74.9 | - | - | - |
PaLM | 62B | 84.8 | 80.5 | - | 79.7 | 77.0 | 75.2 | 52.5 | 50.4 |
PaLM-cont | 62B | 83.9 | 81.4 | - | 80.6 | 77.0 | - | - | - |
PaLM | 540B | 88.0 | 82.3 | - | 83.4 | 81.1 | 76.6 | 53.0 | 53.4 |
LLaMA | 7B | 76.5 | 79.8 | 48.9 | 76.1 | 70.1 | 72.8 | 47.6 | 57.2 |
LLaMA | 13B | 78.1 | 80.1 | 50.4 | 79.2 | 73.0 | 74.8 | 52.7 | 56.4 |
LLaMA | 33B | 83.1 | 82.3 | 50.4 | 82.8 | 76.0 | 80.0 | 57.8 | 58.6 |
LLaMA | 65B | 85.3 | 82.8 | 52.3 | 84.2 | 77.0 | 78.9 | 56.0 | 60.2 |
from llama.
Thank you so much. I'm a big help. May I ask about the model structure, I'll open a separate issue. Let's create a new issue! @gdmcdonald
from llama.
Thanks for calling out the benchmark issue - we will make sure this is taken into account for future papers/models. thanks!
from llama.
Related Issues (20)
- Research dedicated license? HOT 2
- Bug HOT 1
- "$CPU_ARCH" not found HOT 2
- Unable to download LLAMA models from https://llama.meta.com/llama-downloads HOT 1
- Download.sh does nothing HOT 2
- No special tokens added in tokenizer
- Downlaod.sh is throwing 403 Foribdeen error, when using a freshly generated URL/token HOT 2
- Getting 400 error on https://llama3-1.llamameta.net/Meta-Llama-3.1-405B-MP8/consolidated.00.pth HOT 1
- How to to run Meta-Llama-3.1-70B-Instruct on the MATH TEST HOT 1
- AnT1nG-Meta-llana
- Llama
- A problem with tokenizer.model from HuggingFace HOT 2
- LLaMA 3.1 Model with Image Input is Faster than Text-only Input
- Llama 3.1: The output text is truncated HOT 2
- Access to checkpoints
- Do you open source of your internal evaluations library ?
- Meta AI issue to append duplicated messages after the answer when questions have context
- Meta-Llama-3.1-70B-Instruct does not appear to have a file named config.json HOT 2
- Colons in filenames when using llama download make them incompatible with windows.
- Remove person from background
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llama.