num_classes vs num_tokens about madewithml

[GokuMohandas]

Hi @Aman0807, this is a great question and is actually a major source of bugs even in very mature companies. Typically, when I write a function, I want to make it as modular and reusable as possible. So that I can use it for my current context but also for future contexts for my team. In our task here, the number of classes happens to be the same as the number of tokens but in someone else's task, they may still require padding but tokens may not make any sense to them if they're not doing exactly our task. So when writing functions, it's a good practice to use names that are universal (but still descriptive).

I've often seen functions written using specific variables that exist for the current context it's been written for. And then when others reuse the same function (copy/paste or from an internal ML library) they get errors because that specific variable may not exist or it doesn't make sense for their context.