bStats syntax for Skript about skript

[TheAbsolutionism]

data can be heavily abused or manipulated

Although this is true, where users could easily

1. Delete the lines of code, disallowing the information to be logged
2. Editing it for the information to be logged in a different service / statistics
3. Manipulating it to have higher statistics

These should be listed more of as a con rather than a write off.
The sole intention of this addition is to allow developers to view the statistics of their script(s), that they publicly release, to have a sense of gratisfaction. It is not to gloat or compete with others as options 2 and 3 benefit those. If users were to do either, I have no clue what they would gain from it besides a false sense of accomplishment.
Even though this feature would not be useful to all users of Skript, it would be a great addition to the ones that would not only use it, but use it correctly.

bStats might retract the implementation in future after realising how wrong this will potentially go.

Only time will tell I presume

[Moderocky]

What's the problem with stats?

We're not against the idea in principle, but we have some major concerns about its practicality.

Currently, our biggest concern is that the data will not provide any accurate or useful value.
We worry that the data will be, at best, inconsistent with reality and, at worst, completely innaccurate.
This is because scripts exist in a chop-and-change ecosystem where users are able to (read: encouraged to) download things and play with them or rewrite them entirely, at which point they're no longer a thing that should be counted in the original statistic.

As a result, numbers are likely to be significantly over-reported, and so the data will not provide any valuable knowledge to the developer.
Statistics are not there to be an ego boost. If the reporting is inaccurate (beyond a respectable error bar) then this whole idea is defunct.

Why are plugins different from scripts?

Plugin statistics are possible because a plugin is a concrete and identifiable thing. It has a namespace, a signature, a manifest. We have version numbers, Git hashes and all sorts of data zipped inside the plugin. It's really easy to tell what is and isn't your plugin. Of course, you can decompile and edit little bits, but you'd never do that on a large scale because it's impractical to do so rather than rebuilding your own plugin from its source.
Compiled plugins avoid the issues above by being annoying to edit, more than anything else.

A script is a text file: it's small, it's easy to edit, you can chop and change bits as you like, and users are encouraged to do so. Very few users download a script and don't go in and edit something. Once a user has downloaded and edited your script, it is no longer yours. It might be completely unrecognisable from yours: the user might delete half of it, or paste bits and pieces into their own scripts.
How do we recognise the identity of a text file? How do we know when it has changed incomparably from its original version?

Can this be resolved?

Sort of. The easiest way to correct for modifications would be to keep an eye on the hash of the file (which we already have) or something based on its constituent structures.

The hard part is determining when something has been unreasonably modified. Obviously, changing a few options at the top of the file shouldn't invalidate the metrics, and I don't see a problem with tweaking a few strings here and there (people like to add themed colours, or add their server name in messages), but once you start to mess with the actual functional content of the script (e.g. removing some commands? Adding your own function? Breaking the script up across multiple files?) then it should no longer report to the same metric.
Personally, I think a tree & branch hash is probably the best way to estimate this, but it'll probably take some experimenting to come up with a reasonable idea.

I think the easiest way of accomplishing this would be to have some kind of publishing command that takes in a script and generates its version-identity hash, which would need to be submitted to the metrics tracking site as an acceptable version.
The site would have to exclude requests that submitted an unexpected hash (i.e. had incomparably changed from a real version).

Security is a non-issue

There is practically no chance of syntax being included to collect custom graphs or charts.
We would provide the basic statistics through the plugin itself: Minecraft version, Skript version, server & player count, maybe some non-invasive insights into the script itself.
The script would just need to include a structure for the metric ID:

metrics id 17044

There would be no avenue for users to submit raw text or data beyond that.
More than this, I don't see what would be valuable to collect (without taking up half the script file with some kind of chart section).

[Moderocky]

I'll try to get here the author of bStats so that they can comment on this, and we can see if they are able to implement such feature when registering a Skript on the website.

It may be possible (or necessary) to send Skript data via our own proxy in which we can do the filtering and such, but then we'd need to set our proxy up with bstats. We have a few ideas but I think it will take time to brainstorm.

And yeah, while usage of custom charts would be really valuable to most, actually adding it in a natural-language syntax such as skript might be hard (for example, to build a Drilldown Pie Chart you need to build a 3 dimensional map <string <string, int>> for the category, value and weight).

Yeah, my worry is that you will have to spend more of your file on setting up the chart than you actually have functional script.

if in the end bStats is not able to add these features we might require to track down skripts, we could make our own platform for this type of tracking, and design it as we like.

This is also a possibility and it's something that we explored about 3 years ago (alongside some kind of skript package manager), it's a lot of work to build and maintain though, and I couldn't find any suppliers who would be willing to sponsor the hosting/networking/proxy costs long-term.
We've got lots of options, we just need to find something that can work with our ecosystem.

[ahmadmsaleem]

It seems rather impractical to me, perhaps it would be more beneficial as it is as an addon

[Dere-Wah]

It seems rather impractical to me, perhaps it would be more beneficial as it is as an addon

Mh, the point is having a separate addon and requiring server owners to download it specifically is even more impractical.

Also I'm sure we can figure out syntax to make the building process of a SkriptMetric class easy and intuitive.

[sovdeeth]

My main worries are about the malleability of scripts. Plugins have the advantage of being rather gatekept in knowledge; you can't super easily go in and edit values unless you know what you're doing and have the tools. This means that the reported values will usually always be accurate. With scripts, you don't have this guarantee at all. I worry this will lead to a single stat id having dozens of slightly different versions out there, all reporting different or false information back. It's also super easy to poison stats that way.

I think if this were to be added, we'd probably be best off not adding custom metrics and keeping it to a fixed structure.

[Dere-Wah]

My main worries are about the malleability of scripts. Plugins have the advantage of being rather gatekept in knowledge; you can't super easily go in and edit values unless you know what you're doing and have the tools. This means that the reported values will usually always be accurate. With scripts, you don't have this guarantee at all. I worry this will lead to a single stat id having dozens of slightly different versions out there, all reporting different or false information back. It's also super easy to poison stats that way.

I think if this were to be added, we'd probably be best off not adding custom metrics and keeping it to a fixed structure.

While yes, Skripts are definitely easier and malleable, no real harm could be done to individual metrics. bStats has filter options in place where you can add your own filters to your graphs if you notice any weird data. This alone would cleanup any edited strings (such as script version strings). But again, owners usually access Skripts to edit Options for Messages, features or items types. Version strings edit or stuff like that doesn't even happen that often and is a problem as much as it is in the Java Plugin world.

[NotSoDelayed]

bStats has filter options in place where you can add your own filters to your graphs if you notice any weird data

This is one of the issue Skript is trying to avoid — prevention is better than cure. Even if bStats are happy with collaborating with Skript to implement such feature, these sort of data can be heavily abused or manipulated wrongly where bStats might retract the implementation in future after realising how wrong this will potentially go. Users should not worry about any security issues, although for this case is indirectly, but this is definitely one of the concern I can forsee from a POV as a SkriptLang member.

[Dere-Wah]

This is one of the issue Skript is trying to avoid — prevention is better than cure.

bStats was created exactly with that in mind. Data manipulation, if you are a malicious user, is easy as hell, whether you're on Skript or on Java. They have security options in place, but in order to collect metrics that's how it has to be done, and they have experience handling the requests load.

Even if bStats are happy with collaborating with Skript to implement such feature, these sort of data can be heavily abused or manipulated wrongly where bStats might retract the implementation in future after realising how wrong this will potentially go.

Manipulating Metrics would only fake more users to a skript or just show no data at all, as if metrics were disabled. And if people are sending custom strings with offensive stuff or leaking data via custom charts, well, again that can be filtered out, but it can be done as easily via a malicious POST request.

They already gave the idea their blessing, so I don't see why we're still debating about whether or not they will like this.

The only case where we need to be cautious, is with the implementation: Scripts get reloaded much more often than Plugins and if we are not careful with how creating metrics classes works, we'll risk creating a new instance after each sk reload and end up duplicating data. This would be unintended behaviour and would spam requests without the user even knowing, and that's where we'd be at fault. (This is already handled in my Addon, for example.)

[Romitou]

I agree with the arguments that Skript doesn't seem suitable for such a Skript implementation due to Skript's flexibility in editing values.

Another argument I'd like to add to this conversation is utility. As far as plugins are concerned, I can see useful data for a plugin developer that would enable him to better understand how his plugin is used by its users. As far as Skript is concerned, I don't see what useful information could be sent as metrics, apart from a potential "script version".

Which brings me to another question: will it really be used? Do developers have a strong interest in such a metrics system? In fact, this is the first time I've seen anyone ask for such a feature, which leads me to wonder whether it's worth it, given the various counter-arguments that have been expressed above.

If such a feature were to be added to Skript, it would obviously be necessary to disable the sending of metrics in Skript only (either by default, or with Skript configuration) in my opinion.

[Dere-Wah]

What's the problem with stats?

We're not against the idea in principle, but we have some major concerns about its practicality.

Currently, our biggest concern is that the data will not provide any accurate or useful value. We worry that the data will be, at best, inconsistent with reality and, at worst, completely innaccurate. This is because scripts exist in a chop-and-change ecosystem where users are able to (read: encouraged to) download things and play with them or rewrite them entirely, at which point they're no longer a thing that should be counted in the original statistic.

As a result, numbers are likely to be significantly over-reported, and so the data will not provide any valuable knowledge to the developer. Statistics are not there to be an ego boost. If the reporting is inaccurate (beyond a respectable error bar) then this whole idea is defunct.

Why are plugins different from scripts?

Plugin statistics are possible because a plugin is a concrete and identifiable thing. It has a namespace, a signature, a manifest. We have version numbers, Git hashes and all sorts of data zipped inside the plugin. It's really easy to tell what is and isn't your plugin. Of course, you can decompile and edit little bits, but you'd never do that on a large scale because it's impractical to do so rather than rebuilding your own plugin from its source. Compiled plugins avoid the issues above by being annoying to edit, more than anything else.

A script is a text file: it's small, it's easy to edit, you can chop and change bits as you like, and users are encouraged to do so. Very few users download a script and don't go in and edit something. Once a user has downloaded and edited your script, it is no longer yours. It might be completely unrecognisable from yours: the user might delete half of it, or paste bits and pieces into their own scripts. How do we recognise the identity of a text file? How do we know when it has changed incomparably from its original version?

Can this be resolved?

Sort of. The easiest way to correct for modifications would be to keep an eye on the hash of the file (which we already have) or something based on its constituent structures.

The hard part is determining when something has been unreasonably modified. Obviously, changing a few options at the top of the file shouldn't invalidate the metrics, and I don't see a problem with tweaking a few strings here and there (people like to add themed colours, or add their server name in messages), but once you start to mess with the actual functional content of the script (e.g. removing some commands? Adding your own function? Breaking the script up across multiple files?) then it should no longer report to the same metric. Personally, I think a tree & branch hash is probably the best way to estimate this, but it'll probably take some experimenting to come up with a reasonable idea.

I think the easiest way of accomplishing this would be to have some kind of publishing command that takes in a script and generates its version-identity hash, which would need to be submitted to the metrics tracking site as an acceptable version. The site would have to exclude requests that submitted an unexpected hash (i.e. had incomparably changed from a real version).

Security is a non-issue

There is practically no chance of syntax being included to collect custom graphs or charts. We would provide the basic statistics through the plugin itself: Minecraft version, Skript version, server & player count, maybe some non-invasive insights into the script itself. The script would just need to include a structure for the metric ID:

metrics id 17044

There would be no avenue for users to submit raw text or data beyond that. More than this, I don't see what would be valuable to collect (without taking up half the script file with some kind of chart section).

Thanks for the reply, it was really helpful and honestly, yeah, Skript's nature is a "chop off parts and mix in together functions" for a lot of players. And yeah, once a script gets mixed in with other parts, metrics become basically useless, as you're not tracking your work anymore but what is left of it.

I found your approach with hashes really interesting, and it might actually solve the problem above mentioned. I'll try to get here the author of bStats so that they can comment on this, and we can see if they are able to implement such feature when registering a Skript on the website.

And yeah, while usage of custom charts would be really valuable to most, actually adding it in a natural-language syntax such as skript might be hard (for example, to build a Drilldown Pie Chart you need to build a 3 dimensional map <string <string, int>> for the category, value and weight).

if in the end bStats is not able to add these features we might require to track down skripts, we could make our own platform for this type of tracking, and design it as we like.

[Bastian]

Hey there, bStats author here 👋

I think there was a small misunderstand here, maybe caused by me not being familiar with the Skript ecosystem. From my understanding, one goal of Skript is to allow server owner to write individual scripts for their own server. This is what I had in mind when @Dere-Wah approached me with the idea: Allow server owners to write scripts that can collect stats for their own server, mainly by using custom charts.

For example, if a sever owner is interested in how many diamonds are mined on their server, they could create a custom line chart that tracks the "mining rate" of diamonds over time.

Edit: This ofc. only makes sense when server owners want to share the stats with their users. Otherwise an offline solution that just dumps the data into a CSV file or similar is most likely the better choice.

[Bastian]

This is what I had in mind when @Dere-Wah approached me with the idea: Allow server owners to write scripts that can collect stats for their own server, mainly by using custom charts.

After thinking about it some more, I'm not so sure how well suited bStats is for this use case. bStats is based on the assumption that there are many servers sending data to bStats, and that small "mistakes" (e.g. data not being sent due to server restarts) cancel out. This is not the case for sigle-server-statistics. A tailor-made solution for server owners could also do many things differently than bStats, which has to make some compromises due to its scale (no live data, etc.). So at best, bStats can cover this use case, but it will probably only be a suboptimal solution.

[NotSoDelayed]

For example, if a sever owner is interested in how many diamonds are mined on their server, they could create a custom line chart that tracks the "mining rate" of diamonds over time.

@Bastian You got half of the idea there. Our fellow user here proposed a suggestion about they would like to create a (potentially) useful resource using Skript, which is an English-like scripting language, and allows monitoring their scripts usages through bStats, which it is not feasible due to the concerns mentioned all above.