Implement more human-like mouse movements with pyclick about os-bot-color

[kelltom]

I actually was looking at this exact project a few days ago when looking for example usage of Pyclick. Seems like a good place to start for sure. This file in particular.

I think adding some sort of temporary Pyclick support, for now, would be pretty safe and beneficial. I can imagine it would need some refactoring if I go through with the PyAutoGUI swap in the future.

I was actually thinking of making a development branch for most of the new feature requests. I am in the process of making a YT tutorial and it would be ideal to have the repo in a familiar state for newcomers - in other words, I'm slightly afraid of the tutorial going obsolete too quickly, so I want to be careful with the updates. A development branch would at least allow for these larger changes to be pushed all at once at a later date accompanied by a new tutorial/updated documentation. There are even mild plans of a GUI rework, which would be a fairly massive job, so that's a little glimpse into my unofficial plans 😅

[Travis-Barton]

Hey I see that this should be implemented after #34 but I think I can work it into the current flow really easily. Should I put it in just so we have the option in the meantime?

[Travis-Barton]

Also in the dependence list I see another Osrs bot that we might want to look at for learning purposes. They don't have your contour detector and that's the best feature imo, but they do have some interesting stuff!

[Travis-Barton]

Thats smart! it'll keep things clean!

[Travis-Barton]

Want to assign the temp mouse change to me? I can get that up and running tonight I think.

I'll just do the following:

1. add the package to the requirements
2. import it inside the mouse file
3. make the pyclick package one of the options in addition to None and rand called bezier
4. Should it be the default?

[kelltom]

Sure, sounds good!

I think that if it turns out to be really solid in terms of behaving as expected, it should be the default. My last stint with Pyclick kind of made me unimpressed with it, but I think I was just using it incorrectly.

I could see the options being like bezier, gaussian, and None. Then for whatever reason, the user can select the curve that fits their situation best!

[kelltom]

Note there is some other interesting work happening in #40

[Travis-Barton]

That chat is super interesting! Missclicks are a problem I've encountered using a cow killer and the Rock Crab killer, so finding an improvement to that would be awesome!

[kelltom]

@Travis-Barton I've implemented Pyclick and it is working pretty well so far! There are some demo videos on the Discord and you can checkout the feature branch here to try it for yourself: https://github.com/kelltom/OSRS-Bot-COLOR/tree/35-implement-more-human-like-mouse-movements-with-pyclick

I'm still looking to add things like random sleeps/waits, or maybe even intentional mistakes

[kelltom]

Also noteworthy:

[Travis-Barton]

I'll check it out tonight!

It looks sound, lots more knobs to tweak and randomize

[kelltom]

This is also good to know: https://www.youtube.com/watch?v=qzhwSUP1Ypw&ab_channel=AndyLi

[Travis-Barton]

Something like that is something to shoot for. I think we can do some good workarounds leading up to that point, like switching to his level 3 from our level 2 uniform distributed click spot. In my agility bots I make several recorded click tracks using the tool we made and randomly choose amongst them while adding some randomness, I think that will keep me clear for a little while! That's kind of like a poor man's version of what he has.

[Travis-Barton]

Is there a way to keep the input parameters the same? it'll make merges more smooth.
for time variance we could add a quick number to move_to and just square it or something so it doesn't stack up and cause crazy slowdowns.
something like

for curve_x, curve_y in HumanCurve((start_x, start_y),
                                            destination,
                                            offsetBoundaryX=offsetBoundaryX,
                                            offsetBoundaryY=offsetBoundaryY,
                                            knotsCount=knotsCount,
                                            distortionMean=distortionMean,
                                            distortionStdev=distortionStdev,
                                            distortionFrequency=distortionFrequency,
                                            tween=tween,
                                            targetPoints=targetPoints
                                            ).points:
            pag.moveTo((curve_x, curve_y), duration=np.abs(rd.gaus(0,  time_variance**2)))
            start_x, start_y = curve_x, curve_y

or maybe a sleep parameter like

sleep_increment = np.abs(rd.gaus(0, time_variance))/targetPoints
for curve_x, curve_y in HumanCurve((start_x, start_y),
                                            destination,
                                            offsetBoundaryX=offsetBoundaryX,
                                            offsetBoundaryY=offsetBoundaryY,
                                            knotsCount=knotsCount,
                                            distortionMean=distortionMean,
                                            distortionStdev=distortionStdev,
                                            distortionFrequency=distortionFrequency,
                                            tween=tween,
                                            targetPoints=targetPoints
                                            ).points:
            pag.moveTo((curve_x, curve_y))
            start_x, start_y = curve_x, curve_y
            time.sleep(sleep_increment)

this is sudo code btw idk it if runs the way I think I will. We can make the default 0 too so it won't effect future code.

[kelltom]

The only issue with keeping a duration param is that it kind of conflicts with the targetPoints param. The speed at which the mouse travels seems to be dictated by the number of 'points' in the path, so we can't really tell the function how long we want the approximate duration to be (a duration value seemed to be implemented by Pyclick out of the box, but it was so inaccurate that I just removed it). I think it'll naturally be pretty random, since the distance the mouse has to travel also affects the total duration of the mouse movement. Currently, I have it set to be a random targetPoints value when you call the move_to function with no extra arguments, but maybe it makes sense to make that normally distributed. Might be worth looking at and playing with. Might need to chart the data to see if there are any obvious patterns in the clicks.

As for the click distributions themselves (e.g., the position of the clicks), I think there are multiple people working on that in Discord at the moment, so I'll see what they come up with and decide how to slot that in later.

I've merged my Pyclick integration into the development branch for now.