fcor / arjs-gestures

Rotate and zoom with touch gestures on any Image tracking or Marker Based AR.js scene

AR.js & A-Frame Gestures

Example of using gesture events on AR.js with A-Frame. This work is based on this example from 8th Wall.

Scale and rotate 3D elements from your AR.js scene using gesture-detector and gesture-handler components.

Try now!

Image Tracking

🚀Open this sample on your phone and scan this picture

Marker Tracking

🚀Open this sample on your phone and scan this marker

Installation

Import this file if you want default touch events. Keep reading to learn how to extend it.

<script src="https://raw.githack.com/fcor/arjs-gestures/master/dist/gestures.js"></script>

How it works?

gesture-detector listens to regular touch events directly on a-scene and emits a custom event indicating how many fingers were involved ("one", "two", "three" or "many") and passing some details of the event, like the position, spread and coordinates where user touched the screen. This component was developed by 8th Wall for their A-Frame based demos and can be found here.

gesture-handler adds listeners for custom gesture events, emitted by gesture-detector. This component should be placed on the 3D element we want to control and it automaticaly detects if the marker or image is found or lost to ensure the element could only be manipulated if it's actually visible. This component could be customized via properties. Currently supports pinch to zoom and finger spin for rotating the element.

Properties

Property	Description	Default Value
enabled	Whether gesture controls are enabled.	true
rotationFactor	Factor for controlling rotation	5
minScale	Minimum scale applied to the 3D element	0.3
maxScale	Minimum scale applied to the 3D element	8

Examples

Image Tracking

<a-scene
  arjs="trackingMethod: best; sourceType: webcam; debugUIEnabled: false;"
  embedded
  renderer="logarithmicDepthBuffer: true;"
  vr-mode-ui="enabled: false"
  gesture-detector
  id="scene"
>
  <a-nft
    type="nft"
    url="https://arjs-cors-proxy.herokuapp.com/https://raw.githack.com/AR-js-org/AR.js/master/aframe/examples/image-tracking/nft/trex/trex-image/trex"
    smooth="true"
    smoothCount="10"
    smoothTolerance=".01"
    smoothThreshold="5"
    raycaster="objects: .clickable"
    emitevents="true"
    cursor="fuse: false; rayOrigin: mouse;"
  >
    <a-entity
      gltf-model="https://arjs-cors-proxy.herokuapp.com/https://raw.githack.com/AR-js-org/AR.js/master/aframe/examples/image-tracking/nft/trex/scene.gltf"
      scale="5 5 5"
      position="50 150 0"
      class="clickable"
      gesture-handler="minScale: 0.25; maxScale: 10"
    >
    </a-entity>
  </a-nft>
  <a-entity camera></a-entity>
</a-scene>

Marker Tracking

<a-scene
  arjs
  embedded
  renderer="logarithmicDepthBuffer: true;"
  vr-mode-ui="enabled: false"
  gesture-detector
  id="scene"
>
  <a-assets>
    <a-asset-item
      id="bowser"
      src="https://cdn.glitch.com/06bd98b4-97ee-4c07-a546-fe39ca205034%2Fbowser.glb"
    >
    </a-asset-item>
  </a-assets>

  <a-marker
    preset="hiro"
    raycaster="objects: .clickable"
    emitevents="true"
    cursor="fuse: false; rayOrigin: mouse;"
    id="markerA"
  >
    <a-entity
      id="bowser-model"
      gltf-model="#bowser"
      position="0 0 0"
      scale="0.05 0.05 0.05"
      class="clickable"
      gesture-handler
    >
    </a-entity>
  </a-marker>
  <a-entity camera></a-entity>
</a-scene>

Credits

Kudos to 8th wall for sharing their A-Frame Manipulate example!

Bowser 3D model was made by santiago3052008 and can be found here

arjs-gestures's Issues

Loading multiple models

I am using ar.js with gestures.js to make a sneakers showroom. I preloaded my models and loaded them all to my scene when marker is found but one model is show, others are loaded with the scale of "0 0 0"... Added event listener (click) to switch between the models. Everytime the model is tapped, the scale for current model is set to "0 0 0" while the scale for the next model is set to "2 2 2". The problem I have is following: Every time I try to zoom in to the model using gestures, first model is zoomed, not the current "active" model. Funny thing is, I do not have the same problem for rotation. Moving my finger rotates the "active" model. The reason I loaded all the models at the same time is to reduce the time needed between model switching. If I get this correctly, my problem is that zooming scales all of the models at the same time but only first one is shown, or am I wrong? Likewise it rotates all the models at the same time but because their scale is set to "0 0 0" their rotation is not visible. Any tips please

Can't manipulate the model (not via marker)

Hi, I have loaded the model without the marker. Here is my code:

<head>
  <script src="https://aframe.io/releases/1.0.4/aframe.min.js"></script>
  <script src="https://raw.githack.com/AR-js-org/AR.js/master/aframe/build/aframe-ar.js"></script>
 <script src="gesture-detector.js"></script>
  <script src="gesture-handler.js"></script>
</head>

<body>
  <a-scene
    arjs
    embedded
    renderer="logarithmicDepthBuffer: true;"
    vr-mode-ui="enabled: false"
    gesture-detector
    id="scene"
  >
    <a-assets>
      <a-asset-item
        id="bowser"
        src="https://cdn.glitch.com/06bd98b4-97ee-4c07-a546-fe39ca205034%2Fbowser.glb"
      >
      </a-asset-item>
    </a-assets>
    <a-entity
      id="bowser-model"
      gltf-model="#bowser"
      position="0 0 0"
      scale="0.05 0.05 0.05"
      class="clickable"
      gesture-handler
    ></a-entity>
    <a-entity position="0 0 3.8" camera></a-entity>
  </a-scene>
</body>

But I can't manipulate with model by this way. Could you help me? Thanks!

The second example does not work ！

Multiple object

I have multiple cubes, and I'm wondering how to interact with them separately?

About AR marker

Hi, @fcor, great project!

I would like to change marker, so where can I generate marker?

How can i can moving object3d by onefingermove?

I am now able to scale and rotate, thank you very much. How can I move objects using a single finger, like drag and drop?

Nice work! How to change the marker？Thank you!

Nice work! How to change the marker？Thank you!

How Can I set autorate?

Can this work with the location-based method?

Not really an issue just a question! This is super cool and I am excited to have come across it, it works really well. Thank you!

I did try applying it to a location-based object (i.e. no marker) and it doesn't work with that. I know you've not suggested that it should, but I wondered if there's any way to make it work with location-based or would that be impossible?

Disable vertical rotation

Is there a way to disable the vertical rotation and only allow horizontal rotation of the models?

Image Tracking

https://github.com/Carnaux/NFT-Marker-Creator xd

issue

hi I've tried to transfer every single file over to my own repository but my 3d entity is not responding to any gestures. why is this the case?

How to use the example under vue or ES6 module？

Hi!
I want to use this demo in a modular form, but neither AR.js nor Aframe.js has a modular demo. How can i do it?

Keep the model in the center of the screen

When the scanned marker shows the model，I want to keep the model on the screen and not move with the marker.
How can i do it？

Customise scaling of height and width

Hi,
I was wondering if we can do customise scaling of the height and width for the 3d model indivdually, i have tried adding threefingermove and fourfingermove for customising the width and height respectively. However, it seems like not working due to the scaling calculation for the change of model size.

This is the part for the init:

init: function () {
    this.customScaleHorizontal = this.customScaleHorizontal.bind(this);
    this.customScaleVertical = this.customScaleVertical.bind(this);
    this.handleScale = this.handleScale.bind(this);
    this.handleRotation = this.handleRotation.bind(this);

    this.initialScale = this.el.object3D.scale.clone();
    this.scaleFactor = 1;
  },

This is the part for updating the eventlistener:

update: function () {
    if (this.data.enabled) {
        this.el.sceneEl.addEventListener("onefingermove", this.handleRotation);
        this.el.sceneEl.addEventListener("twofingermove", this.handleScale);
        this.el.sceneEl.addEventListener("threefingermove", this.customScaleHorizontal);
        this.el.sceneEl.addEventListener("fourfingermove",this.customScaleVertical);

    } else {

        this.el.sceneEl.removeEventListener("onefingermove", this.handleRotation);
        this.el.sceneEl.removeEventListener("twofingermove", this.handleScale);
        this.el.sceneEl.removeEventListener("threefingermove", this.customScaleHorizontal);
        this.el.sceneEl.removeEventListener("fourfingermove", this.customScaleVertical);

    }
  },

This is the part for removing the eventlistener:

remove: function () {
    this.el.sceneEl.removeEventListener("onefingermove", this.handleRotation);
    this.el.sceneEl.removeEventListener("twofingermove", this.handleScale);
    this.el.sceneEl.removeEventListener("threefingermove", this.customScaleHorizontal);
    this.el.sceneEl.removeEventListener("fourfingermove", this.customScaleVertical);

  },

This is the part for customizing the width:

customScaleHorizontal: function (event) {
     this.scaleFactor *=
      1 + event.detail.spreadChange / event.detail.startSpread;

    this.scaleFactor = Math.min(
       Math.max(this.scaleFactor, this.data.minScale),
       this.data.maxScale
     );

    this.el.object3D.scale.x = this.scaleFactor * this.initialScale.x;
    this.el.object3D.scale.y = this.el.object3D.scale.y;
    this.el.object3D.scale.z = this.el.object3D.scale.z;

   },

This is the part for customizing the height:

customScaleVertical: function (event) {
      this.scaleFactor *=
        1 + event.detail.spreadChange / event.detail.startSpread;

      this.scaleFactor = Math.min(
        Math.max(this.scaleFactor, this.data.minScale),
        this.data.maxScale
      );

      this.el.object3D.scale.x = this.el.object3D.scale.x;
      this.el.object3D.scale.y = this.scaleFactor * this.initialScale.y;
      this.el.object3D.scale.z = this.el.object3D.scale.z;
    },

And this is the only part i have changed for gesture-detector part:

getEventPrefix(touchCount) {
        const numberNames = ["one", "two", "three", "four"];

        return numberNames[Math.min(touchCount, 4) - 1];
    }

I'm not sure whether is my logic wrong or what, please help me.
Thank you!

How to replace other photos

Hello, I tried it with another photo and found no effect. I want to know if this program has any special requirements for photos, and how can I replace photos

How to stop animation on touch

Hi,

Thanks for the great work - I'm using the gesture-handler / gesture-detector and it is working well.

I recently added the animation="property: rotation; to: 0 360 0; loop: true; dur: 10000" to a-entity to have the models rotate automatically.

When this happens you can still pinch to zoom but can no longer use one finger to rotate the object

Do you know how to disable the rotation animation on touch?

Thanks in advance!

Camera error

hello, when i open the example some text says : "Webcam error Name: NoReadableError Message: Could not start video source".

It doesn' work in PC like Chrome browser

the second demo works fine in mobile phone like my iPhone Safari.
but it can't detect two-finger-move in PC like MacOS Chrome for your second demo.
for 8thwall demo : https://8thwall.8thwall.app/manipulate-aframe/ (it works fine)

Really really needs your help!!