pixielabs / cavy

An integration test framework for React Native.

Cavy

Cavy is a cross-platform, integration test framework for React Native, by Pixie Labs.

Cavy tests allow you to programmatically interact with deeply nested components within your application. Write your tests in pure JavaScript and run them on both Android and iOS.

Cavy tests look like this:

export default function(spec) {
  spec.describe('A list of the employees', function() {
    spec.it('can be filtered by search input', async function() {
      await spec.exists('EmployeeList.JimCavy');
      await spec.fillIn('SearchBar.TextInput', 'Amy');
      await spec.press('Button.FilterSubmit');
      await spec.notExists('EmployeeList.JimCavy');
      await spec.exists('EmployeeList.AmyTaylor');
    });
  });
}

📋 Requirements

• React Native >= 0.59
• React >= 16.8.0

👶 Getting started

Get set up with Cavy by following our installation guide.

You might also want to check out some articles and watch talks about Cavy to find out a bit more before you write code.

If you need some inspiration, head over to Cavy's sample app, follow the instructions in the README, and see Cavy in action.

📘 Documentation

Full documentation and guides for Cavy can be found on our website.

🗺️ Development roadmap

Take a look at our public Pivotal Tracker to see what we're currently working on, and what features we plan to add to Cavy next.

💯 Contributing

When making changes to Cavy, it's useful to have the CavyTester app running in development for regression testing.

Follow the instructions it's own README on how to get the tester app running against a local version of the Cavy library.

Here you'll also find instructions on adding new test cases to ensure your functionality is fully tested. Please do this :)

Before contributing, please read the code of conduct.

• Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
• Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
• Fork the project.
• Start a feature/bugfix branch.
• Commit and push until you are happy with your contribution.
• Remember to submit a PR to DefinitelyTyped to update the type definitions if you've changed a function's inputs or outputs.
• Remember to submit a PR to update the documentation if you've changed how something works.
• Please try not to mess with the package.json, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so we can cherry-pick around it.

cavy's Issues

Add table of contents to README

The README is getting pretty long, let's add a table of contents.

Add a `await spec.pause(i)` helper

Sometimes you just want your spec to pause or slow down, maybe to visually confirm something, or because it's too fiddly to use exists to wait for something to happen.

const pause = function(i) {
  const p = new Promise((resolve, reject) => {
    setTimeout(function() {
      resolve();
    }, i);
  });

  return p;
}

Clear AsyncStorage between test cases

For example if your application uses AsyncStorage to store logged-in details, it can be challenging to test the login form.

Add documentation about available spec helpers

Or at least link to where they are defined.

Change Cavy to a devDependency in the sample app

cavy/sample-app/EmployeeDirectory/package.json

Line 10 in 0b6fd5b

"cavy": "^0.1.0",

Question: Integration testing of (non-UI) RN modules

Thanks for Cavy! It is great for integration testing RN Components.

Are there any tips for how to test non-UI modules. What I really want to do is call a method on my RN module and assert the return value. I want to run the tests on real devices because that way I exercise the native Android/iOS code too.

Currently, I can create a wrapper component but it might not scale.

Ability to change DEFAULT_WAIT_TIME

Wrapper around a functional component gives me an error

import React, { Component } from 'react';
import { View, Image, Text, Linking } from 'react-native';
import CardSection from './CardSection';
import Button from './Button';
import { hook, wrap } from 'cavy';

class AlbumDetail extends Component {

render(){

	**const wrappedComponent = wrap(Button);**
	return(
	    <CardSection>
		**<wrappedComponent onPress = {()=>Linking.openURL(this.props.album.url)} 
                           ref = { this.props.generateTestHook('FlatList.Button')}
		       buy me
		/>**
	    </CardSection>
            );
    }

}

Here Button is a functional component that I am trying to use and test inside the CardSection component. It keeps giving me the following error. Can you please help me with this?

TaskQueue: Error with task : Expected a component class, got [object Object].

Testing Native + RN App

Hi, I saw Cavy's demo and am really interested in using it for our E2E tests. I have a couple of questions though -

1. We are in process of migrating our iOS App from native (objective-c) to React Native. As of now around 20% of the features/screens have been migrated to RN. Can I use Cavy to test those 20% RN part of the app ? Or Cavy can only be used for apps which are fully developed in RN ?

2. Can Cavy run tests on real iOS/Android devices ?

Importing Cavy breaks Jest tests that use react-test-renderer

Hi, myself and my group used Cavy for one of our projects, and we loved it apart from this one thing - we were all super new to React Native and had under two weeks to learn the framework and make an app, which is why I've not made a pull request for this straight away (I'm not sure how hacky it is, or if there's a more elegant solution, it worked for us so we moved on), but hopefully this helps somewhat.

Problem

Given a simple test like the one below (or any test that renders components like this):

it('renders correctly', () => {
  const tree = renderer.create(
    <SearchBar />
  );
});

We get the following error/stack trace:

TypeError: Cannot read property 'add' of undefined

      at node_modules/cavy/src/hook.js:60:25
      at commitAttachRef (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:5545:1)
      at commitAllLifeCycles (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:6262:1)
      at Object.invokeGuardedCallback$1 (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:2180:6)
      at invokeGuardedCallback (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:2127:25)
      at commitAllWork (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:6376:1)
      at workLoop (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:6649:1)
      at Object.invokeGuardedCallback$1 (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:2180:6)
      at invokeGuardedCallback (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:2127:25)
      at performWork (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:6764:1)

Solution

We threw in a quick fix which simply involved making the generateTestHook() method (within node_modules/cavy/src/hook.js) return early if 'this.context.testHooks' ('this' being some sort of wrapper, we didn't have time to learn more about that object) wasn't defined:

if(!this.context.testHooks) { return }

So the whole method looks like:

generateTestHook(identifier, f = () => {}) {
      return (component) => {
        if(!this.context.testHooks) { return }
        if (component) {
          this.context.testHooks.add(identifier, component);
        } else {
          this.context.testHooks.remove(identifier, component);
        }
        f(component);
      }
    }

Cavy works in react-native 0.42.3?

I'm trying to test my simple app with cavy, but cavy doesn't work well.

Here is my configuration:

react-native: 0.42.3
react-native-cli: 2.0.1

When I install cavy, shows up a warning:

npm WARN [email protected] requires a peer of [email protected] but none was installed.

When run at android, shows an error:

undefined is not a function (evaluating 'this.props.generateTestHook('PaginaInicial.PontoButton')')

I had imported hook from cavy, and exported: export default hook(PaginaInicial);

I'm really new in react-native and cavy, so sorry if my issue it's not about cavy;

Tks;

Do nothing if testing is disabled

Add a sample application

Support triggering gesture callbacks, e.g swipe vertical/horizontal

How do I prevent Cavy from resetting?

Does not work with React Native 0.46+

PropTypes need to be imported from 'prop-types' as they are removed from the main react library since RN 0.46+ was released.

Support for create-react-native-app / Expo

Is there a way to tie in Cavy with create-react-native-app (https://facebook.github.io/react-native/docs/getting-started.html)?

This doesn't seem to use the AppRegistry and I'm not sure how go about adding Cavy. Here are the files given by default:

It would be good to be able to add Cavy and try it out at the most basic level and build from there.

EDIT: I see a similar issue to #23 when I add my own main.js
undefined is not an object (evaluating '_this2.context.testHooks.add')

Injection App Wrapper

@jalada I was wondering if you not use something like Root Siblings Library to inject changes on the Appregistry.

Thus, everything could be more transparent and you avoid the App wrapper.
In addition, you can only execute this code in DEV mode.
This will avoid your production code to be messy.

What do you think about it?

[Feature request] Should support test loop

Hi team, we love your work, please help support test loop so that QCs can run test-cases of complex UI flows totally by JS code AND by a given fixed number of time
Tons of thanks
Regards

Error when change Component to test

Hi, i have an issue, please help me to fix this. My issue is :

1. I add test for component A (Screen A have button A)
2. Automation click button A and go to screen B
3. In screen B, i have button B and i add ref={this.props.generateTestHook('ButtonB')} for button B => it show error : "underfine ....testhooks.remove"

My specs file :
spec.describe('Click to change tab', function() {

spec.it('click second tab footer', async function() {
  await spec.press('Footer.SecondTab');
  await spec.pause(2000);
  await spec.press('SecondTab.Viewbarcode');
  await spec.pause(2000);
  await spec.press('ButtonB');
  await spec.pause(2000);
});

});

CircleCI

Is it possible to run tests on CircleCI?

Update sample app React Native version

We know Cavy works with the latest React Native, but we should update our sample-app acordingly.

Handle arrays of components in TestHookStore.add

Integrating with wix/react-native-navigation

Our app is using wix/react-native-navigation which as per the diagram below changes the implementation of the app. Which means, wrapping the app under a single shared Tester component doesn't work in the same way.

Just wondering if anyone here has any ideas or solutions to this?

Is it possible to use Jest with Cavy?

Sorry if this is a dumb question -- I haven't used Cavy or Jest, only read through the docs.

Jest's ability to take snapshots and compare them seems powerful.

Is it possible to do that within Cavy tests? Would it be possible to write a Cavy test that navigates to a specific screen, grabs some JSX tree, then uses Jest to compare it to a snapshot?

this.props.generateTestHook is not a function

I just got into unit testing with cavy for a react-native android/ios application. I tried to adapt the instructions from the README.md on the project, but it seems like that it doesn't work.
I alway get the error message as shown in the title.

Screenshot of the error:

The code, of the Search bar that I want to test.

'use strict';

import React, {Component} from 'react';
import {
  View,
  Text,
  TouchableOpacity,
  TouchableNativeFeedback,
  Platform,
  StyleSheet,
  TextInput,
  ToastAndroid,
} from 'react-native';

import IonIcon from 'react-native-vector-icons/Ionicons';
import MaterialIcon from 'react-native-vector-icons/MaterialIcons';

import {colors, texts} from '../imports';
import {TouchableComponent} from '../elements';

import { hook } from 'cavy'; // Thien: testing mit cavy

const CAN_USE_MIC = Platform.OS === 'android';
const ICON_SIZE = 30;

export class SearchBar extends React.Component {

  props : {
    onTextChanged: ?(text : string) => void;
    onTextSubmitted: ?(text : string) => void;
    enableSpeech: ?boolean;
  };
  onChangeText: ()=>void;
  onEndEditing: ()=>void;

  text : string;

  static get defaultProps(){
    return {
      enableSpeech: true,
    };
  }

  constructor(){
    super();
    this.onChangeText = this.onChangeText.bind(this);
    this.onEndEditing = this.onEndEditing.bind(this);
  }

  /**
   * Stores the current given text and triggers the onTextChanged prop function.
   * Useful for list filtering for each character input
   * @param text The current text
   */
  onChangeText(text : string){
    this.text = text;
    if(this.props.onTextChanged){
      this.props.onTextChanged(text);
    }
  }

  /**
   * Triggers the onTextSubmitted prop function.
   * Useful for list filtering when input is finished
   * Since the event does not pass the current text value, it is required to take it from the `text` property
   * of the object
   */
  onEndEditing(){
    if(this.props.onTextSubmitted){
      this.props.onTextSubmitted(this.text);
    }
  }

  render(){
    return (
      <View style={styles.container}>
        <View style={styles.searchIconContainer}>
          <IonIcon name={`${Platform.OS === 'ios' ? 'ios' : 'md'}-search`} size={ICON_SIZE} color={'#ccc'} />
        </View>
        <View style={styles.searchInputContainer}>
          <TextInput
            //ref="input"
            ref={this.props.generateTestHook('SearchBar.TextInput')}
            placeholder={texts.searchBar.searchPlaceholder}
            style={{flex: 1}}
            returnKeyType={'go'}
            onChangeText={this.onChangeText}
            onEndEditing={this.onEndEditing}
            selectionColor={colors.grassGreen}
          />
        </View>
      </View>
    );
  }
}

const TestableScene = hook(SearchBar);
export default TestableScene;

I'm running on Arch Linux with npm 5.3.0 and tested the app on an HTC10 Android 7.0.

Thanks in advance.

EDIT:
Now I get following error..
undefined is not a function (evaluating 'this.props.generateTestHook('SearchBar.Textinput')')

Documentation for writing your own spec helper function

As per #29.

already used ref as function

In my app I have ref={(ref) => { this.props.exposeRef(ref) }}
How can I integrate cavy in such cases?

Add support for ReactNative-web

Could it be possible to add support for ReactNative-web and test web applications the same way as mobile ones?

Sample App doesn't work with Android.

Hey Guys,

Very interested in replacing Appium with Cavy for our Android/RN app but ran into some trouble getting the sample app working on Android.

If you can update the sample app to make it easier to get up and running, that would be really helpful. Or, if I'm missing something, let me know.

Thanks!

Joshua

Support stateless components without a wrapping function

Basically everything has trended towards stateless functional components, using Recompose etc. Nobody wants to use class components anymore. And cluttering your codebase with additional wrappers also is far from ideal.

I feel like there should be a lower level and more automated connection to component instances based on a simple id prop you supply them. What about ways to monkey patch into the internals of React and make something like that happen??

Update documentation for setting up sample app

As per #60, the sample app README tells you to follow the upstream instructions, but they're a little out of date now. We should do our own instructions.

Whilst we're at it, we could improve some other aspects of the sample app, e.g. replace all the people with types of cavy 😎

How to move component??

Can i automation moving or rotate component by this library?

Difference from appium tests

Could you please elaborate on how this is different from appium tests?
Thanks

Add unit tests

It'd be good to have unit tests running so that we can make code changes confidently. Some decisions would need to be made on the choice of framework and such. I tried it out using jest/travis-ci, but it might be worth considering other alternatives.

Branch

Travis Build

Add Support for React-Native-Router-Flux

Is it possible to add support for React Native Router Flux? Right now if I try to hook Cavy with React Native Router Flux, it gives a blank screen and breaks very simple test cases.

Support before and after test case/suite hooks (e.g. before each, before all, after each, after all)

Popular test frameworks has ability to use before and after callbacks in describe sections. Why there are no such callbacks and is it in roadmap?

Add a code of conduct

Perhaps based on https://www.contributor-covenant.org/

Is possible to use with redux ?

I try to use with redux but I don't have suceessed

does this work on react-native Alert.alert component as well?

seems not possible atm, anyways just askin if this is possible

Custom functions before/after tests

Hi guys, I am using cavy but I need to set up some custom logs if the test works or fails. For example I use reactotron to log the data and instead of the common console.log. I think it is a good approach have the possiblity of passing two functions as parameters to the 'spec.it' function, one for success and one for the fail.

export default function(spec) {
  spec.describe('Login', function() {
    spec.it('works', async function() {
      await spec.press('PublicListScreen.LoginButton')
      await spec.fillIn('Scene.Login.Username', '[email protected]')
      await spec.fillIn('Scene.Login.Password', 'PedroPedro')
      await spec.press('Scene.Login.Button')
    },
    function() { return console.tron.log(`LOGIN TEST SUCCESS  ✅`) }, //Success function
    function() { return console.tron.log(`LOGIN TEST ERROR  ❌`) }, //Fail function
    )
  })
}

I have made this changes in a fork. https://github.com/pgonzalez-santiago/cavy
you can see my changes in here: master...pgonzalez-santiago:master

What do you think?

Redux

I'm trying to integrate this with my app (which is using redux), how can I manage to makes my component works?
I see that you guys are using context in the constructor of the hook method.
I guess this is messing with my redux provider.
Any ideas to make it works?

Does not work with Mobx?

I cannot find a mailing list so this goes here for now: I am working on a react-native project which uses Mobx. I cannot seem to get Cavy working, but it works fine if I remove Mobx. The existing code wraps the App with observer from mobx-react/native (documentation).

The error I am getting is that generateTestHook cannot be found.

Can anybody give me advice on how to combine the two or Is this a known incompatibility?

Cannot call a class as a function

I'm getting the following error when wrapping a react-native into a stateful component with cavy:

'Cannot call a class as a function'

import { hook, wrap } from 'cavy'
import { Button, TouchableOpacity, View } from 'react-native'

...
const WrappedComponent = wrap(Button)

...
    return (
	<WrappedComponent
		title='theTitle'
		onPress={() => {}}
		ref={this.props.generateTestHook('theButton')}
	/>
    )

Any thoughts on how to resolve this?

Uses different hooks method for fillIn

Hello, this issue is more a question:
Is there a way to override the fillIn and press method with other function on the component?

Something like :

  async fillIn(identifier, str, fn: Function) {
    const component =  await this.findComponent(identifier);
    component.props[fn](str);
  }

I'm already overriding theses function in my component which makes cavy unusable with my codebase.
If I could have a way to pass the function to trigger the fillIn would be nice.

What is the recommended way to separate cavy from production?

I was thinking of doing

render() {
let app = <App/>
if (__DEV__) {
app = <Tester><App/></Tester>
}
return app
}

class Button extends Component {}
if (__DEV__) {
Button = hook(Button)
}

The second one won't work since it needs props.generateTestHook, but in production won't generateTestHook be unnecessarily adding to store?

Add logo to README

Here it is:

Trigger native keyboard search button from a TextInput

Hey, very new to your library and evaluating if we can use on our project. We have a TextInput that is used to capture a search query. When done, the user taps the native search button to move to the next screen.

We'd like to trigger the native keyboard search button action once cavy has entered some text. Something like this:

    spec.it('Search for maths', async function () {
      const searchText = 'maths';
      await spec.fillIn('searchText', searchText);
      await spec.submit('searchText');
      await spec.pause(1000);
      //...
    });

Submit doesn't exist but is there something we can use?

clearAsyncStorage not working

<Tester
  specs={testSuitesArray}
  store={testHookStore}
  waitTime={1000}
  startDelay={500}
  clearAsyncStorage
>
  <Preload />
</Tester>

Not working for me

Support automation

Just tried cavy today and it looks cool. One thing that I would like to understand is how do you plan to automate it, meaning, run a console command and get 0/1 result. Any insight appreciated.