Hi, thanks for a great library, I just wondered whether you had done any work to integrate the output of tuneup tests into Jenkins i.e. generate JUnit style results?

Many thanks

The assertScreenMatchesImageNamed function needs to specify a default threshold value. The docs should also be updated to acknowledge this tweak ability.

Your lib was the reason for me to start with UIAutomation - THANKS!

It appears that the new Instruments version (6.0) alters its folder structure slightly, which is causing the run script in test_runner to fail. I understand this script is deprecated, but I still use it and others may too. The line I changed to fix it for myself was:

#AutomationInstrument.bundle becomes AutomationInstrument.xrplugin in the path with Instruments 6.0
TEMPLATE = `[ -f /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Instruments/PlugIns/AutomationInstrument.xrplugin/Contents/Resources/Automation.tracetemplate ] && echo "#{SDKROOT}/Developer/Library/Instruments/PlugIns/AutomationInstrument.xrplugin/Contents/Resources/Automation.tracetemplate" || echo "#{XCODE_ROOT}/../Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.xrplugin/Contents/Resources/Automation.tracetemplate"`.chomp.sub(/^\s+/, "")

#old code
#TEMPLATE = `[ -f /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Instruments/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate ] && echo "#{SDKROOT}/Developer/Library/Instruments/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate" || echo "#{XCODE_ROOT}/../Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate"`.chomp.sub(/^\s+/, "")

Just giving you guys a heads-up.

Maybe there are other ways to deal with a hanging Instruments process? A command line option to se the timeout would also be needed.

timeout = 15

  begin
    PTY.spawn(*command) do |r, w, pid|
      running = true
      while running do
        if IO.select([r], nil, nil, timeout) then
          line = r.readline
          puts line
          _, date, time, tz, type, msg = line.match(/^(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2}) ([+-]\d{4}) ([^:]+): (.*)$/).to_a

          case type
          when "Fail"
            failed = true
          end
        else
          failed = true
          running = false
          warn "\n### Timeout #{timeout} reached without any output - " +
            "killing Instruments (pid #{pid})..."
          begin
            Process.kill(9, pid)
            w.close
            r.close
            Process.wait(pid)
          rescue PTY::ChildExited
          end
          puts "Pid #{pid} killed."
        end
      end
    end
  rescue Errno::EIO
  rescue PTY::ChildExited => e
    STDERR.puts "Instruments exited unexpectedly"
    exit 1
  end

I looked at the Element Tree fro the keyboard in iOS 6.0 and compared it with your typeString function and here are the changes:

The "shift", "more, letters", "more, numbers" are now UIAKeys() instead of UIAButtons() in iOS 6.0 keyboard.

I construct the parameter across iPhone Simulator and want to change a language.
I thought that I could realize that I wanted to live on the following code.

diff --git a/test_runner/run b/test_runner/run
index b52ef4d..48000da 100755
--- a/test_runner/run
+++ b/test_runner/run
@@ -60,6 +60,10 @@ opts_parser = OptionParser.new do |opts|
   opts.on("-x", "--xunit", "Create Xunit formatted test result file in the output directory") do
     options.xunit = true;
   end
+
+  opts.on("-a", "--args <...>", "All following arguments will be passed on to Instruments") do |args|
+    options.args = args
+  end
 end

 unless ARGV.length >= 3
@@ -188,6 +192,7 @@ options.env_vars.to_a.each do |pair|
   command << "-e"
   command.concat(pair)
 end
+command << options.args
 puts "command=" + command.map { |s| s.sub(/\A(.*\s.*)\Z/, '"\1"') }.join(' ') if options.verbose

I come to be able to use it in this way.

$ tuneup/test_runner/run App.app test.js output -a -AppleLanguages \(ja\)

Because I am not used, I am sorry not pull request.

Calling typeString fails with the following error:

Error: (2012-10-09 15:14:20 +0000) - VerboseError: Cannot perform action on invalid element: UIAElementNil from target.frontMostApp().keyboard().buttons()["shift"]

Below is the section of my UIAElement tree dump related to the keyboard. As you can see, the "shift" key is present. I don't know what you can do to fix this.

UIAWindow "(null)" {{256, 0}, {768, 1024}}
        elements: {
            UIAImage "kb-drop-shadow-top.png" {{0, 416}, {1, 16}}
            UIAImage "kb-drop-shadow-top.png" {{0, 416}, {1, 16}}
            UIAImage "kb-drop-shadow-bottom.png" {{0, 416}, {1, 17}}
            UIAImage "kb-drop-shadow-bottom.png" {{0, 416}, {1, 17}}
            UIAImage "kb-drop-shadow-center.png" {{0, 416}, {28, 245}}
            UIAImage "kb-drop-shadow-center.png" {{0, 416}, {28, 245}}
            UIAKeyboard "(null)" {{0, 416}, {1024, 352}}
            elements: {
                UIAKey "q" {{0, 425}, {93, 79}}
                UIAKey "w" {{93, 425}, {93, 79}}
                UIAKey "e" {{186, 425}, {93, 79}}
                UIAKey "r" {{279, 425}, {93, 79}}
                UIAKey "t" {{372, 425}, {93, 79}}
                UIAKey "y" {{465, 425}, {93, 79}}
                UIAKey "u" {{558, 425}, {93, 79}}
                UIAKey "i" {{651, 425}, {93, 79}}
                UIAKey "o" {{744, 425}, {93, 79}}
                UIAKey "p" {{837, 425}, {93, 79}}
                UIAKey "Delete" {{930, 425}, {94, 79}}
                UIAKey "a" {{38, 511}, {92, 79}}
                UIAKey "s" {{130, 511}, {92, 79}}
                UIAKey "d" {{222, 511}, {92, 79}}
                UIAKey "f" {{314, 511}, {92, 79}}
                UIAKey "g" {{406, 511}, {92, 79}}
                UIAKey "h" {{498, 511}, {92, 79}}
                UIAKey "j" {{590, 511}, {92, 79}}
                UIAKey "k" {{682, 511}, {92, 79}}
                UIAKey "l" {{774, 511}, {92, 79}}
                UIAButton "Return" {{866, 511}, {158, 79}}
                UIAKey "shift" {{0, 597}, {91, 79}}
                UIAKey "z" {{91, 597}, {91, 79}}
                UIAKey "x" {{182, 597}, {90, 79}}
                UIAKey "c" {{272, 597}, {91, 79}}
                UIAKey "v" {{363, 597}, {90, 79}}
                UIAKey "b" {{453, 597}, {91, 79}}
                UIAKey "n" {{544, 597}, {90, 79}}
                UIAKey "m" {{634, 597}, {91, 79}}
                UIAKey "," {{725, 597}, {90, 79}}
                UIAKey "." {{815, 597}, {90, 79}}
                UIAKey "shift" {{905, 597}, {119, 79}}
                UIAKey "more, numbers" {{0, 683}, {273, 79}}
                UIAButton "Next keyboard" {{136, 683}, {137, 79}}
                UIAKey "dictation" {{181, 683}, {91, 79}}
                UIAKey "space" {{272, 683}, {543, 79}}
                UIAKey "more, numbers" {{815, 683}, {118, 79}}
                UIAButton "Hide keyboard" {{933, 683}, {91, 79}}
            }
            UIAImage "kb-drop-shadow-bottom.png" {{0, 766}, {1024, 17}}
        }

I updated Ruby to 1.2.4 and Xcode to 6.1, and tuneup js to latest version, then I try to use the auotomation, but got following error:

Launching instruments. Will retry 0 more times if it doesn't initialize

2014-10-29 15:47:33.674 instruments[64570:249034] WebKit Threading Violation - initial use of WebKit from a secondary thread.
Instruments Trace Error : Target failed to run: The operation couldn’t be completed. (FBSOpenApplicationErrorDomain error 4.) : Failed to launch process with bundle identifier

any idea? Thanks.

The documentation of assertWindow is very thorough, but how would I assert, e.g., that the navigationBar doesn't have a leftButton? Is the following correct?

assertWindow({
  navigationBar: {
    leftButton: function(button) {
      assertNull(button);
    }
  }
});

When switching between physical devices, it would be convenient to get the UDID dynamically. A command line option is needed, or perhaps a special value for the --device option (like "dynamic").

    ioreg_output = `ioreg -w 0 -rc IOUSBDevice -k SupportsIPhoneOS`
    if ioreg_output =~ /"USB Serial Number" = "([0-9a-z]+)"/ then
      options.device = $1
    else
      raise "Couldn't get the UDID using ioreg"
    end

Apple's "Instruments" crashes when Javascript UIAutomation tests throw a primitive, at least for version 4.5 (4523). This makes Tuneup.js unusable for our tests. For example, create a bare test

var target = UIATarget.localTarget();
throw "Github";

I've documented this further at http://apple.stackexchange.com/questions/69484/unknown-xcode-instruments-crash#answer-72996 .

Could Tuneup be adjusted to throw Javascript Objects, rather than a Primitives?

I use find(criteria,varName) method to find some elements, like this:
var views = window.find({"nameRegex":"test"},"");
(the text "test" is in my tablecell)
and then I check the value in the views. Sometimes it really works, but most of the time it gets a empty array, it loos like unstable. Is my tableview contain too much text?

I'm building an iOS app and the only pod I'm currently using is tuneup_js and the problem I'm facing is that my project tries to link against the Pods-library, which can't be built as there is no Objective C code in there.

I guess this might be an issue with cocoapods rather than tuneup itself, but as cocoapods is the recommended way of installing it this might be worth resolving.

My solution was to simply not link against the Pods-library for the time being.

Please let me know if I submit this this issue to cocoapods instead. And thanks for a neat library.

I am receiving the following message it seems after exactly 2 "tests" and instruments stopping. Anyone have any thoughts?

2013-04-02 10:29:13.669 instruments[4661:6207] Automation Instrument ran into an exception while trying to run the script. UIAScriptAgentSignaledException
2013-04-02 14:29:13 +0000 Fail: An error occurred while trying to run the script.
Instruments : Stopping Instrument(s)

Before discovering tuneup_js (which I love) and its typeString (which is slow and there are other keyboards complaining about it), I did the following to get the keyboard to work, and work fast:

var TypeString = function (str) {
    var keyboard = UIATarget.localTarget().frontMostApp().keyboard();
    for (i = 0; i < str.length; i += 1) {
        keyboard.typeString(str[i]);
    }
};

By typing one key at a time I am able to type letters (capitol and lowercase), numbers, characters (!) and it does it fast. I'm curious if other people have tried that and whether or not the tuneup typeString should be replaced with this code. Any thoughts?

is there any support for setting up and tearing down before and after tests?

There may not be much need for this, but at the moment I'm struggling because my app is left in the same state after each test run, and I need to reset manually .... perhaps I'm missing some instruments/automation preference ....

Many thanks in advance
CHEERS> SAM

Syntax error or mismatched braces?

Instruments error Script threw an uncaught JavaScript error: Unexpected token 'else' on line 222 of uiautomation-ext.js

Commit 16a0344 has bug.
Commit dca3a4f is ok.

Please someone could star or unstar?

I updated to iOS 8 SDK, and also updated tuneup js, but I got following error when I try to run automation:
ruby(22747,0x7fff76253300) malloc: *** error for object 0x100801a08: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
./runTestCase.sh: line 119: 22747 Abort trap: 6 $BASEDIR/automation/tests/tuneup/test_runner/run $PROJECT $BASEDIR/automation/tests/$PROJECT/$TESTCASE.js $BASEDIR/automation/results/ $TEMPLATE_OPTION $PLATFORM_OPTION -c -v

any idea? Thanks/

Would like to see this as a pod. https://github.com/CocoaPods/CocoaPods/wiki/A-pod-specification

Bundle paths are a pain. The problem with finding them automatically is that there may be multiple bundles to choose from. I usually want the newest. Below, I apply a rule of thumb for simulator vs. device bundles. A command line option would be needed for app_name, or a convention for the app_bundle parameter (e.g. if app_bundle is a name without path or extension that would be interpreted as the app_name).

Below I print all the found bundle paths. You probably want that only when --verbose is specified...

mdfind also has some surprises. The Finder settings are different on my 2 Macs, so kMDItemDisplayName ends in ".app" on one of them, but not the other. (Try mdls too see). I ended up checking both kMDItemDisplayName and kMDItemAlternateNames. I'm not sure it's bulletproof, though. And I have seen people reporting issues related to indexing and symbolicatecrash, so mdfind may not work for everyone.

  # List existing bundles, newest first
  bundle_paths = `mdfind -0 'kMDItemDisplayName == "#{app_name}.app" || kMDItemAlternateNames == "#{app_name}.app"' | xargs -0 ls -dt`.split(/\n/)
  puts "Found bundles (newest first):", bundle_paths if bundle_paths.size > 0
  if options.device
    matching_bundles = bundle_paths.grep(/\/(Debug|Release)-iphoneos\//)
  else
    matching_bundles = bundle_paths.grep(/\/iPhone Simulator\/|\/(Debug|Release)-iphonesimulator\//)
  end
  if matching_bundles.size > 0
    app_bundle = matching_bundles[0]
  end

This is on iOS 9.1. Seeing this error:

2015-11-02 23:00:32 +0000 Error: typeString caught error: VerboseError: Cannot perform action on invalid element: UIAElementNil from target.frontMostApp().keyboard().keys()["Delete"]

The addition of the following line has significantly slowed the start-up time of my tests (imposing a 20 second pause):
https://github.com/alexvollmer/tuneup_js/blob/master/uiautomation-ext.js#L5

I may be missing something - is there some way to restore normal performance (configuration change or otherwise)?

I am using tuneup_js in my test script. I am wondering if it is possible to pass input arguments to javascript.

There are actually two questions.

1. Is it possible to pass input arguments to java script (test_script.js in the following example command)? Note that this is not about input arguments to the application?

instruments -w my_device_id -t /path_to/Automation.tracetemplate my_app -e UIASCRIPT test_script.js -e UIARESULTSPATH result_path

1. If the answer to 1 is true, how can I pass the command line input arguments for javascript into Tuneup_js test function used in the javascript.

The version of xcode is 4.2
Thanks a lot

Does anyone have an example script using tuneup.js? It would be really helpful to see how it's being used

I was trying to extend UIALogger and it sems it canno be extended. Here is the error I am receiving:

Script threw an uncaught JavaScript error: 'undefined' is not an object (evaluating 'destination[property] = source[property]') on line 6 of lang-ext.js

To ensure the function I was adding was ok, I was able to extend UIATarget with my function successfully.

Any ideas ?

Sorry, this is more a question than an issue.

When we run instruments from command line. we use -e UIARESULTSPATH to specify where the results will be saved. However, after we run it, only the .plist files are save to the folder I pointed it to. Instruments still save all the .trace file to the default location. The .trace file is the file I can open and view in Instruments.

I'm just wondering is there a way to save the .trace file to where I want it to be?

I would like to add basic setup teardown functionality to the tests.
I have a working code which checks if there is a setup, teardown function is defined it calls them before the execution, which is similar to the other unit test frameworks.
Any thoughts on this ? If we have to support for suite level setup teardown what should be the preferred approach ?

I had small problem with pgrep. My system didn't come with it pre installed. So I had to do two things before I could move on

1. brew install proctools (assumes homebrew installed)
2. change the pgrep directory to /usr/local/bin/pgrep -> line 117 in run

Would you mind to give us some example of how to use the extend functions in automation-ext.js?

Hi,

I've used imageAsserter to compare two images. Although the pic in screen_diff showed almost all red, the assertionPassed still returns True. Do you have any idea why it is like this? My assumption is that once the red area exceeds certain amount, it would throw false.

Regards,
Yamei

As of iOS 5.0, you can type a string using target.frontMostApp().keyboard().typeString(). The typeString method we add on UIATextView and UIATextField should check for the existence of the keyboard method, and use it if possible.

Thanks for this great library, which makes scripts more like test cases. Now I am wondering if it can do more.

WIth UIAutomation 4.2, I am able to run tests in command line. However, I have to look through the output to check if all tests passed. This does not work for Continuous Integration build system. Is there anything available to help my case? Or I have to write an extra piece of software to parse the output then decide to fail the build or not. It helps even if tuneup_js can print out a summary that says if any test failed.

To help understanding. The following illustrates how I tick off test runs on command line and its output.

Thanks a lot!

Chexin

$ instruments -w 43f5e9e749005e145fcf0f1b5d9346e2e2b4ecf4 -t ~/Library/"Application Support"/Instruments/Templates/Template_login_tuneup_II MyApp

2012-01-12 11:31:25 -0800 Start: Test checking login then cancel
2012-01-12 11:31:25 -0800 Debug: target.frontMostApp().keyboard().typeString("[email protected]")
2012-01-12 11:31:28 -0800 Debug: target.frontMostApp().mainWindow().secureTextFields()["password"].tap()
2012-01-12 11:31:28 -0800 Debug: target.frontMostApp().mainWindow().secureTextFields()["password"].tap()
2012-01-12 11:31:28 -0800 Debug: target.frontMostApp().keyboard().typeString("password")
2012-01-12 11:31:29 -0800 Debug: target.frontMostApp().mainWindow().buttons()["rememberMe"].tap()
2012-01-12 11:31:29 -0800 Debug: target.frontMostApp().mainWindow().buttons()["launch"].tap()
2012-01-12 11:31:31 -0800 Debug: target.frontMostApp().mainWindow().buttons()["cancel"].tap()
2012-01-12 11:31:31 -0800 Pass: Test checking login then cancel

2012-01-12 11:31:31 -0800 Start: Test checking login then cancel (II)
012-01-12 11:31:25 -0800 Debug: target.frontMostApp().keyboard().typeString("[email protected]")
2012-01-12 11:31:28 -0800 Debug: target.frontMostApp().mainWindow().secureTextFields()["password"].tap()
2012-01-12 11:31:28 -0800 Debug: target.frontMostApp().mainWindow().secureTextFields()["password"].tap()
2012-01-12 11:31:28 -0800 Debug: target.frontMostApp().keyboard().typeString("password")
2012-01-12 11:31:36 -0800 Debug: target.frontMostApp().mainWindow().buttons()["rememberMe"].tap()
2012-01-12 11:31:36 -0800 Debug: target.frontMostApp().mainWindow().buttons()["launch"].tap()
2012-01-12 11:31:37 -0800 Debug: target.frontMostApp().mainWindow().buttons()["cancel"].tap()
2012-01-12 11:31:37 -0800 Pass: Test checking login then cancel (II)

Instruments Trace Complete (Duration : 16.793417s; Output : /Users/somebody/workspace/SVN/rtoss-trunk-1/build-client-iOS-appstore_d/ui_testing/instrumentscli14.trace)

Javascript 1.6 (supported by Instruments) already has a map method for arrays, but it sends parameters in a different order. lang-ext.js overwrites that definition, which breaks other code (e.g. underscore.js).

Could you remove the conflicting definition of map, change the parameters or prefix it?
Alternatively, you could move "extend" to a different file, so lang-ext.js becomes optional.

/Users/johan/UIAutomation/tuneup_js/test_runner/xunit_output.rb:78:in add_status': undefined method<<' for nil:NilClass (NoMethodError)
from /Users/johan/UIAutomation/tuneup_js/test_runner/run:237
from /Users/johan/UIAutomation/tuneup_js/test_runner/run:237:in each' from /Users/johan/UIAutomation/tuneup_js/test_runner/run:237 from /Users/johan/UIAutomation/tuneup_js/test_runner/run:228:inspawn'
from /Users/johan/UIAutomation/tuneup_js/test_runner/run:228
from /Users/johan/UIAutomation/tuneup_js/test_runner/run:227:in `chdir'
from /Users/johan/UIAutomation/tuneup_js/test_runner/run:227