plu / simctl

Ruby interface to xcrun simctl

CV - Johannes Plunien

Summary

• Working in Software Engineering since 2004, with focus on iOS since 8 years
• Had different roles from backend/frontend to DevOps to iOS engineer
• Living north of Munich with my wife and two daughters, working remotely for eBay Kleinanzeigen (Berlin) as an iOS Developer

I see myself as a generalist, not afraid to touch other technologies than what I was hired for. Often launching Android Studio, comparing the Android app to the iOS app in my current job to debug something, or to check with Charles Proxy how and when they make a request to the backend to ensure the two different platforms behave the same.

Recent Open Source Contributions

• cli/cli - GitHub command line interface - Support gh release edit
• apple/swift-package-manager - Swift Package Manager - Limit concurrent HTTP requests when downloading binary dependencies
• fastlane/fastlane - fastlane - Fix duplicate use_system_scm option

Experience

+++ means I got a lot in touch with that technology while being in this position, does not mean I'm an expert in it.

05/2014 - today: Senior iOS Developer @ eBay Kleinanzeigen (remote since 2016)

+++ Swift
++  Jenkins, Objective-C, Ruby
+   GitHub Actions, Go, Java, Groovy, TypeScript

11/2013 - 04/2014: Senior Software Engineer @ metaquark GmbH

+++ Objective-C, Ruby
++  Bamboo, Capistrano
+   Puppet

01/2012 - 10/2013: Senior Software Engieer @ XING

+++ Objective-C, Perl, Ruby
++  Capistrano, Chef, Jenkins
+   VMware

04/2011 - 12/2011: Software Consultant @ ConSol MENA

+++ ColdFusion, Perl
++  Microsoft SQL Server

07/2009 - 03/2011: Senior Software Engineer @ XING

+++ MySQL, Perl, Ruby
++  German, Jenkins
+   RabbitMQ

09/2004 - 06/2009: Software Consultant @ ConSol Software GmbH

+++ Perl
++  Oracle

Side Projects

Current

simctl - Ruby interface to xcrun simctl (used by fastlane) - not very actively maintained, at least no new features are being added currently.

Past

Between 05/2017 and 07/2018 I've built and maintained the iOS Poloniex App (and Android App) as my own product in my personal App Store Account. Was finally acquired by Poloniex in 07/2018. Also started the Android App and implemented the first few screens, until I finally asked a friend for help, who knows a lot more about Android than I do.

simctl's Issues

Update json gem to resolve CVE

Latest version of simctl (1.6.8) is using json 2.2.0 which is vulnerable to this CRITICAL cve https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10663

It looks like Dependabot already updated the json gem to use json 2.3.1 which will patch the vulnerability.

Any chance we could get a new release of this gem so that the CVE can be resolved? We use fastlane, and fastlane depends on this gem, so it's coming up in some vulnerability scans we run.

Ruby 3.2 support

simctl doesn't work with Ruby 3.2.

Error:

NoMethodError: undefined method `=~' for false:FalseClass
.gem/ruby/3.2.0/gems/simctl-1.6.8/lib/simctl/device.rb:30:in `!~'
.gem/ruby/3.2.0/gems/simctl-1.6.8/lib/simctl/device.rb:30:in `available?'

The method =~ was removed on Ruby 3.2

Simctl doesn't work with XCode 8.1 / iOS 10.1

Trying to launch a simulator with iOS10.1 in XCode 8.1 build tools fails with:

#<SimCtl::Runtime:0x007fb11027c3d8>
#<SimCtl::DeviceType:0x007fb110285118>
Creating device Test device
/Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/executor.rb:11:in `read': execution expired (Timeout::Error)
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/executor.rb:11:in `block in execute'
	from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/open3.rb:217:in `popen_run'
	from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/open3.rb:99:in `popen3'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/executor.rb:10:in `execute'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/command/list.rb:27:in `list_devices'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/command/list.rb:11:in `device'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl.rb:32:in `method_missing'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/device.rb:158:in `block (2 levels) in wait!'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/device.rb:157:in `loop'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/device.rb:157:in `block in wait!'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/device.rb:156:in `wait!'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/command/create.rb:20:in `create_device'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl/command/reset.rb:22:in `reset_device'
	from /Library/Ruby/Gems/2.0.0/gems/simctl-1.5.5/lib/simctl.rb:32:in `method_missing'
	from scripts/multisim_test.rb:84:in `<main>'

xcode-select -p with unescaped characters

Hey Johannes,

Within our server setup we use the SimCtl gem to spin up simulators for test automation. One of the problems we have is that we support multiple versions of xcode. Each version of xcode is separated by a space, and then the version number (so you get Xcode 9.1.0, Xcode 8.3.3, etc).

This however breaks the SimCtl gem since the var HOME in path.rb doesn't escape shell specific characters.

A simple fix would be to use shellwords to escape the outcome of xcode-select -p

Best regards,
Sander Saelmans

Unable to shutdown device in current state: Creating

Not sure if this is a simctl problem or something with the Simulator.app or CoreSimulatorService itself. Sometimes when we try to delete simulators we run into the following error:

[Hanamura] Deleting simulator with identifier 887FF917-1981-4508-85FA-0B31DA7FA6B8
/Users/jenkins/.rvm/gems/ruby-2.3.1/gems/simctl-1.6.1/lib/simctl/executor.rb:14:in `block in execute': An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=163): (RuntimeError)
Unable to shutdown device in current state: Creating
	from /Users/jenkins/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/open3.rb:205:in `popen_run'
	from /Users/jenkins/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/open3.rb:95:in `popen3'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/gems/simctl-1.6.1/lib/simctl/executor.rb:10:in `execute'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/gems/simctl-1.6.1/lib/simctl/command/shutdown.rb:9:in `shutdown_device'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/gems/simctl-1.6.1/lib/simctl.rb:39:in `method_missing'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/gems/simctl-1.6.1/lib/simctl/device.rb:174:in `shutdown'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/lib/hanamura/simulator.rb:41:in `teardown'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/lib/hanamura/command/test_all.rb:57:in `block (2 levels) in run'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/lib/hanamura/command/test_all.rb:51:in `each'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/lib/hanamura/command/test_all.rb:51:in `block in run'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/lib/hanamura/command/test_all.rb:44:in `each'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/lib/hanamura/command/test_all.rb:44:in `run'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/gems/claide-1.0.1/lib/claide/command.rb:334:in `run'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/lib/hanamura.rb:7:in `run'
	from /Users/jenkins/Jenkins/workspace/ios-monorepo-monocomputer-master-tests/tooling/hanamura/bin/hana:6:in `<top (required)>'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/bin/hana:23:in `load'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/bin/hana:23:in `<main>'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
	from /Users/jenkins/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'

This is more or less how we setup/tear down simulators:

    def self.setup(destination, enable_hw_keyboard, use_fbsimctl = false, output)
      hash = self.parse(destination)
      Logger.log("Starting simulator with values #{hash}", output)
      if hash['OS'].eql? 'latest'
        simulator_runtime = SimCtl::Runtime.latest('ios')
      else
        simulator_runtime = SimCtl.runtime(name: "iOS #{hash['OS']}")
      end
      simulator = SimCtl.reset_device "Unit Tests @ #{hash['name']} #{hash['OS']}", SimCtl.devicetype(name: hash['name']), simulator_runtime
      if enable_hw_keyboard
        simulator.settings.update_hardware_keyboard!(enable_hw_keyboard)
      end
      Logger.log("Starting simulator with identifier #{simulator.udid}", output)
      unless use_fbsimctl
        simulator.launch
        simulator.wait(90) { |device| device.state == :booted && device.ready? }
      else
        cmd = [
          "/usr/local/bin/fbsimctl #{simulator.udid} boot"
        ]
        Executor.execute!(cmd, output)
      end
      simulator
    end

    def self.teardown(destination, output)
      hash = self.parse(destination)
      Logger.log("Searching for simulator with values #{hash}", output)
      simulator = SimCtl.device(name: "Unit Tests @ #{hash['name']} #{hash['OS']}")
      if simulator.nil?
        Logger.log("Simulator not found", output)
        return
      end
      Logger.log("Deleting simulator with identifier #{simulator.udid}", output)
      simulator.shutdown
      simulator.kill
      simulator.wait { |device| device.state == :shutdown }
      simulator.delete
    end

fastlane-plugin-simctl uses old version

I followed the instructions on this page. When I run bundle install --deployment, it says Installing fastlane-plugin-simctl 0.2.0 despite saying Installing simctl 1.6.5 above it. The file vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-simctl-0.2.0/lib/fastlane/plugin/simctl/helper/simctl_helper.rb contains the old version of that file instead of the one that is bundled with 1.6.5. The source for the plugin is also installed under vendor/bundle/ruby/2.4.0/gems/simctl-1.6.5/fastlane-plugin-simctl/ but it seems to be using the source from the plugin folder.

It seems fastlane-plugin-simctl needs to also be updated and pushed to Rubygems.org

Initing a new Device with UUID

I am trying to stash a Device to later use in a instance of Fastlane. Would it be possible to recreate a Device instance if I stash the UUID of a sim?

If not implemented, I would guess xcrun simctl list --json could be parsed to verify if a given UUID is valid and get the current status, etc.

Add support for `simctl pair`

$ xcrun simctl pair help
Create a new watch and phone pair.
Usage: simctl pair <watch device> <phone device>

I didn't see reference to the word "pair" in this repo, so it looks like it's not supported. I can easily work around it in a ruby script using:

`xcrun simctl pair #{watch_device.udid} #{phone_device.udid}`

But it would be nice not to have to.

Add support for creating device with a specific architecture

Hi, I'm happily using this CLI and i noticed at a recent Xcode upgrade that some of my project needed to run on simulators which run with architecture of arch x86_64.

It would be really nice if "create_device" function had another param called "arch" or something of the sort.

Thanks for the help in advance

Running concurrent jobs on Jenkins with fastlane/scan puts CoreSimulatorService in a deadlock state

Duplicate of fastlane/fastlane#14528 (comment)

Issue Description

We rely on fastlane/scan to perform the tests on our Jenkins CI server.

• When running non concurrent jobs (1 Jenkins job at a time), we got no issue.
• When running concurrent jobs (Multiple fastlane/scan at the same time), jobs get blocked.

We rely on fastlane-plugin-simctl to execute xcrun simctl commands.

We execute up to 12 concurrent Jenkins jobs.

Most of the time, the Jenkins jobs run fine.
Sometimes the xcrun simctl commands are not responding at some point, so the jobs are stuck during simulator creation or deletion.

The process com.apple.CoreSimulator.CoreSimulatorService seems stuck in a deadlock.
We ran sample on the process and the sample analysis did exhibit :

Dispatch Thread Soft Limit: 64 reached in 8117 of 8121 samples -- too many dispatch threads blocked in synchronous operations

Killing the simulators does not end the deadlock. We have to
killall -9 com.apple.CoreSimulator.CoreSimulatorService

Command executed

private_lane :jenkins_test do |options|
	setup_jenkins(result_bundle: false)
	sh 'defaults write com.apple.iphonesimulator ConnectHardwareKeyboard 0'
	simctl(type: 'iPhone Xʀ',
		   block: lambda { |action, device|
		   action.scan(
					   skip_detect_devices: true,
					   destination: ["id=#{device.udid}"],
					   workspace: "#{Dir.pwd}/MyApp.xcworkspace",
					   scheme: options[:scheme],
					   xcargs: "-UseModernBuildSystem=NO",
					   clean: true,
					   code_coverage:true,
					   derived_data_path: '.',
					   output_directory: "./test_output/",
					   fail_build: false,
					   result_bundle: "TestResults" # To generate test reports
					   )
		   })
end

$ bundle exec fastlane jenkins_test scheme:"MyApp" configuration:"Debug"

Environment

🚫 fastlane environment 🚫

Stack

Key	Value
OS	10.14.4
Ruby	2.3.7
Bundler?	false
Git	git version 2.20.1 (Apple Git-117)
Installation Source	/usr/local/bin/fastlane
Host	Mac OS X 10.14.4 (18E226)
Ruby Lib Dir	/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib
OpenSSL Version	LibreSSL 2.6.5
Is contained	false
Is homebrew	false
Is installed via Fabric.app	false
Xcode Path	/Applications/Xcode.app/Contents/Developer/
Xcode Version	10.2

System Locale

Variable	Value
LANG	en_US.UTF-8	✅
LC_ALL	en_US.UTF-8	✅
LANGUAGE	en_US.UTF-8	✅

Latest iOS incorrect

After installing XCode 8.1, when we do SimCtl::Runtime.latest(:iOS) we get iOS 8.4, where we should get iOS 10.1.

Any ideas why this is?

kill/shutdown 1 simulator

Hey Johannes,

Suppose we have 5 simulators running, are you aware of any command to kill/shutdown only a single of them .. If you kill the process/simctl shutdown, all simulators will be killed :\

Add support for taking screenshots

As of Xcode 8.2, we can now take screenshots through the simctl tool by doing something like:

$ xcrun simctl io booted screenshot

It would be nice if the gem also provided an interface for grabbing this screenshot, with options to save to a named file, but perhaps also to grab the bytes from stdout.

I imagine a new method on the Device class would be something like:

device.screenshot(type: "png", filename: "foo.png")

install! triggers: Mach error -308 - (ipc/mig) server died

Hi Johannes,

We're seeing a quite frequent problem when trying to install apps in the simulator and I'm wondering if you've ever encountered it:

We're running the following command to start the simulator and install the app:

# launch the simulator
@device.launch!
@device.wait! {|d| d.state == :booted}

# install the app
@device.install!(@app_path)

Quite often, the install! step fails and we get the following output:

An error was encountered processing the command (domain=NSMachErrorDomain, code=-308):
The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
(StandardError)

I found this thread on the dev forums https://forums.developer.apple.com/message/70514#70514

That lead me to these crash reports: https://gist.github.com/garriguv/d6a35a39ec87e33254917948c130d357

Not super helpful but I've filed a radar.

I've had to add a pretty long sleep before trying to install the app. That reduces the occurrences but it obviously not optimal...

dyld: app was built for iOS 10.0 which is newer than this simulator 9.2

Hi,

I've been experimenting with

device = SimCtl.device(udid: device_id)
device.boot
device.wait(timeout) { |d| d.state == :booted && d.ready? }

... to get a device up and running. It works on 64bit devices, but fails on 32 bit devices.

The following ruby script on the cosole outputs the details

ENV['SIMCTL_DEBUG']="true"
device_id="C21A5716-6FDB-4DB9-90F1-975BDB66EC80"
timeout = 240
device = SimCtl.device(udid: device_id)
device.boot
device.wait(timeout) { |d| d.state == :booted && d.ready? }

output

2.1.3 :010 > device.wait(timeout) { |d| d.state == :booted && d.ready? }
xcrun simctl list -j devices
xcrun simctl list -j runtimes
xcrun simctl spawn C21A5716-6FDB-4DB9-90F1-975BDB66EC80 /Applications/Xcode/8.0.0/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/bin/launchctl list
RuntimeError: dyld: app was built for iOS 10.0 which is newer than this simulator 9.2
Child process terminated with signal 5: Trace/BPT trap

	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/executor.rb:14:in `block in execute'
	from /Users/build/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/open3.rb:199:in `popen_run'
	from /Users/build/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/open3.rb:93:in `popen3'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/executor.rb:10:in `execute'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/command/spawn.rb:13:in `spawn'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl.rb:39:in `method_missing'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/device.rb:183:in `spawn'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/device_launchctl.rb:12:in `list'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/device.rb:116:in `ready?'
	from (irb):10:in `block in irb_binding'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/device.rb:199:in `block (2 levels) in wait'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/device.rb:198:in `loop'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/device.rb:198:in `block in wait'
	from /Users/build/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/timeout.rb:91:in `block in timeout'
	from /Users/build/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/timeout.rb:35:in `block in catch'
	from /Users/build/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/timeout.rb:35:in `catch'
	from /Users/build/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/timeout.rb:35:in `catch'
	from /Users/build/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/timeout.rb:106:in `timeout'
	from /Users/build/.rvm/gems/ruby-2.1.3/gems/simctl-1.6.1/lib/simctl/device.rb:197:in `wait'
	from (irb):10

All using xcode 8.0.0, on ElCapitain

Find device by identifier

Hi, is there a way to find a specific simulator based on a udid?

Calling device.runtime throws exception for runtimes that are not installed

If you have old simulators from an earlier version of Xcode and no longer have that runtime installed, calling device.runtime will throw an exception:

SimCtl::RuntimeNotFound: Could not find a runtime matching {:identifier=>"com.apple.CoreSimulator.SimRuntime.iOS-11-1"}

I'm not sure what the "correct" behavior is here. The simulator exists (has a directory on-disk and is reported by simctl), but has an invalid runtime.

execution expired error while create simulator

Hello!
I'm facing the issue when fastlane exit before the simulator has been created.


So fastlane exit and do not delete the sim.
And for the next build sim present and such an issue not be repeated this time.
I saw that timeout for this - 90 seconds: https://github.com/plu/simctl/blob/master/fastlane-plugin-simctl/lib/fastlane/plugin/simctl/helper/simctl_helper.rb#L7
Could the timeout be as a parameter so users can increase it on their needs?
As well I saw the warmup script that as I understand should prepare a simulator but I don't see it has been used anywhere.
What is the purpose of it or how to use it in fastlane?
https://github.com/plu/simctl/blob/master/lib/simctl/command/warmup.rb

Error trying to launch a recently created device: device.plist not readable!

I tried to follow the example provided in the README.md and got the following:

/Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/CFPropertyList-2.3.2/lib/cfpropertylist/rbCFPropertyList.rb:348:in `load': File /Users/renzo.crisostomo/Library/Developer/CoreSimulator/Devices/A326F174-3482-4A2B-A75C-3FF29DFB47F8/device.plist not readable! (IOError)
    from /Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/CFPropertyList-2.3.2/lib/cfpropertylist/rbCFPropertyList.rb:253:in `initialize'
    from /Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/simctl-1.5.0/lib/simctl/device.rb:146:in `new'
    from /Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/simctl-1.5.0/lib/simctl/device.rb:146:in `plist'
    from /Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/simctl-1.5.0/lib/simctl/device.rb:30:in `devicetype'
    from /Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/simctl-1.5.0/lib/simctl/command/launch.rb:22:in `launch_device'
    from /Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/simctl-1.5.0/lib/simctl.rb:22:in `method_missing'
    from /Users/renzo.crisostomo/.rvm/gems/ruby-2.3.1/gems/simctl-1.5.0/lib/simctl/device.rb:67:in `launch!'
    from test.rb:13:in `<main>'

Any ideas what could go wrong? I make sure to have the latest simctl and CFPropertyList.