feature request: add `dyld uuid` command about ipsw

[blacktop]

I've added a --json flag to ipsw dyld info (the long term plan was always to expose a RESTful API from ipsw so others could query it from other langs/platforms)

$ ipsw dyld info test-caches/iPhone12,1_N104AP_19A5261w/dyld_shared_cache_arm64e --json | jq .

{
  "magic": "dyld_v1  arm64e\u0000",
  "uuid": "F5097CDC-9388-382F-8D13-AAE184497224",
  "platform": "iOS",
  "max_slide": 57638912,
  "mappings": [
    {
      "Name": "__TEXT",
      "Address": 6442450944,
      "Size": 1339211776,
      "FileOffset": 0,
      "SlideInfoOffset": 0,
      "SlideInfoSize": 0,
      "Flags": 0,
      "MaxProt": 5,
      "InitProt": 5
    },
    {
      "Name": "__DATA_CONST",
      "Address": 7815217152,
      "Size": 49840128,
      "FileOffset": 1339211776,
      "SlideInfoOffset": 1548271616,
      "SlideInfoSize": 32768,
      "Flags": 4,
      "MaxProt": 3,
      "InitProt": 1
    },
<SNIP>
  ],
  "dylibs": [
    {
      "name": "/usr/lib/system/libdispatch.dylib",
      "version": "1319.0.0",
      "uuid": "FFA2D8B7-6140-3795-9FAD-8615E27E5BD2"
    },
    {
      "index": 1,
      "name": "/usr/lib/system/libdyld.dylib",
      "version": "1.0.0",
      "uuid": "0943406A-75D8-3BA2-AEAC-5CA47E119B45"
    },
    {
      "index": 2,
      "name": "/usr/lib/libicucore.A.dylib",
      "version": "68.2.0",
      "uuid": "9AEA2189-8B39-3F26-835D-947D82717FA0"
    },
    {
      "index": 3,
      "name": "/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation",
      "version": "1835.100.0",
      "uuid": "4B514B9D-2B35-32CA-A93F-481D3A903177"
    },
    <SNIP>

[doronz88]

This is so awsome! thank you :)
Also I think that --json data should be the same I see without the json flag, since you usually would assume an optional output flag doesn't add data for viewing. Since this flag is much more than output change it at least would deserve better documentation in the help message.

Also, I think the output should be indented so people can view it nicely (especially if data is not available otherwise).

And the RESTful api sounds amazing!

[doronz88]

I've just noticed that when running ipsw dyld -l -V it gives out more information and includes the loading addresses. It would be nice to have that also included in the json format

[blacktop]

Added load address to JSON and am now respecting the other flags in the dyld info cmd. It's not a perfect reproduction of the non-json output as that will require more careful thought and a refactor of my "formatter" code (because now there is complex logic there to avoid showing info that is all zeros (removed in a newer version of dyld_header_format etc)

So we can leave this issue open as a reminder to me to REALLY add JSON full output for this cmd and eventually every cmd in the path to a RESTful API.

[blacktop]

Also, I think the output should be indented so people can view it nicely (especially if data is not available otherwise).

So I always felt like if someone wanted to read the JSON you'd run it through jq otherwise you would want the JSON minimized to minimize the overall size of the data to then be consumed somewhere else? Do you disagree? I know a lot of your tools output pretty JSON, but I usually don't read LONG json myself and if I do I use jq

[doronz88]

Usually I want my tools to be as self-contained as possible, meaning the users don't require any extra binaries for their usage. I chose to output as JSON so it's easy to query from other programs and also because it's pretty readable as-is and doesn't require too much effort for formatting the best output. I think both approaches are just as fine as long as the tool is consistent in their usage for every command - so it's fine as it is now :)

Also, I started implementing the feature I discussed earlier and just came to the realization the UUID of the images as seen in their LOAD command isn't the one I get from getting the stackshot. I used your Dyld.bt to debug it and surprise:

As it turns out, all the binaries from the system partition (even non-dylibs such as /sbin/launchd and eveyone else) are also found in the DSC but in another region that isn't yet explored. The UUID I get when querying the stackshot from the device is the one found there and is different from the mach-o's UUID as it appears in the DSC LOAD command. I'm currently still trying to figure out how to parse this section.

BTW, this also means that when XCode tries to symbolicate dumps from the device it will use these other-UUIDs.

[blacktop]

Wow! I'm VERY interested in what you find. There is still this mysterious NEW data structure in the dsc that I cant figure out. In Dyld.bt I refer to it as unknown10. Maybe it's somehow related??

[blacktop]

There is also a new DriverKit dsc that I haven't looked at yet. Maybe take a look at that? I don't extract it yet with ipsw so you'd have pull it out manually. I'll also take a look soon

[blacktop]

nvm, it's not the DriverKit dsc stuff:

Header
======
Magic             = "dyld_v1  arm64e"
UUID              = E1C123EE-360D-3345-AAC1-437D66355AF1
Platform          = driverKit
Max Slide         = 0x7FC98000 (ASLR entropy: 17-bits)
Num SubCaches     = 0
SubCache Group ID = 0xb28
Sym SubCache UUID = 85B8C120-FB40-3750-9FAC-0C96C06D1303

Local Symbols (nlist array):      0MB, offset:  0x0004F82A0 -> 0x000514910
Local Symbols (string pool):      0MB, offset:  0x000514910 -> 0x00054DC07
Code Signature:                   0MB, offset:  0x0004F4000 -> 0x0004F8000
ImagesText Info ( 27 entries):   0KB, offset:  0x000000468 -> 0x0000007C8
Patch Info:                       0MB, address: 0x1864DC370 -> 0x1864EDA9C
Closures:                         0MB, address: 0x1864EF548 -> 0x1864F1960
Closures Trie:                    0KB, address: 0x1864F1960 -> 0x1864F1BA0
Shared Region:                    0GB, address: 0x180000000 -> 0x1864F4000

Mappings
========
|     SEG      | INITPROT | MAXPROT |        SIZE         |          ADDRESS           |       FILE OFFSET        |  SLIDE INFO (V3) OFFSET  | FLAGS |
|--------------|----------|---------|---------------------|----------------------------|--------------------------|--------------------------|-------|
| __TEXT       | r-x      | r-x     | 0x00368000 (3.6 MB) | 0x180000000 -> 0x180368000 | 0x00000000 -> 0x00368000 |                          | 0     |
| __DATA_CONST | r--      | rw-     | 0x00008000 (33 kB)  | 0x182368000 -> 0x182370000 | 0x00368000 -> 0x00370000 | 0x003d4000 -> 0x003d8000 | 4     |
| __DATA       | rw-      | rw-     | 0x00028000 (164 kB) | 0x184370000 -> 0x184398000 | 0x00370000 -> 0x00398000 | 0x003dc000 -> 0x003e0000 | 0     |
| __AUTH       | rw-      | rw-     | 0x00010000 (66 kB)  | 0x184398000 -> 0x1843a8000 | 0x00398000 -> 0x003a8000 | 0x003e4000 -> 0x003e8000 | 1     |
| __AUTH_CONST | r--      | rw-     | 0x0002c000 (180 kB) | 0x1843a8000 -> 0x1843d4000 | 0x003a8000 -> 0x003d4000 | 0x003ec000 -> 0x003f0000 | 5     |
| __LINKEDIT   | r--      | r--     | 0x00120000 (1.2 MB) | 0x1863d4000 -> 0x1864f4000 | 0x003d4000 -> 0x004f4000 |                          | 0     |
| __TEXT       | r-x      | r-x     | 0x00004000 (16 kB)  | 0x00000000 -> 0x00004000   | 0x004f4000 -> 0x004f8000 |                          | 0     |

Images
======
   1:  0X180002000  /System/DriverKit/System/Library/Frameworks/AudioDriverKit.framework/AudioDriverKit  (1.0.0) uuid: CB2925C9-93AB-308B-BA34-6B0363CC1425
   2:  0X18002A000  /System/DriverKit/System/Library/Frameworks/DriverKit.framework/DriverKit  (1.0.0) uuid: 29338F76-AB43-3139-A29F-FD88B2EFD40D
   3:  0X180066000  /System/DriverKit/System/Library/Frameworks/HIDDriverKit.framework/HIDDriverKit  (1.0.0) uuid: 8DC32462-1467-3060-9797-4A9B02544BE3
   4:  0X1800A6000  /System/DriverKit/System/Library/Frameworks/HIDDriverKit.framework/HIDDriverKit_development  (1.0.0) uuid: E1E65504-4057-3906-83A8-8F46EDCED0A7
   5:  0X1800E6000  /System/DriverKit/System/Library/Frameworks/NetworkingDriverKit.framework/NetworkingDriverKit  (1.0.0) uuid: 2A2EF7BD-A311-32EB-9B45-AEB447460D47
   6:  0X1800FE000  /System/DriverKit/System/Library/Frameworks/NetworkingDriverKit.framework/NetworkingDriverKit_development  (1.0.0) uuid: C5BE6504-6D13-3DC1-AFE5-2BAE4F5C3EB0
   7:  0X180116000  /System/DriverKit/System/Library/Frameworks/PCIDriverKit.framework/PCIDriverKit  (1.0.0) uuid: F62423DC-8927-374A-B142-238240BCBF0E
   8:  0X18011A000  /System/DriverKit/System/Library/Frameworks/SerialDriverKit.framework/SerialDriverKit  (1.0.0) uuid: 3E5DFF96-AC40-3948-93CD-D5C7777318C4
   9:  0X180122000  /System/DriverKit/System/Library/Frameworks/USBDriverKit.framework/USBDriverKit  (1.0.0) uuid: 6B4B1E32-BEE9-37D1-904F-0F15A0A8892D
  10:  0X18012E000  /System/DriverKit/System/Library/Frameworks/USBSerialDriverKit.framework/USBSerialDriverKit  (1.0.0) uuid: 7B6B2149-E495-3E76-A8D9-48A2648E13A2
  11:  0X180132000  /System/DriverKit/System/Library/PrivateFrameworks/MallocStackLogging.framework/MallocStackLogging  (1.0.0) uuid: 7EECF315-A5F0-3B5F-A701-2EC4B0E32614
  12:  0X18013E000  /System/DriverKit/usr/lib/libSystem.dylib  (1308.0.0) uuid: 0128A55B-9F23-3B54-BC9C-C4390A5266D7
  13:  0X180142000  /System/DriverKit/usr/lib/libc++.dylib  (907.1.0) uuid: BB0AE5B9-A262-3C00-98BC-A6C9526F4864
  14:  0X180156000  /System/DriverKit/usr/lib/libc++abi.dylib  (907.1.0) uuid: 6CCFE9A3-6586-394D-A271-C28A4EE2046C
  15:  0X18016A000  /System/DriverKit/usr/lib/system/libcompiler_rt.dylib  (102.3.0) uuid: 2F52B956-0EA1-3E94-9938-F227B13E3141
  16:  0X18016E000  /System/DriverKit/usr/lib/system/libcorecrypto.dylib  (1167.0.0) uuid: A4B6FCE8-E508-3428-A3CE-2E65C809446C
  17:  0X1801EA000  /System/DriverKit/usr/lib/system/libdispatch.dylib  (1319.0.0) uuid: 3FA9A052-2D0A-3FBC-B072-055B94B0AC5B
  18:  0X180232000  /System/DriverKit/usr/lib/system/libdyld.dylib  (1.0.0) uuid: 4947F012-3A00-3732-A6C3-E2CF7DFAB1C5
  19:  0X180236000  /System/DriverKit/usr/lib/system/libmacho.dylib  (985.0.0) uuid: 71C26403-62F9-3C3A-8B5E-306AA07A1566
  20:  0X18023A000  /System/DriverKit/usr/lib/system/libsystem_blocks.dylib  (79.0.0) uuid: EFB18189-D5B2-3510-BE69-2E43E0C0CD96
  21:  0X18023E000  /System/DriverKit/usr/lib/system/libsystem_c.dylib  (1496.0.0) uuid: F231E79E-3CA8-36E9-A224-17117E18EB4D
  22:  0X1802B2000  /System/DriverKit/usr/lib/system/libsystem_kernel.dylib  (7938.0.0) uuid: 2C6B9C7B-692A-3157-8D3B-365C85008BC0
  23:  0X1802E6000  /System/DriverKit/usr/lib/system/libsystem_m.dylib  (3202.0.0) uuid: 53D900EE-AE7E-3E26-8FEA-C26403C61A55
  24:  0X18031E000  /System/DriverKit/usr/lib/system/libsystem_malloc.dylib  (360.0.0) uuid: D941914F-094D-31DC-95DD-5F2823E585CA
  25:  0X18034A000  /System/DriverKit/usr/lib/system/libsystem_platform.dylib  (269.0.0) uuid: 5ECFEE47-C56F-31EE-94DF-BCF7724C994C
  26:  0X180352000  /System/DriverKit/usr/lib/system/libsystem_pthread.dylib  (484.0.0) uuid: EE73D00F-8138-3846-8E3C-9CC041F52FC1
  27:  0X18035E000  /System/DriverKit/usr/lib/system/libsystem_trace.dylib  (1363.0.0) uuid: 84A38548-7F63-38E4-ACB4-8ED994D618B7

[blacktop]

010 wasn't able to find the string /sbin/launchd in the dsc. I believe if it was in there it would be mentioned somewhere?

[doronz88]

You are right and that was mistake. It's true that many of the binaries from the OS appear there but it seems not all of them. /usr/libexec/lockdownd does appear there with its UUID. Today I'll keep trying to figure out this section structure.
Since this matter now is completely unrelated to the opened issue I rather we discuss it's developement somewhere else. You can PM me if you'd like, or if you rather we still talk in here.

Anyhow, feel free to close the issue :)

[doronz88]

It's a really hack-ish way but it works:
https://github.com/doronz88/pymobiledevice3/pull/90/files

The pymobiledevice3/dsc_uuid_map.json will now contain maps for UUID-imagePath for every unique DSC file so we can now symbolicate return addresses ourselves.

[blacktop]

VERY nice! 👍

[blacktop]

Working on the 🆕 emulator I had a friend tell me I should look at closures as they include initializers (valuable for emulation setup 😉 ) and in so I discovered that data you mentioned a LONG time ago. So these binaries aren't embedded in the shared_cache, but their "closures" are. So for completeness here is that data (which will soon be exposed in a NEW ipsw dyld image command):

NOTE: I cannot yet parse these in the iOS15 caches.

❯ ipsw dyld image 18H17__iPhone9,1_3_iPhone10,1_4/dyld_shared_cache_arm64 /usr/libexec/lockdownd

Name:              /usr/libexec/lockdownd
Flags:             objc|exe|readonly_data|precomp_objc
UUID:              30202A62-07CD-3128-9CCD-CCE69AE5CBBF
CDHash:            300cc709eeeef2cdb62402f21e8d0c0969ff73c1
Code Signature:    offset: 0x000e3f90-0x000e7860, size: 14544
Total VM Size:     0xec000

Mappings:

FILE OFFSET	FILE SIZE	VM SIZE	PROT
0x00000000	0x00000000	0x00000000	---
0x00000000	0x000bc000	0x000bc000	r-x
0x000bc000	0x00014000	0x00014000	-w-
0x000d0000	0x00008000	0x00008000	rw-
0x000d8000	0x00010000	0x00014000	r--

Dependents:

	regular ) /System/Library/PrivateFrameworks/UserManagement.framework/UserManagement
	regular ) /System/Library/Frameworks/CoreTelephony.framework/CoreTelephony
	regular ) /usr/lib/libamsupport.dylib
	regular ) /System/Library/PrivateFrameworks/CrashReporterSupport.framework/CrashReporterSupport
	regular ) /System/Library/Frameworks/LocalAuthentication.framework/LocalAuthentication
	regular ) /System/Library/PrivateFrameworks/CoreDuet.framework/CoreDuet
	regular ) /System/Library/PrivateFrameworks/CoreUtils.framework/CoreUtils
	regular ) /System/Library/PrivateFrameworks/FindMyDevice.framework/FindMyDevice
	regular ) /System/Library/Frameworks/MobileCoreServices.framework/MobileCoreServices
	regular ) /System/Library/PrivateFrameworks/MobileActivation.framework/MobileActivation
	regular ) /System/Library/PrivateFrameworks/SpringBoardServices.framework/SpringBoardServices
	regular ) /System/Library/PrivateFrameworks/ManagedConfiguration.framework/ManagedConfiguration
	regular ) /usr/lib/libextension.dylib
	regular ) /System/Library/PrivateFrameworks/ServiceManagement.framework/ServiceManagement
	regular ) /System/Library/PrivateFrameworks/AppSupport.framework/AppSupport
	regular ) /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
	regular ) /System/Library/Frameworks/IOKit.framework/IOKit
	regular ) /System/Library/Frameworks/SystemConfiguration.framework/SystemConfiguration
	regular ) /System/Library/PrivateFrameworks/MobileStorage.framework/MobileStorage
	regular ) /System/Library/PrivateFrameworks/Bom.framework/Bom
	regular ) /System/Library/Frameworks/Security.framework/Security
	regular ) /System/Library/PrivateFrameworks/MobileWiFi.framework/MobileWiFi
	regular ) /System/Library/PrivateFrameworks/MobileInstallation.framework/MobileInstallation
	regular ) /System/Library/PrivateFrameworks/MobileKeyBag.framework/MobileKeyBag
	regular ) /System/Library/PrivateFrameworks/CoreTime.framework/CoreTime
	regular ) /usr/lib/liblockdown.dylib
	regular ) /usr/lib/libz.dylib
	regular ) /usr/lib/libMobileCheckpoint.dylib
	regular ) /usr/lib/libMobileGestalt.dylib
	regular ) /usr/lib/libAccessibility.dylib
	regular ) /usr/lib/libobjc.dylib
	regular ) /usr/lib/libMobileGestaltExtensions.dylib
	regular ) /usr/lib/librpcsvc.dylib
	regular ) /System/Library/Frameworks/CoreServices.framework/CoreServices

Init Order:

	/usr/lib/system/libsystem_blocks.dylib
	/usr/lib/system/libdispatch.dylib
	/usr/lib/system/libxpc.dylib
	/usr/lib/system/libsystem_trace.dylib
	/usr/lib/librpcsvc.dylib
	/usr/lib/libc++.dylib
	/usr/lib/libobjc.dylib
	/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
	/usr/lib/libnetwork.dylib
	/usr/lib/libextension.dylib
	/System/Library/PrivateFrameworks/CoreDuet.framework/CoreDuet
	/usr/lib/libcryptex_core.dylib
	/usr/lib/libcryptex_interface.dylib
	/usr/lib/libcryptex.dylib
	/System/Library/Frameworks/CoreText.framework/CoreText