sketch-hq / sketch-document

Monorepo for Sketch document JSON Schemas and TypeScript types

.sketch document

Sketch stores documents in .sketch format, a zipped archive of JSON formatted data and binary data such as images.

Sketch file format schemas and APIs.

This project contains the file format specification and APIs to work with Sketch documents and document elements in TypeScript.

• @sketch-hq/sketch-file-format: JSON Schema describing the .sketch document format.
• @sketch-hq/sketch-file-format-ts: TypeScript types to strongly type objects representing Sketch documents, or fragments of Sketch documents in TypeScript projects.
• @sketch-hq/sketch-file: TypeScript APIs to read and write .sketch files.

Development

Pre-release

ℹ️ This section is work in progress until we release our first pre-release. Pre-releases will be used as a staging area to publish changes to the specification required by an as-yet unreleased version of the Sketch Mac app.

1. Create a new branch to track the pre-release, e.g. v5
2. Read the changesets pre-release docs
3. Enter pre-release mode on the branch yarn changeset pre enter {tag}
4. Commit the changes and push the branch. This branch will act as a stand-in for main for all work related to the pre-release
5. PR into the new branch with feature branches, calling yarn changeset as per normal to signal intents to publish. Since this is a pre-release it's likely that we'll be marshalling a major version bump
6. Publishing pre-releases is not automated, so when you're ready to publish the pre-release call yarn changeset version and then yarn release

Release

1. Following changes, run yarn changeset on the feature branch to document changes.
2. If changes have been made to the @sketch-hq/sketch-file-format make sure to include @sketch-hq/sketch-file-format-ts in the changeset.
3. On merge, changesets will produce a release pull-request named Version Packages with all changes since the last release.
4. Merge the release pull-request maintained by the changesets GitHub Action to publish new releases to NPM.

sketch-document's Issues

Build and release schema

Flesh out details on what needs to be done, i.e. GitHub, NPM/GitHub registry, adding a license, questions around private internal specs, etc. (we’re still trying to get sketch GitHub but can fallback to sketch-hq).

Artboard layers can only appear within a page's layers, and no deeper

Missing props in user schema

User document object can also contain,

• expandedSymbolPathsInSidebar
• expandedTextStylePathsInPopover
• expandedLayerStylePathsInPopover
• libraryListCollapsed

No more `changeIdentifier` for symbol masters in version 134?

Hi there,

We noticed that Sketch 69 doesn't reference this property anywhere and it's being removed from older documents when you save them with 69 — but the spec still mentions it. Wondering if that's just an oversight or something.

Thanks!

Add DocumentState to the file Spec

This is a new property on document which will be added by https://github.com/sketch-hq/Sketch/issues/30429

At the moment the only thing this property will contain is information on the cloud share - it'll be empty for normal documents. The contents of cloudShare are the same as that currently written to the User.json file under document/cloudShare. The idea is to remove this information from User.json at some point, but it's still there as well for backward compatibility.

  "documentState" : {
    "cloudShare" : {
      "viewerCanUpdate" : 0,
      "id" : "615164b7-1b51-49e4-be08-33aedabc8150",
      "updatedAt" : "2020-06-17T14:54:49.378Z",
      "owner" : {
        "id" : "fad05bfc-1c06-41f4-9e91-39eb5df06765",
        "__typename" : "PublicUser",
        "updatedAt" : "2020-06-17T14:54:12.899Z",
        "createdAt" : "2020-06-17T14:54:12.899Z"
      },
      "publicUrl" : "https:\/\/staging.sketch.cloud\/s\/615164b7-1b51-49e4-be08-33aedabc8150",
      "shortId" : "615164b7-1b51-49e4-be08-33aedabc8150",
      "isPrivate" : 1,
      "currentVersion" : {
        "id" : "b9f12673-44eb-4fb9-bcfe-9336cec237df",
        "updatedAt" : "2020-06-17T14:54:49.378Z",
        "createdAt" : "2020-06-17T14:54:49.378Z"
      },
      "createdAt" : "2020-06-17T14:54:49.378Z",
      "name" : "Sketch Document",
      "libraryEnabled" : 0
    }
  },

\cc @jedrichards - please ping me if you have any questions.

Add MSPatchInfo to the file spec

The contents go like this:

MSPatchInfo {
  lastIntegratedPatchID: string
  statusInfo: MSCollaborationStatusInfo {
    status: string
    uploadedPatches: array of strings
    isConsolidating: bool
    shareID: string
    baseVersionID: string
    versionID: string
  }
  MSPatchContainer {
    localPatches: an array of MSPatch objects
  }

This object contains the information we need to track the state of collaboration documents and should really only be present in documents within the hidden cloud folder on users machines.

The contents of MSPatch objects are described in Notion

@jedrichards hope that's useful. I'm not sure how all this stuff works, so ping me if you need more info.

Missing do_objectID in ColorAsset/GradientAsset schemas

Looks like ColorAsset/GradientAsset schemas are missing do_objectID properties however they're generated for file format 121

Release schemas for document version 120

Update file format readme versioning info

Rework integration testing in sketch-file-format to account for new Sketch schema smoke test

At the moment sketch-file-format tests against a specific version of Sketch, but with the new testing strat the file format no longer needs knowledge of a specific Sketch version.

It might be useful however, to retain some sort of script in the file format repo that enables testing the schemas against an arbitrary Sketch file, for use during development.

Font Embedding Beta Update

We have a change-since-initial-beta situation where we’re dropping two font-embedding related model attributes from DocumentData (agreedToFontEmbedding and autoEmbedFonts ). I’ve made the necessary changes to the app (including adding a new document version entry for 131 though that may end up supplanted by someone else's change :-) ) and am requesting assistance updating the format. This is in relation to Sketch-hq/Sketch#30452.

Release schemas for document version 119

Incorrect usage of ExclusiveMinimum

Hi,

Looks like gradient schema is incorrect according to JSON Schema 7 in the following lines: https://github.com/sketch-hq/sketch-file-format/blob/d61c8a2e3d81ecfe0890d39e1a48f2add77801cb/schema/objects/gradient.schema.yaml#L10-L13

exclusiveMinimum should be defined on numeric instances according to spec. Is it a bug or
some extension like #28?

File Format: Address all questions/comments in the code marked with `TODO`

Override Name attribute in Override Value

Greetings,
I am currently working with some sketch files and I am having some trouble understanding the override-name attribute. I have a symbol instance that contains a couple of overridable values, however, it contains a value that I have not seen in the override-name.schema.yaml.
The following image contains the value that I am getting for the overrideName for a specific OverrideValue.

However, in the override-name.schema.yaml there is no specification for _layerStyle. Furthermore, would that specification describe multiple UUIDs (as seen in the screenshot)? 🤔
Thanks in advance for the help! 😄

Override Name

title: Override Name
description: Defines the valid string patterns for an override name
oneOf:
  - type: string
    pattern: '^[0-9A-F]{8}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[0-9A-F]{12}_stringValue$'
  - type: string
    pattern: '^[0-9A-F]{8}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[0-9A-F]{12}_symbolID$'
  - type: string
    pattern: '^[0-9A-F]{8}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[0-9A-F]{4}\-[0-9A-F]{12}_image$'

Override Value

title: Override Value
description: Defines an individual symbol override
type: object
optional:
  - do_objectID
properties:
  _class: { const: overrideValue }
  do_objectID: { $ref: ../utils/uuid.schema.yaml }
  overrideName: { $ref: ./override-name.schema.yaml }
  value:
    oneOf:
      - { type: string }
      - { $ref: ../utils/uuid.schema.yaml }
      - { $ref: ./file-ref.schema.yaml }
      - { $ref: ./data-ref.schema.yaml }

FillType Pattern is wrong

https://github.com/sketch-hq/sketch-file-format/blob/master/schema/enums/fill-type.schema.yaml#L7 is supposed to be 4 and not 2

ColorControls property units

Looks like ColorControls schema defines invalid 0-1 float units for its properties: https://github.com/sketch-hq/sketch-file-format/blob/d61c8a2e3d81ecfe0890d39e1a48f2add77801cb/schema/objects/color-controls.schema.yaml#L7-L10

However the values vary from -100% to 100%:

Add Color Variables to sketch-file-format (model version 130)

Summary

Color Variables introduce two new structures to the document data file format and add one optional property to MSColor instances.

An overview of the implementation be found in Notion, and the corresponding branch is at Sketch-hq/Sketch #28967

Colors

MSColor instances now include an optional property swatchID of type String, that links to the objectID of a MSSwatch instance.

Shared Swatches

Shared swatches are saved in a MSSharedObjectContainer subclass named MSSwatchContainer. The objects property contains an array of MSSwatch instances.

MSSwatch instances are subclasses of MSSharedObject where the value property is an instance of MSColor.

{
…
  "sharedSwatches": {
    "_class": "swatchContainer",
    "objects": [
      {
        "_class": "swatch",
        "do_objectID": "6450AB9F-65F7-483A-80E9-1BE8A7727562",
        "name": "Ruby",
        "value": {
          "_class": "color",
          "alpha": 1,
          "blue": 0.1254901960784314,
          "green": 0.1254901960784314,
          "red": 0.8784313725490196
        }
      },
    ]
  }
…
}

Foreign Swatches

{
…
  "foreignSwatches": [
    {
      "_class": "MSImmutableForeignSwatch",
      "do_objectID": "F059ECC4-ADA8-4E67-81C6-0DC05F4235AB",
      "libraryID": "4531B31C-EC9F-4CBA-831B-BE972592957C",
      "sourceLibraryName": "#29388-library",
      "symbolPrivate": false,
      "remoteSwatchID": "7C0DBB6D-C198-40A6-AE58-DC4463302C72",
      "localSwatch": {
        "_class": "swatch",
        "do_objectID": "92C941E4-5913-4D34-90C8-988CA6E65C9D",
        "name": "Green",
        "value": {
          "_class": "color",
          "alpha": 1,
          "blue": 0.1999174529568891,
          "green": 0.8024230072463768,
          "red": 0.1069846376086679
        }
      }
    }
  ],
…
}

Color is hard to parse. Perhaps expose hexvalue?

I'd like to do a color comparison but its pretty complicated to do with color being broken up into RGBA values.

artboard.backgroundColor != "#ffffff"

Right now I have to check the values of each component which is no fun.

Backport recent fixes to 1.*.* and 2.*.*

Recent correctness fixes have only been released in version series 3.*.* but they are also applicable to historical versions too. They should be backported and released too.

Incorrect class name for inner shadows

While the spec states that inner-shadow should have a _class of shadow, files created with Sketch 59.1 have a class of innerShadow.

Example inner shadow from Sketch 59.1

{
  "_class": "innerShadow",
  "isEnabled": true,
  "blurRadius": 3,
  "offsetX": 0,
  "offsetY": 1,
  "spread": 0,
  "color": {
    "_class": "color",
    "alpha": 0.5,
    "blue": 0,
    "green": 0,
    "red": 0
  },
  "contextSettings": {
    "_class": "graphicsContextSettings",
    "blendMode": 0,
    "opacity": 1
  }
}

`paragraphStyle` additional property `allowsDefaultTighteningForTruncation` is not allowed

The paragraph-style schema appears to be missing the allowsDefaultTighteningForTruncation property, triggering the following error in documents created with Sketch 59.X or greater

layerTextStyles.objects.0.value.textStyle.encodedAttributes.paragraphStyle:
Additional property allowsDefaultTighteningForTruncation is not allowed

Port repo scripts to TypeScript

Smart layout axis not work

The inferred-layout-axis.schema.yaml schema is not working as what I saw after unzipped a sketch file where integer 0 represents Horizontal and 1 represents Vertical.

Where

enum:
  - 0
  - 1
  - 2
enumDescriptions:
  - None
  - Horizontal
  - Vertical

Should be

enum:
  - 0
  - 1
enumDescriptions:
  - Horizontal
  - Vertical

Fill in all schema `description` fields and tidy up `TODOs`

Text behaviour has a third enum value 2 (fixed width and height)

No value for key `frame` in `page`

The specs for page specifies inheritance from abstract-layer which has a required frame property
I can't find the key frame in a page saved by Sketch v61.2

Wrong title in strikethrough-style.schema

The title in schema/enums/strikethrough-style.schema.yaml is wrong

Please add strikethroughStyle

v3

/**
 * Defines text style
 */
export declare type TextStyle = {
    _class: 'textStyle';
    verticalAlignment: TextVerticalAlignment;
    encodedAttributes: {
        paragraphStyle?: ParagraphStyle;
        MSAttributedStringTextTransformAttribute?: TextTransform;
        underlineStyle?: UnderlineStyle;
        strikethroughStyle?: {};
        kerning?: number;
        MSAttributedStringFontAttribute: FontDescriptor;
        textStyleVerticalAlignmentKey?: TextVerticalAlignment;
        MSAttributedStringColorAttribute?: Color;
    };
};

please add strikethroughStyle enum

PointString pattern

HI,

Looks like PointString pattern ^{-?\d+(.\d+)?, -?\d+(.\d+)?}$ does not allow exponential notation of numbers: "{3.061616997868383e-17, 0.5}" fails, however this string was generated for version: 121 of sketch file format.

Bump document model version to 133

Migrating document color presets to color variables requires a new document model migration and thus a new version. See issue #29375.

This version is declared as MSDocumentCopyColorAssetsToSwatches = 133 in MSDocumentVersion.h.

Abstract layer schema missing optional `sharedStyleID`

The abstract layer schema is missing the sharedStyleID property that's used to link a layer to a given shared style (shared styles being stored at the document level).

The sharedStyleID is optional, and should be a UUID.

Add workspace spec to the file format

Unnecessary sharedStyleID in shape-group.schema

In shape-group.schema, the sharedStyleID property is already included in abstract-group.schema

MVP JSON Schema file format

Create a hand-crafted MVPish v1 of the file format in this repo, using plain JSON Schema. This should act as a conversation starter, and a concrete example that other parties can use to explore the problem space.

@nthState's automatically generated schema can act as a starting point.

We should explore ways of managing a large JSON Schema as part of this process. For example,

• Are there ways of splitting it up into smaller more manageable files, and then combining into the final schema later?
• What tooling is available to lint a schema
• Maybe use prettier to autoformat the JSON consistently
• Etc

Address recent issues raised by raveclassic in sketch-file-format repo

Add support for model version 125

This is a model-only bump to support Sketch 64. There will be no other changes to the schema for this version.

Incorrect property existence validation for objects

Hi,

Looks like current notation for property existence in objects is inconsistent with JSON Schema 7.

Document schema correctly defines pages property as required: https://github.com/sketch-hq/sketch-file-format/blob/d61c8a2e3d81ecfe0890d39e1a48f2add77801cb/schema/document.schema.yaml#L6-L7

However AbstractDocument schema uses a custom field optional: https://github.com/sketch-hq/sketch-file-format/blob/d61c8a2e3d81ecfe0890d39e1a48f2add77801cb/schema/abstract-document.schema.yaml#L2-L3 which is not supported directly by JSON Schema 7. Even more it's not use in the typescript codegen at all:
https://github.com/sketch-hq/sketch-file-format-ts/blob/00fe473753775917324f2f635e6f4bf9af700dc6/scripts/utils.ts#L48-L60

Is such behaviour intentional or am I missing something?

Incorporate new reference files repo

Add `missingLibraryFontAcknowledged` to `MSForeignSymbol` and `MSForeignTextStyle`

MSForeignTextStyle and MSForeignSymbol have both gained a Boolean property missingLibraryFontAcknowledged. This change is part of #29224 (targeting develop) and was added in this commit.

Release schemas for document version 121

Release schema versions for Sketch 58, 59

This should be done once we have release/publish mechanism.

Note: the file format schemas seem more closely associated with the document version that the Sketch version.

Optional alignment in ParagraphStyle

Looks like alignment property in ParagraphStyle schema should also be added to optional list: https://github.com/sketch-hq/sketch-file-format/blob/d61c8a2e3d81ecfe0890d39e1a48f2add77801cb/schema/objects/paragraph-style.schema.yaml#L4-L9

It's missing in some of the TextStyles generated for 121 file format.

Standalone Sketch File Format v1.0.0 ☂️

Progress

• Spike feasibility of autogenerated schema (https://github.com/BohemianCoding/Sketch/issues/24735#issuecomment-525272025)
• MVP JSON Schema File Format (#2)
• Release v1 https://www.npmjs.com/package/@sketch-hq/sketch-file-format 🎉

---

This issues discusses how a v1.0.0 of a standalone Sketch File Format project might work and look like.

A standalone Sketch File Format would have the following important properties:

• Serve as a single source of truth for the format
• Allow disparate teams within Sketch B.V. to organise around a shared specification
• Can be transformed into useful formats, including —
  • Documentation
  • TypeScript types (typesafe documents or document fragments in JS/TS projects)
  • JSON Schema (validate documents)
  • Swift models (for Sketch.app)
  • GraphQL schemas (for backend)
  • Anything else
• Defines the interchange format for an ecosystem of internal products and 3rd party integrations
• Treated as a product in its own right
• Public, published and working in the open
• Independently versioned (i.e. deliberately ignorant of any Sketch.app version), with a predictable, measured release cadence
• No other Sketch B.V. product dependencies (root of the tree)
• Easily maintainable with modern tooling and CI to ensure internal correctness

✂️ Hand-crafted vs auto-generation

We've already tried auto-generating a file format specification from the Sketch code base. It was an incredibly useful learning exercise, but has critical drawbacks:

• It doesn't solve the core organisational issue, which is that the relationship between Sketch and the file format is inverted (Sketch should consume and conform to a public file format, rather than potentially mutating it without warning every release)
• It proved difficult to generate a 100% correct and sufficiently rich format automatically from the Sketch code base

Therefore, maintaining a hand-crafted file format as a product seems like the best next step to explore.

👑 Source format

The source format defines the technology we use to maintain the file format source, but isn't prescriptive about the formats that can be derived from it. Indeed, the objective here is to pick a source format that enables fluent transformation to as many other output formats as possible.

There are two main candidates:

JSON Schema

JSON Schema itself is an obvious choice. It has rich semantics which allows it to fully describe complex models, and has been widely adopted with tooling support in many languages and platforms. The JSON Schema specification process has been adopted by the IETF and sees active development.

A potential downside is that JSON Schema is just raw JSON, and as such the experience of maintaining a complex/large JSON Schema document by hand could have some challenges. We could look into JSON Schema $refs to split the schema into manageable chunks.

TypeScript

Defining the file format in code using TypeScript would bring a few benefits:

• TypeScript compiler could check correctness during compilation
• Use tooling and IDE integrations to enhance the maintenance experience (formatting, linting)
• Code separation into modules would enable the schema to be broken into smaller logical chunks more easily than manually managing JSON files
• TypeScript is similar to Swift, so editing a schema in TypeScript could potentially be easier for native developers, rather than having to learn the JSON Schema spec (we'd have to gather opinions here)
• No need to implement a TypeScript output format separately (which we're sure to need immediately)

The TypeScript AST is well documented, and would serve as a robust data format to enable transformation to other formats. We could write our own TypeScript to JSON Schema transformer for example, or use existing tools like typescript-json-schema. Tooling for TypeScript to Swift transformation exists in the wild too, e.g. quicktype.

We'd have to ensure that we can sufficiently describe the existing file format model in TypeScript before choosing this route, since the range of type information and their interrelationships it can describe is more limited than JSON Schema (this could turn out to be a good thing - since its also simpler).

Swift

There could be a compelling case made for using Swift for the source format, since the most important consumer of the file format will be Sketch.app. The feasibility of this would hinge around the suitability of using the Swift AST as a source data format for onward transformation into JSON Schema, TypeScript and others.

Is the Swift AST sufficiently mature? What does the existing tooling and documentation around it look like? Does it limit the potential for 3rd parties to easily write their own transformation scripts? This would need investigating by a native developer.

💁‍ Output formats

We would expect the community to use the file format to generate diverse formats that work with their specific workflows and toolchains, therefore we don't need to be exhaustive in the output formats we initially implement.

A v1.0.0 of the file format could simply include the source format itself, whatever it is. Additional output formats that we're certain to need internally (e.g. TypeScript, or Swift) could be looked at as a follow on task.

Rather than pollute the core file format repo with generator code for other formats, these could be housed in separate repos (GitHub Actions could eventually be used to trigger builds when a new file format is released).

🌳 Structure

The following structural elements and types should be describable by the file format,

• A definition for the root document type and every class/object type that can appear as a child in the document
• Limit object properties to allowable types (e.g. layers array can only contain elements of union type Layer, for example)
• Encode whether properties are optional
• Descriptive enums (e.g. all possible values of _class, or colorProfile properties etc)
• Handy union types not explicitly appearing in the Sketch.app model, but useful for developers parsing/generating documents in the wild (e.g. a type representing a union of all layer types)
• If practical reproduce some aspects of the Sketch.app model class hierarchy (main benefit here would be to reduce duplication in the file format source, rather than attempting to mirror the internals of the native app, e.g. define an abstract type for layers, which could share common properties amongst all concrete types that behave as a layer)

⚠️ Change management

There's no beating around the bush - the main driver of change in the file format would be Sketch.app.

Rather than deny that reality the objective is to formalise the relationship, ensure changes to the file format are explicit and deliberate and to implement a process where new versions of the file format are released ahead of time, or concurrently with the version of Sketch.app that supports it.

The change management process could look something like,

• File format versioned independantly of Sketch.app, following semver
• Sketch documents written in the file format no longer declare their compatibility with Sketch.app, rather they declare the version of the file format they conform to
• Limit the pace at which new features are added to the file format - since this churns the format and burdens the entire ecosystem. Suggest annually, or less
• This would mean stacking up Sketch features that require new features or changes to the file format and releasing them together in a dedicated Sketch release that advanced the file format by major version. Example scenario,
  1. Discuss and define the set of changes needed to support new features for a special release of Sketch that introduces new features to the file format
  2. Pre-release v2.0.0-rc1 of the file format supporting said features early on
  3. Iterate through release candidates while preparing for main Sketch release, e.g. v2.0.0-rc2, v2.0.0-rc3, etc.
  4. Then headline the relevant Sketch release "Now outputs in v2 of the file format" and simultaneously release the final v2.0.0 version of the file format
  5. Interested 3rd parties could have followed along with the release candidates too, so upgrading to the final release would hopefully be a painless version bump. For those who didn't have their eye on the ball the release would be documented with a changelog and ready to go with freshly published schemas and types, so upgrading to a new version of the file format and fixing any compile errors hopefully also a much improved experience than the current arrangement allows
• Bug fixes could be released as patch versions as often as needed to fix inconsistencies between the file format and versions of Sketch released in the wild

🤖 Hosting

The primary hosting mechanism could be tagged GitHub releases on the file format repo.

Thereafter, derived output formats could be hosted in whatever fashion makes most sense for their community/ecosystem. E.g. npm for TypeScript types etc.

🤔 For further discussion

How will the file format fit into the native Sketch.app workflow?

The responsibility here will lie with the Sketch dev team to figure out an approach for ensuring the versions of Sketch they release output documents that validate against the file format.

It's important to note that, technically speaking, a v1.0.0 of a standalone file format should precisely match the schema of files outputted by the current version of Sketch. So initially the changes to the Sketch development process might centre more around testing, than large internal changes.

We need input from native developers but a few strategies could include,

• Generate Swift code from the file format
• Unit tests
• Integration tests validating documents against a JSON Schema

Add object IDs to container objects

Object IDs on shared containers are now persistent. Specifically these are the layerStyles, layerTestStyles and sharedSwatches containers on the document, plus layerSymbols for legacy support.

All of these included only _class and objects properties and would require the addition of:

do_objectID: { $ref: ./utils/uuid.schema.yaml }

• shared-style-container.schema.yaml
• shared-text-style-container.schema.yaml
• swatch-container.schema.yaml
• symbol-container.schema.yaml for legacy support

Add feature preview variant

A additional variant FEATURE_PREVIEW has been introduced.

File format schema to TypeScript transform script

• Create a new repo to house the transform script
• File format schemas will be imported as a dependency, and used as a data source
• Use TypeScript compiler API and AST to do the actual transformation from json schema to TS
• Repo will also publish an npm module containing the types

Finalise guide for working on the file format

Notes from Notion:

• Goal is to communicate clearly that it's not us dictating changes on the file format but this is the place for everyone making changes on the file persistence layer to define and document them.
• Is it possible to provide a sample scenarios for (non-)breaking change?
• Make clear this is going first and foremost about helping robust products across our organisation

Add changesets to sketch-file-format

• Changesets CLI
• Documentation
• Bot
• Action

SymbolMaster.overrideProperties is an empty array

From sketch-hq/sketch-file-format-ts#8