Progress
This issues discusses how a v1.0.0 of a standalone Sketch File Format project might work and look like.
A standalone Sketch File Format would have the following important properties:
- Serve as a single source of truth for the format
- Allow disparate teams within Sketch B.V. to organise around a shared specification
- Can be transformed into useful formats, including —
- Documentation
- TypeScript types (typesafe documents or document fragments in JS/TS projects)
- JSON Schema (validate documents)
- Swift models (for Sketch.app)
- GraphQL schemas (for backend)
- Anything else
- Defines the interchange format for an ecosystem of internal products and 3rd party integrations
- Treated as a product in its own right
- Public, published and working in the open
- Independently versioned (i.e. deliberately ignorant of any Sketch.app version), with a predictable, measured release cadence
- No other Sketch B.V. product dependencies (root of the tree)
- Easily maintainable with modern tooling and CI to ensure internal correctness
✂️ Hand-crafted vs auto-generation
We've already tried auto-generating a file format specification from the Sketch code base. It was an incredibly useful learning exercise, but has critical drawbacks:
- It doesn't solve the core organisational issue, which is that the relationship between Sketch and the file format is inverted (Sketch should consume and conform to a public file format, rather than potentially mutating it without warning every release)
- It proved difficult to generate a 100% correct and sufficiently rich format automatically from the Sketch code base
Therefore, maintaining a hand-crafted file format as a product seems like the best next step to explore.
👑 Source format
The source format defines the technology we use to maintain the file format source, but isn't prescriptive about the formats that can be derived from it. Indeed, the objective here is to pick a source format that enables fluent transformation to as many other output formats as possible.
There are two main candidates:
JSON Schema
JSON Schema itself is an obvious choice. It has rich semantics which allows it to fully describe complex models, and has been widely adopted with tooling support in many languages and platforms. The JSON Schema specification process has been adopted by the IETF and sees active development.
A potential downside is that JSON Schema is just raw JSON, and as such the experience of maintaining a complex/large JSON Schema document by hand could have some challenges. We could look into JSON Schema $refs to split the schema into manageable chunks.
TypeScript
Defining the file format in code using TypeScript would bring a few benefits:
- TypeScript compiler could check correctness during compilation
- Use tooling and IDE integrations to enhance the maintenance experience (formatting, linting)
- Code separation into modules would enable the schema to be broken into smaller logical chunks more easily than manually managing JSON files
- TypeScript is similar to Swift, so editing a schema in TypeScript could potentially be easier for native developers, rather than having to learn the JSON Schema spec (we'd have to gather opinions here)
- No need to implement a TypeScript output format separately (which we're sure to need immediately)
The TypeScript AST is well documented, and would serve as a robust data format to enable transformation to other formats. We could write our own TypeScript to JSON Schema transformer for example, or use existing tools like typescript-json-schema. Tooling for TypeScript to Swift transformation exists in the wild too, e.g. quicktype.
We'd have to ensure that we can sufficiently describe the existing file format model in TypeScript before choosing this route, since the range of type information and their interrelationships it can describe is more limited than JSON Schema (this could turn out to be a good thing - since its also simpler).
Swift
There could be a compelling case made for using Swift for the source format, since the most important consumer of the file format will be Sketch.app. The feasibility of this would hinge around the suitability of using the Swift AST as a source data format for onward transformation into JSON Schema, TypeScript and others.
Is the Swift AST sufficiently mature? What does the existing tooling and documentation around it look like? Does it limit the potential for 3rd parties to easily write their own transformation scripts? This would need investigating by a native developer.
💁 Output formats
We would expect the community to use the file format to generate diverse formats that work with their specific workflows and toolchains, therefore we don't need to be exhaustive in the output formats we initially implement.
A v1.0.0 of the file format could simply include the source format itself, whatever it is. Additional output formats that we're certain to need internally (e.g. TypeScript, or Swift) could be looked at as a follow on task.
Rather than pollute the core file format repo with generator code for other formats, these could be housed in separate repos (GitHub Actions could eventually be used to trigger builds when a new file format is released).
🌳 Structure
The following structural elements and types should be describable by the file format,
- A definition for the root document type and every class/object type that can appear as a child in the document
- Limit object properties to allowable types (e.g.
layers
array can only contain elements of union type Layer
, for example)
- Encode whether properties are optional
- Descriptive enums (e.g. all possible values of
_class
, or colorProfile
properties etc)
- Handy union types not explicitly appearing in the Sketch.app model, but useful for developers parsing/generating documents in the wild (e.g. a type representing a union of all layer types)
- If practical reproduce some aspects of the Sketch.app model class hierarchy (main benefit here would be to reduce duplication in the file format source, rather than attempting to mirror the internals of the native app, e.g. define an abstract type for layers, which could share common properties amongst all concrete types that behave as a layer)
⚠️ Change management
There's no beating around the bush - the main driver of change in the file format would be Sketch.app.
Rather than deny that reality the objective is to formalise the relationship, ensure changes to the file format are explicit and deliberate and to implement a process where new versions of the file format are released ahead of time, or concurrently with the version of Sketch.app that supports it.
The change management process could look something like,
- File format versioned independantly of Sketch.app, following semver
- Sketch documents written in the file format no longer declare their compatibility with Sketch.app, rather they declare the version of the file format they conform to
- Limit the pace at which new features are added to the file format - since this churns the format and burdens the entire ecosystem. Suggest annually, or less
- This would mean stacking up Sketch features that require new features or changes to the file format and releasing them together in a dedicated Sketch release that advanced the file format by major version. Example scenario,
- Discuss and define the set of changes needed to support new features for a special release of Sketch that introduces new features to the file format
- Pre-release v2.0.0-rc1 of the file format supporting said features early on
- Iterate through release candidates while preparing for main Sketch release, e.g. v2.0.0-rc2, v2.0.0-rc3, etc.
- Then headline the relevant Sketch release "Now outputs in v2 of the file format" and simultaneously release the final v2.0.0 version of the file format
- Interested 3rd parties could have followed along with the release candidates too, so upgrading to the final release would hopefully be a painless version bump. For those who didn't have their eye on the ball the release would be documented with a changelog and ready to go with freshly published schemas and types, so upgrading to a new version of the file format and fixing any compile errors hopefully also a much improved experience than the current arrangement allows
- Bug fixes could be released as patch versions as often as needed to fix inconsistencies between the file format and versions of Sketch released in the wild
🤖 Hosting
The primary hosting mechanism could be tagged GitHub releases on the file format repo.
Thereafter, derived output formats could be hosted in whatever fashion makes most sense for their community/ecosystem. E.g. npm for TypeScript types etc.
🤔 For further discussion
How will the file format fit into the native Sketch.app workflow?
The responsibility here will lie with the Sketch dev team to figure out an approach for ensuring the versions of Sketch they release output documents that validate against the file format.
It's important to note that, technically speaking, a v1.0.0 of a standalone file format should precisely match the schema of files outputted by the current version of Sketch. So initially the changes to the Sketch development process might centre more around testing, than large internal changes.
We need input from native developers but a few strategies could include,
- Generate Swift code from the file format
- Unit tests
- Integration tests validating documents against a JSON Schema