prinzhorn / scrollmeister

Open-source JavaScript framework to declaratively build scrolling experiences

What is Scrollmeister?

Scrollmeister is an open-source JavaScript framework to declaratively build scrolling experiences. Using custom elements (<scroll-meister>, <element-meister> and <shadow-meister>) you can create complex interactive scrolling pages without a single line of code. All you need is an HTML editor and you're good to go, you can even render the pages on the server. Scrollmeister comes with it's own layout engine called Guides Layout. It was built from the ground up with scrolling interactions in mind. This makes it ridiculously performant (on both desktop and touch devices).

What is it not?

Scrollmeister solves a particular problem really well, but it is important to understand when not to use Scrollmeister.

• Scrollmeister is not a drop-in library that you can use with your existing website. To achieve its flexibility and performance Scrollmeister needs full control over the elements of a web page. However, Scrollmeister does not lock you into its system in the sense that you can do whatever you want inside of a <element-meister> Element.
• If you're creating a "regular" public facing website then Scrollmeister might not be the right choice. If all you need is a parallax header image then just use a jQuery plugin. Scrollmeister is meant for complex scrolling interactions spanning multiple elements, e.g. stories or presentations.

Docs

Check out the website and demos for now, this is highly WIP.

scrollmeister's Issues

Error when schema contains reserved words

E.g. definining an attach prop would break the behavior

I think we can just collect all the properties at runtime

https://docusaurus.io/

Allow spacing around scroll-meister

I personally need this for the sidebar inside the editor.

Either through a spacing prop or if we end up using native scrolling (I'm looking at you, iOS) we can track the size of scroll-meister and add padding using the parent.

Web workers

E.g. preload images or dolayout (might not work as we work on the references)
https://github.com/RevillWeb/img-2

<content-meister>

Wrapping contents in the layout behavior is fragile and makes it hard for other behavior to access the inner element. Let's always wrap the contents (React would already provide them wrapped). See comment in LayoutBehavior

RatioType needs to accept floats as well

Not just width/height format but just a plain parseFloat

Merge multiple CSS transforms / sync styles across behaviors

We need a way to set style across all behaviors in sync. E.g. the layout and (transition + ) translate behavior both need to update transforms. We shouldn't touch style directly inside a behavior ever, but have something merge them. E.g. the layout behavior uses translate() to position the element and the translate behavior translates it vertically as you scroll (appear.vertical in legacy code). They both don't need to know about each other. This top-level style-applier could also notify you in console if two behaviors update the same style, e.g. color and have a conflict because they overwrite each other. So, could this maybe be handled by, say, a css behavior? It could be implicitly added to every single component. Can then be accessed using this.el.css. It works transparently using setters, so no need to pass objects around with functions. this.el.css.color = 'red' will just work. Basically it can be used like regular this.el.style, but it has magic powers like adding prefixes and merging transforms. It would reset it's internal state after each render, so that it can keep track if e.g. this.css.color has been set multiple times. Setting this.el.css.transform multiple times will just merge them. this.el.css.opacity could just multiply all of them. This would result in some GSAP-level flexibility.

In order for the css behavior to do this in sync, it needs to coordinate a few things.

1. It needs to be the first one to be notified of guidelayout::layout and scroll events
2. It then iterates over all behaviors giving them a chance to set styles (iterating over all of them could simply mean trigger an even that they can listen for)
3. After all styles have been collected, they can be applied to the element and the internal buffer is reset

disconnectedCallback and parentNode

Currently we detach every single behavior when an element is disconnected. However, the dependency check uses el.parentNode. We should have a special mode for detaching/cleaning up everything. This is different from regular batch detaching.

fullscreen behavior

Like a lightbox. The layout behavior needs to be fullscreen-aware. In fact it has the fullscreen state. However, a fullscreen behavior would render the close button, a different cursor and would call enter/exit on the layout behavior.

The media behavior needs to react to it as well, because usually we don't want fit: cover in fullscreen. By adding a transition for width/height/transform we get some nice reveal effect when triggering fullscreen.

Transition support needs to be added to the the StyleMerger as well.

events vs declarative

see also #9

let's unify the behavior API a bit. It should be transparent to a behavior if it was added lazy. The render method should just be called with the props of the dependency behaviors once they update.

Instead of pushing data to the behaviors (like react does with props) the behaviors will pull them in using events (because our behaviors are completely decoupled). However, we do it declaratively instead of subscribing to events manually.

We already know when props have changed (change event), now we need to unify the state (e.g. interpolate behavior does not use state but stores stuff on the instance). Btw. the change event already fires for state changes as well. And we can update state in update before the event triggers.

Update behavior properties without setAttribute

Properties can currently only be updated using setAttribute. This works well for quickly editing them in dev tools or when react re-renders the element. However, when programmatically setting them it could be more convenient to access individual properties directly. Reading works already by using element.behavior.props.key to access a parsed key and getAttribute can be used to get the whole thing. What's missing is a way to set an individual key, e.g. layout.guides without affecting others such as layout.spacing (setAttribute would always require all props as a string). Setting it should merge it back into the attribute.

Problem: this will trigger attributeChangedCallback and parse the whole thing again. For some prop types such as LayoutDependencyType this might have performance implications. Instead we could just update the internal representation like a-frame does. In debug mode we can then force-sync the attribute, which will cause a full re-parse. a-frame uses a flushToDOM method.

So first we need to add a getter/setter for every key defined in the schema. This is simple and just calls the serializer/parser for each defined type and updates props. The tricky part is not everything can trivially be serialized back, e.g. LayoutDependencyType. We might have to store the original value (e.g. selector or 'skip 2') attached to the array.

signals / slots / classes /css behavior

Add/remove classes using signals/slots. Does not depend on any other behavior, but is often used with the layout behavior. E.g. for layout:center:enter events.

E.g.

<el-meister layout='...' css signals=''></el-meister>

So...how do we want the syntax for the signals behavior to actually look like? We want it to be able to send signals to other elements as well (identified by #id). So we need to specify a target (e.g. #nav), when to send the signal (e.g. layout:viewport:enter), the signal to send (or is there just a single slot?) and the data to transport (e.g. which class to add/remove like +is-open or what to do like play on a video).

Chrome Android video scrubbing laggy

Basically it fires the seeked event and seeking is false, but it did not render the video yet.

We need to force the video to be actually drawn. It currently only does that once I stop scrolling (which is faked, so it's not an issue with scroll/touchmove events blocking it). I guess Chrome will only upload the video to the GPU (we translate all elements) once it does not move anymore.

height: inherit for followers

Original scrollmeister had this. What it basically does is set height to leaderHeight.

I need this for the editor so I can have an outline around a group of elements (focused) by using consume: 3 and pinning.

It's useful for other pinned stuff that you want to have the same height as the leader(s)

Can someone please tell me why fake scrolling is so much smoother on IOS?

I've compiled a bundle with native scrolling (as described in #26).

Demos with fake touch scrolling: https://www.scrollmeister.com/demos/
Demos with native touch scrolling: http://playground.scrollmeister.com.s3-website.eu-central-1.amazonaws.com/demos/

Let's take the "Parallax ALL the things" demos. I open it and slowly but constantly scroll down until the end using my iPad 2 mini. The version with fake scrolling is buttery smooth. The native scrolling version has at least two huge janks at some point, pausing for maybe half a second.

Can someone with a Mac please fire up remote debugging and ask the performance monitor what's causing it?

https://www.scrollmeister.com/demos/parallax-all-the-things.html

vs

http://playground.scrollmeister.com.s3-website.eu-central-1.amazonaws.com/demos/parallax-all-the-things.html

If you have time feel free to compare the other demos as well. The jank can be observed in most of them.

Error handling

Instead of just throwing, render the error on the page inside the el-meister. Users of Scrollmeister might not even know what the JavaScript console is. They just want to reload the page and see if something went wrong, e.g. if they forgot a dependency.

Timing issue with appending new items

This right here

scrollmeister/src/behaviors/GuidesLayoutBehavior.js

Line 141 in 4568a84

let nodes = Array.prototype.slice.call(this.el.querySelectorAll('[layout]')).map(el => el.layout);

will map all components, even if layout is not attached yet. Need to filter el.behaviors.hasOwnblabla(thing)

Add the Version as Scrollmeister.version

And also add a header comment to the dist files

viewport-events behavior

not just enter/leave but you can define anchors exactly like the interpolate behavior

or maybe the interpolate behavior could optionally emit events at the right moment?

media behavior

3d transforms and backface-visibility: hidden; (can we use the magic style setter on the media element or wrapper?)

Firefox performance

Something must have changed with the recent releases. The measured scroll performance is great, but I perceive it as janky. It's not an issue with rendering too slow or anything. Not sure what is going on yet.

importing regl crashes ie9

It doesn't have proper error handling and expects to only be required in webgl environments

gl-effect behavior

Works on <img> and <video>. We already have most of the logic implemented. I think what we'll do is let people write the fragment shader and import it by name into the behavior. And also have some in the core (it's just like 10 lines of code).

It will depend on the interpolate behavior.

Use https://github.com/regl-project/regl instead of doing the thing by hand (which we already have working, but new toys are always cool)

<script type="text/glsl" id="pixelate">
precision highp float;

uniform float progress;
uniform vec2 resolution;
uniform sampler2D imageTexture;

varying vec2 uv;

void main() {
	//This way we see the pixels much earlier. We skip the boring part were they're just like a little bigger than original.
	float easedProgress = pow(progress, 0.1);

	//The grid size in whole squares, e.g. 15x7.
	vec2 grid = floor(resolution * (1.0 - easedProgress) + 0.5);

	//The size of a single pixelated block, e.g. 0.1x0.07 (10%x7% of the texture size).
	vec2 pixelSize = 1.0 / grid;

	//We use the color at the center of the block for all pixels inside of it.
	vec2 transformedUV = (floor(uv / pixelSize) / grid) + (pixelSize / 2.0);

	gl_FragColor = texture2D(imageTexture, transformedUV);
}
</script>
<!--
    This will apply the pixelate shader to the <img>. Progress is mapped to the alpha interpolate value (so your shader can accept any number actually.).
-->
<el-meister interpolate="alpha: ..." gl-effect="shader: pixelate; param: alpha;">
    <img src="...">
</el-meister>

<element-meister> -> <section-meister>

"element" doesn't say much. content-meister and shadow-meister are both semantic names.

What element-meister really is is a section I guess.

Media queries / responsive behaviors

In theory this can be done in userland. A responsive behavior could simply overwrite other attributes (behaviors) depending on different conditions. E.g.

<el-meister
    layout="guides: center right"
    responsive="mobile: layout: 'guides: left right';"
>

But nesting attributes as strings inside attributes looks hell and no. Just no.

Instead we can make this part of the very core, since it is an essential feature anyway. A-frame allows the same component (we call it behavior) to be added multiple times by specifying an id using two underscores. E.g. <a-entitiy thing__foo="" thing__bar="">. Inspired by this idea I think we could use a similar naming convention to make behaviors' properties responsive.

The following example defines an element which uses the left and right guides (basically full width) as a default. It also defines a spacing of 10vh. Now if the desktop condition is met, the layout.guides property is overwritten with left center, making the element span the left half. Spacing is untouched.

<el-meister
    layout="guides: left right; spacing: 10vh 10vh;"
    layout_desktop="guides: left center;"
>

These type of conditions could be anything, not just the browser width. A condition is basically a named function, in this case desktop. But you might as well define a touch condition or a ie9 condition. And of course they don't just apply to the layout behavior, but literally every behavior. And since many behaviors add functionality and not just styling, this allows to declaratively change the functionality without writing any code. For the behavior itself this is completely transparent. It doesn't know about conditions. It just receives new props.

In the above example the layout attribute is actually just a shorthand for layout_default. The default condition is always true.

TODO: the conditions need to have an order/specifity. E.g. the default condition has the lowest specifity and will be overwritten be any other matching condition.

TODO: the observedAttributes getter needs to be aware of all possible conditions to add them to the list. To allow custom conditions in userland we face the same issue as with custom behaviors. They need to be specified before the custom element is registered or otherwise the observedAttributes getter won't know about them.

`default` and conditions

Right now defaults are applied to conditional props as well, bummer.

scrollbar behavior

for scrollMode touch

React etc. virtual DOM mismatch

E.g. React renders something like

<el-meister gl-effect>
    <img src={src}>
</el-meister>

then the gl-effect behavior will later append a canvas to the wrapper

<el-meister gl-effect>
    <img src="...">
    <canvas></canvas>
</el-meister>

now if React re-renders, funny things might happen. Note: that the lazyload behavior is affected as well. It will remove data-src and update src. But React might later overwrite them again. React could return false from shouldComponentUpdate, but if the src actually changes, it is lost. Double note: the lazyload behavior is irrelevant for the WYSIWYG though.

The React docs completely ignore the issue and just say you can render custom elements like every other element (https://reactjs.org/docs/web-components.html). But they're completely missing the point that custom elements can and will manipulate their child DOM. Shadow DOM would solve this by not actually changing the DOM visible to React. But shadow DOM cannot be polyfilled and is not an option for us.

There are articles out there trying to solve the issue like this https://www.sitepen.com/blog/2017/08/08/wrapping-web-components-with-react/ . But manually syncing React and the DOM sounds like a bad idea and is error prone. Even more important it is not transparent and requires the React user to know about all of the internals of the custom element, which is crazy.

I was trying to figure out if React has some built in support to mark certain elements as "do not touch" facebook/react#6622 . But so far they are only relevant to server side rendering and not to our case https://reactjs.org/docs/dom-elements.html#suppresshydrationwarning Also this would be react specific and we need a solution for all the libraries.

Race condition / timing issue with custom behaviors

See also #1, the same applies to custom conditions.

Currently we call Scrollmeister.defineBehavior before customElements.define. This is mandatory because when defining a custom element it will immediately call the static observedAttributes getter once. This implies that we need to know about all behaviors before the element is defined, inside the <head>. If a user includes the scrollmeister.js and then calls defineBehavior it will not work, as the attribute will not be observed. The CustomElementRegistry does not offer a way to redefine or remove a registered element, or else we could do that once a new behavior is added.

I see two solutions. Either we force the user to define behaviors before including the scrollmeister.js script (using some global config object) or we schedule the customElements.define call using raf. The first option is asking for tons of trouble/issues/stackoverflow posts because it is error prone (does it? defining new behavior/conditions is expert stuff that normal users don't need at all. But still, it feels weird to include them in this order). The second solution might have other implications. The browser will first treat the custom element as unknown and then upgrade them a tick later. Need to experiment with this.

clip: true for non-followers

a-frame behavior

Which renders the scene from a <script> tag (TemplateType) only when it is fullscreen. Otherwise it shows the fallback content.

Experiment with native scrolling on touch

If we scroll inside <scroll-meister> we won't have any issues with the address bar triggering layouts. We would get 100% native scrolling. Need to experiment with that as the only question is whether it performs well (since we're not actually scrolling anything the only question is if we can query the scroll offset at 60 fps)

Maybe we don't even need #25 if we can use native scrolling everywhere.

Dev tools / inspector extension

Once #5 is implemented, we should be able to easily make all behaviors with all properties editable using a custom dev tools extensions (like react and redux extensions).

It should be very similar to the styles-tab. Individual properties can be disabled using a checkbox (making them fall back to the default specified in the schema). The values can easily be edited with autocomplete for enum stuff like followerMode. Numeric values like spacing or skip n should be editable using keyup/down as well. Hovering dependencies will highlight the element(s).

Expose scroll velocity to gl-effect schaders

Need to make sure though that the behavior goes back to 0 when you stop

DOM manip perf

From #32

3D model behavior (not in core or extras, just a demo/playground)

https://twitter.com/keithclarkcouk/status/974609122720141312

Progressive enhancement af

<el-meister model="src: some/3d/model.obj;">
    <img src="fallback.jpg">
</el-meister>

Replaces the image with a canvas and does things. Especially the thing where it reacts to the viewport position (interpolate.progress ftw)

Put these things in a separate repo, like scrollmeister-lab. It contains behavior that are not part of core/extras, because they are way to specific.

MutationObserver

E.g. the gallery behavior needs to be updated when an image is added/removed or the width/height attribute of an image is changed. Other are likely affected as well. The gl-effects is currently polling the source (also to track playing video). fluid-text also does currently not react at all if the text changes.

Support is pretty good (IE 11 has it and Android 4.4). However, since we will abstract this anyway in Behavior.js we can likely easily fall back to Mutation Events without needing a complete polyfill.

Behavior.appendChild

similar to listen and style, it will append a node and automatically removes it when the behavior is detached

Can also automatically support sth. like data-scrollmeister-shadow, see #14

youtube behavior pause outside of viewport

similar to the autoplay behavior but only in one direction (pause when leaving the viewport)

Race conditions and lazily added behaviors

The interpolate behaviors requires the layout behavior to know where the element is positioned. The layout behavior requires the guidelayout behavior to actually to the layout. It also provides the interpolate behavior with the necessary scroll events. The transform behavior requires the interpolate behavior to provide the values.

Everything is nice if the behaviors are all added in one batch.

1. The guidelayout will schedule a layout using raf
2. In the mean time the layout behaviors are attached
3. In the next frame the guidelayout will query all children with layout and doLayout
4. The interpolate behavior listens for the guidelayout:layout event and creates the interpolators
5. The interpolate behavior will listen for guidelayout:scroll and also immediately interpolates the values, triggering a change event
6. The transform behavior listens for the interpolate:change event and transforms the element

All this works because the behaviors are synchronously attached in the correct order and because the guidelayout behavior waits for the next frame. Otherwise the children would not have layout yet (but they have already the attribute, just not the behavior).

The problem occurs when one of the behaviors is added lazy, e.g. by using setAttribute at some point. For example say we add the interpolate and transform behaviors at a later point. The interpolate behavior will listen for guidelayout:layout. But there won't be any guidelayout:layout in the near future, unless the window is resized or similar. One possible solution (I'm looking for more elegant and general purpose ones) is to use listenAndInvoke, which will trigger the listener immediately. However, this will then not work in the original case where all behaviors are added in one batch. So basically we'd need to use listenAndInvoke and then check if a layout has happended yet. E.g.

this.listenAndInvoke(`layout:render`, () => {
    //In case we invoke this callback before the very first layout had happened.
    if(this.el.layout.hasLayout) {
        doTheThing();
    }
});

I just realized that guidelayout:layout might be the wrong event for interpolate anyway. We don't care about global layout events, but only for the particular element. So layout:render then?

scroll-into-view behavior

It requires the ^scroll behavior and exposes a method to bring the element into the viewport (by default top/top aligned). The hash-navigation behavior already implements most of what is needed.

el.scrollIntoView.scroll();