responsiveimagescg / ri-usecases Goto Github PK

David Demaree wrote:

Just to offer some insight on this, given that I work on a product (Typekit) that is all about content negotiation, one reason I'm a strong proponent of doing this in the client is that server-side techniques can become very difficult — and very costly — at scale. I wrote a post for our blog last year about why we use JavaScript-based client-side content negotiation for web fonts instead of doing it on the server: http://blog.typekit.com/2011/08/11/better-web-font-loading-with-javascript/

The short version is: commercial CDNs either don't offer a way to do this on the server at scale, or will do it but only as a custom deployment that can be very expensive at virtually any scale. Google can do it server-side for their Web Fonts API because they're Google, and have their own, very robust CDN infrastructure. We initially did it client side because we were on a startup's budget, however using JavaScript we're also able to target browser edge cases (such as the IE rendering mode) that aren't reliably detectable with just a UA string.

I practically guarantee you that if responsive images don't have native content negotiation in the client, adding a JS-based solution is one of the first things I'd need in order to recommend using responsive images at scale.

We need to capture a use case and requirement(s) based on the above.

In 3.6 Matching media features and media types, current Use Case Requirement says:

However, displaying a color image on monochrome media (e.g., paper and e-ink displays) can be problematic: different colors with similar luminosity are impossible to distinguish on monochrome media. This problem is illustrated in the figure below, where it becomes nearly impossible to associate slices of a pie chart with corresponding labels.

(snip)

The only feasible solution is for authors to provide alternative image content that communicates effectively in monochrome media.

This is a common accessibility issue/mistake and fails WCAG 2.0 Success Criterion 1.4.1 Use of Color.

Yet there are some cases where authors cannot modify existing problematic contents, Use Case Requirements should mention this is an accessibility issue and authors are encouraged to avoid relying color alone.

Related resources:

• WebAIM Web Accessibility for Designers
• Good and Bad Examples of Use of Color

web developers have had to created

s/created/create/

a range of custom solutions to for serving

s/to//

The images and video are not being correctly sized for mobile.

http://usecases.responsiveimages.org/ lists src-N as 4th proposal in section 1.1.
While I find the title of the proposal (“Proposal for RespImg Syntax”) very confusing (“src-N Approach” would be better IMO), I think this proposal has been deprecated and shouldn’t be listed here anymore as the section description says it lists all in-development specifications.

The link pointing to the srcset specification ("the 'srcset' attribute)" is broken

I don’t see much difference between:
http://usecases.responsiveimages.org/images/devices.png

And the illustration being used to illustrate the art direction use case:
http://usecases.responsiveimages.org/images/art_direction.jpg

Cropping appears to happen in both examples even if the cropping is more pronounced in the art direction example. If you just look at the two examples side by side, it would be hard to figure out why they were different without the associated content.

I think it would be better to have an example where the resolution switching illustration kept its aspect ratio through all of the sizes of the image.

As discussed on http://ircbot.responsiveimages.org/bot/log/respimg/2015-03-05#T117019 we need to write down what's required in order for authors to be able to make their images go away for some breakpoints.

@aarongustafson - up for the task?

as we have now reached consensus on the UC&Reqs, we need to pass them over to the HTMLWG for a sanity check.

The current description for art direction says:

In a responsive design, it is typical to change the crop of an image so it can be targeted towards the features of a particular display (or set of displays)

The Nokia example shown later demonstrates that sometimes the changes aren’t simply cropping.

Perhaps we could change it to say something like:

“In a responsive design, it is typical to change an image so it can be targeted towards the features of a particular display (or set of displays). Sometimes this means cropping an image. Other times, it can mean a new form of an image that may have different proportions or may be changed in other ways to match changes in the layout.”

MathML support in browsers is spotty with not much hope of improvement in the short term. MathJax, an open source JS lib that can render MathML, does a good job but has performance and reliability issues. Given this situation, it would be nice to use responsive images for equations.

It would be nice to allow a server to produce responsive equation images at request-time. This would allow the server to avoid storing a lot of images for each equation. Instead, it would generate the image directly from the client properties and the MathML.

If the server also had access to the text column width, ambient point size, font, and text color, it could use this information to generate the best image. MathML allows a renderer to do line-breaking. A long equation in a narrow text column could be broken in many lines, resulting in a tall, narrow equation image. The server could also return the speech text for the equation for accessibility purposes. This would allow it to customize the speech based on the particular user's disabilities and preferences.

Since many notations are text-based (chemistry has several such notations), such a mechanism would be useful for more than just mathematics.

We need to get IPR commitments from CG members before we publish.

They need to click on red button:
http://www.w3.org/community/respimg/spec/26/commitments

This issue came up in the IRC (I think it was raised by @MattWilcox), and I wanted to document it and get some more discussion started. Not everyone agrees it's a problem, so please weigh in. I'm also not sure if/how this should be incorporated into a use case, so thoughts on that would be appreciated too.

Consider a situation where a developer wants a small image for a portrait orientation, and a larger image for a landscape orientation. The images have the same content, they are just different sizes.

Example 1:

    <picture>
        <source media="(orientation:portrait)" src="small.jpg" />
        <source media="(orientation:landscape)" src="large.jpg" />
    </picture>

In this case, a user might load the page in portrait mode, then re-orient their device to landscape. When this happens, the larger image would be downloaded. All is fine and good with the world.

However, if the user loads the page in landscape mode first, they download the larger image first. If they re-orient to portrait, there's no reason to download the smaller image. Since the larger image could have been simply resized, downloading the smaller one is a waste of bandwidth. Right now, we don't have a way to prevent this from happening.

One possible solution would be to stipulate that once a large image has been downloaded, smaller images should trigger a resize rather than a download. However, this causes problems with art direction. A developer may want the smaller image to have different content (e.g. be cropped differently) than the larger image.

Example 2:

    <picture>
        <source media="(orientation:portrait)" src="cropped.jpg" />
        <source media="(orientation:landscape)" src="full.jpg" />
    </picture>

In this case, an orientation change in either direction should trigger a download. That means we can't stipulate that a smaller image shouldn't be downloaded.

Another solution might be to group sources with identical content, e.g. with a name attribute.

Example 1a:

    <picture>
        <source name="full" media="(orientation:portrait)" src="small.jpg" />
        <source name="full" media="(orientation:landscape)" src="large.jpg" />
    </picture>

In this case, the browser would know based on name that the two sources have identical content. It would then know that it could safely scale down an image, rather than downloading a smaller version. When loaded in portrait and re-oriented to landscape, the larger version would still be downloaded. When loaded in landscape and re-oriented to portrait, the browser would resize the larger version rather than downloading a new file.

(This might also provide a nice added benefit when switching from portrait to landscape: a scaled-up version of the image could be displayed until the download of the large version is finished.)

Example 2a:

    <picture>
        <source name="cropped" media="(orientation:portrait)" src="cropped.jpg" />
        <source name="full" media="(orientation:landscape)" src="full.jpg" />
    </picture>

In this case, the browser would know based on name that the two sources have different content. It would know that an orientation change in either direction required downloading the appropriate source, and that it couldn't simply resize the image.

It would be nice to not have to leave the spec to file a bug

I'm not going all pull-requesty on this since maybe you don't want to make all these changes. Also this is a mix of typo stuff and style stuff, and don't think I don't love y'all just because I'm picky about word choices and sentence phrasing. A lot of this is straight-up Strunk&White sentence construction and "omit needless words" sort of stuff.

General note - We need to standardize the capitalization of all the H1s and H2s; right now they are a mix of Title Caps and Sentence capitalization schemes. I don't think it matters which one to go with, just pick one and run with it!

line 103 This document reflects the efforts of the Responsive Images Community Group (RICG) to present a comprehensive set of responsive image use case descriptions to the W3C and WHATWG (see bug 17061).

103 The RICG's goal for this document is to capture a complete set of developer requirements for responsive images.

97 These changes in media features and media type can reduce an image's ability to communicate effectively (e.g. images become blurry or pixelated).

99 As the number and varieties of high-density screens has increased (both on mobile and desktop devices), web developers have created custom solutions ...

115 Reliance on semantically inappropriate elements and CSS backgrounds
[The audience for this is people who love/create web standards, right? So we want to drive home that we're currently being forced to ignore standards.]

117 images in HTML without hacking a solution. [or maybe 'hacking together'?]

117 Developers have resorted to using div and other container elements to achieve the desired functionality.

121 Some approaches rely on JavaScript libraries or server-side solutions - both of which add unnecessary complexity to the development process.

130 The following use cases represent situations in which Web developers see a need for a browser-based solution for working with responsive images.

3.1 Art Direction
Web developers often need to provide different versions of the same image in order to communicate effectively across the wide range of screen resolutions and densities available on today's devices. If the screen is small and the image is scaled down, its details cannot be seen. Conversely, if the screen is large a larger image that depicts more information can be shown.

For example, in the left-most image in Figure 1, it is difficult to discern the man's facial expressions, where in the center image his face and gestures are clear. The image on the far right show the poor quality of an over-scaled image.

Fig. 1 Three images showing how zoom and crop can enhance or diminish the effectiveness of an image.

A typical use is to change the crop of an image so it can be targeted towards the features of a particular display (or set of displays):

• On large screens, a large image (e.g. of a figure with a broad background) is shown.
• On smaller screens, shrinking the image can reduce its relevance, usefulness, or legibility. The developer may want to show an alternate crop or layout of the same image.

A related use case is when orientation determines the source of the image, the crop, and how text flows around the image based on the size of the viewport. For example, on the Nokia Browser page describing the Meego browser, the Nokia Lumia is...

Bryan and Stephanie Rieger, the designers of the site, explained that on a wide screen,...

3.2 Design Breakpoints
In Web development, a breakpoint is one of a series of CSS Media Queries that update the styles of a page based on logical matching of media features. A single breakpoint represents a rule (or set of rules) that determines the point at which the contents of that media query are applied to a page’s layout.

[The whole Example & bullets about em's seem misplaced here? It feels redundant to section 3.6, and also more like a requirement - that we need to be able to use the same query params for CSS and images - rather than a use case.]

3.4 Monochrome and high-contrast
Displaying a color image on monochrome media (e.g., e-ink displays) can be problematic; different colors with similar luminosity are impossible to distinguish on monochrome media. Additionally, Microsoft has proposed a "high-contrast" media feature, which would enable developers to know if the user agent is operating in a high-contrast mode. Developers need to be able to serve images specifically designed for monochrome or high-contrast viewing modes.

3.5 Mobile-first and desktop-first responsive design
Many responsive designs use the “mobile-first” development pattern: starting with a simple, linear layout and using media queries to increase layout complexity for larger screen sizes. In such designs, web developers generally serve small images first and increase the size of images as required.

“Desktop-first” responsive design takes the opposite approach, starting with the large-screen design and using media queries to simplify for smaller displays. With this approach the default images served are usually the largest, with smaller images being called when necessary.

Both approaches are based on a single image having multiple source files, where media queries determine which version of the image should be displayed.

[Not sure we need another reminder that our current solutions suck. I mean, the reader is already 60% into this document at this point. If we don't already have them on that point, we're doomed.]

3.6 Relative Units
...when a user zooms into a design, proportionally scaled images can be blurry or pixelated, affecting the image's impact and function.

3.7 Dynamically acquired image data [note singular 'image' in this title]
There are cases where the image data is dynamically generated (e.g., using canvas element and related APIs) or is acquired from the device itself (e.g., from the camera on a phone or tablet). Developers need practical means to interface programmatically with the source of an image or a set of images.

4. Requirements

8: The solution must afford developers the ability to define which version of the image should be used as the fallback image.

9: The solution must allow developers match the media queries used in their design, including using min-width or max-width, and relative units (e.g., em, rem) or fixed units (e.g., px).

[Maybe that should include a link to the part of the CSS spec that talks about what can be used in media queries? Ideally we want them to match 1:1, right? It would suck to be that one person who's actually using min-height and have this solution fail because we only declared it needed to work with width.]

10: - The solution must be responsive to environmental changes in relative units (e.g., when the user increases the base font size of the browser).

[This one may be superfluous if 9 is adjusted to say "everything we can do in a media query, we need to do here."]

Because we were not sanitizing the titles of Github bugs, we left ourselves open to a XSS attack. @marcoscaceres tried to fix this with:

htmlspecialchars($var, ENT_QUOTES, 'UTF-8').

But he is no PHP security expert.

Section '3.6 Matching media features and media types':
In the paragraph that commences with 'Allowing developers to reference images...' there is the word 'infeasible', which is not correct. The correct word is 'unfeasible'.

The current Art Direction figure in the spec is not ideal. It would be nice to have a diagram that both explains the concept and is, itself, also responsive (when used with picturefill).

Link to file a bug is broken

The sentence in the brackets (e.g., the solution would work well with a browser's the preload scanner) at bullet point 10 in section 4. could do with a rewrite:

(e.g. the solution would work well with the browser's preload scanner)

A responsive image is an image that a developer explicit adapts

Should be

A responsive image is an image that a developer explicitly adapts

Per https://www.w3.org/Bugs/Public/show_bug.cgi?id=17061

Hy there,

I would really need to be able to change the additional attributes applied to the <img> tag, in particular those are object-fit, object-position and class.

I need this especially for professional art direction and to compensate for focal points of different renditions.

Suppose the following markup:

<picture>
  <source
    srcset="foo-sm.jpg, [email protected] x2"
    media="(min-width: 200px)"
    object-position="top center"
  />
  <source
    srcset="foo-md.jpg, [email protected] x2"
    media="(min-width: 600px)"
    class="medium-addition-for-a-class"
  />
  <source
    srcset="foo-lg.jpg, [email protected] x2"
    media="(min-width: 900px)"
  />
  <img
    src="foo.jpg"
    srcset="[email protected] x2"
    class="picture__img"
    alt="Foo image"
    object-fit="cover"
    object-position="center"
  />
</picture>

In 3.2 Design Breakpoints, "breakpoints" is used, but never defined, e.g., "Developers currently match specific CSS Media Queries "breakpoints" for images to the breakpoints that they have defined in the CSS of their responsive designs."

Following the "CSS Media Queries" link, there is no definition there either.

Typically breakpoints are used to refer to stopping program execution for debugging, so some clarification here would be good.

There are currently some limitations with Respec (e.g., custom headers, custom logo, etc.) that don't make it well suited to our needs. Need to swap the spec to using Anolis instead.

Perhaps change "the PNG format is better-suited to these needs." to "the PNG and WebP lossless formats are better-suited to these needs."?

WebP Lossless offers faster decoding than PNG, and 25-30% smaller file sizes.

The relative units section in the use-cases document doesn't define the problem well IMO. The linked article is preferring the use of EMs because of an old WebKit bug impacting zoom+/- on px based MQs, that was since fixed.

The real reason EM based breakpoints should be preferred in RWD is that the user's font size preferences can impact the MQs, unlike PX based MQs.

It may also be possible that some devices have em/px ratio other than 16, meaning that they may look broken if breakpoints are set in px only.

I think that's a little outdated. ^_^

@Wilto could you please reach out to key participants of the CG for a review of the UC&Reqs doc? We particularly need help with the accessibility and API design requirements (i.e., have they been sufficiently articulated or are we missing something?).

In the interview with Odin, he raised a good use case for allowing users to say which source they prefer from a set of sources (e.g., "always send me the lowest quality one, I'm roaming!"). This is akin to what Opera Mini supports already - users can select what quality to receive images at. This use case needs to be captured in the UC&Reqs document, a long with a suitable requirement a "truly declarative" model be used... or something similar.

In section 3.2: @media screen (max-width: 41em) {} -> @media screen and (max-width: 41em) {}

Need to consider possible use cases and/or requirements and/or notes about how copying and sharing image URLs would work given multiple sources (and possibly multiple formats).

See comments from Josh Fraser @ Torbit:

The biggest problem we ran into that we didn't expect is the number of people who grab the URL of the image and share it on Facebook, Twitter, email, whatever. This of course breaks for everyone not using a browser that supports it. We can serve the images up each time from a single URL based on the User-Agent, but that's not something you can do at the browser level.

In light of the use case gathering process, I was reading: http://www.codinghorror.com/blog/2008/02/i-repeat-do-not-listen-to-your-users.html

Key bits:

To discover which designs work best, watch users as they attempt to perform tasks with the user interface. This method is so simple that many people overlook it, assuming that there must be something more to usability testing. [It] boils down to the basic rules of usability:

• Watch what people actually do.
• Do not believe what people say they do.
• Definitely don't believe what people predict they may do in the future.

From which I think we should do run a simple developer challenge (except without using the dicky word "challenge" :) ).

It would be a simple test like: "using either picture or img@srcset, show us how you would make this image responsive". And we provide 3 images.

We could learn a lot from it to see what syntax developers prefer to solve the problem. We would need about 400+ responses for the survey to be statistically valid.

In the event that the writing-mode of a container or the document changes from horizontal to vertical, it would be helpful if there was a media query for writing-mode such that an image of a different orientation could be served up with the srcset attribute.

Currently, it is possible to adjust orientation using transform: rotate(-90deg) but a rotated image could end up overflowing the container especially if it is anything other than a square. For live example, please see demo.

Uncheck the toggle to go into horizontal mode and scroll down to the fourth image in the article. When the viewport is narrower than the width, there is a horizontal overflow, because this image was rotated.

Not sure how feasible it is to implement such a feature, but it would be a nice to have.

The group is feeling pretty good about the document now, so we are ready to hopefully move it to the HTMLWG.

See also: #24

11 The solution SHOULD offer an method to leverage breakpoints defined in CSS.
12 The solution SHOULD support a simplified syntax to support primary use case 3.1 (preferably a list of images and their dimensions), in order to reach users of content management systems and those without detailed knowledge of device queries.

As a complete solution to these requirements exists, and these requirements have been expressed repeatedly by a large variety of contributors on the respimg mailing list, these requirements should be considered for inclusion.

The first draft of the "Use Cases and Requirements for Standardizing Responsive Images" needs a proof read and review... and consensus from the CG to formally present them to the HTMLWG.

Going over the use-cases a few days ago, I've noticed that we left out one major use-case: That the solution will have high-performance.

I propose to add a use-case claiming that the solution should be (at least as its default) preloader-friendly.

Thoughts?