Responsive Images Community Group

The picture Element

Editor’s Draft, 14 April 2014

This version:
http://picture.responsiveimages.org
Editor’s Draft:
http://picture.responsiveimages.org
Test Suite:
None Yet
Editors:
Tab Atkins (Google)
Marcos Cáceres (Mozilla)
Mat Marquis
Yoav Weiss
(Opera Software)
Version History:
Commit History
Github commits on Twitter
Participate:
Join the Responsive Images Community Group
Public Mailing List
IRC: #respimg on W3C’s IRC
Twitter
Github

Abstract

The picture element is intended to give authors a way to control which image resource a user agent presents to a user, based on media query and/or support for a particular image format.

The primary purpose of the picture element is to address use cases [respimg-usecases] that are left unaddressed by the srcset attribute [HTML]; the most important being art direction. Other use cases, such as matching media features and media types and matching on supported image formats, are also addressed by this specification.

Table of Contents

1 Introduction

This specification provides developers with a means to declare multiple versions of an image at different resolutions, and, through [MEDIAQ] (CSS Media Queries), a means to create specialized presentations of an image and control when they are presented to a user. This is achieved by introducing the picture element to HTML, and by enhancing the source element to support specifying multiple source urls.

By relying on [MEDIAQ], a user agent can respond to changes in the browsing environment by selecting the image source that most closely matches the browsing environment – thus embodying a technique known as responsive web design directly in the HTML markup. Media features that a user agent can potentially respond to include, but are not limited to, pixel widths and heights, and pixel densities, as well as environmental lighting conditions, changes in orientation, and changes in media type such as going from screen to print.

The picture element remains backwards compatible with legacy user agents (they will use the child img element) while offering the same accessibility options as the existing img element.

1.1 When to use picture

The picture element should be used when an image source exists in multiple densities, or when a responsive design dictates a somewhat different image on some types of screens (“art direction”).

The picture element is not a general replacement for the img element. When there is only a single image source, authors should just use the img element as usual, rather than cluttering their page with additional unnecessary syntax.

1.2 Examples of usage

This example shows a basic usage of the picture element to present the same image at multiple resolutions.
  <picture>
    <source srcset="pic1x.jpg 1x, pic2x.jpg 2x, pic4x.jpg 4x">
    <img alt="A rad wolf"
        src="pic1x.jpg" width="500" height="500">
  </picture>

In this example, the user agent will choose one of the three URLs—“pic1x.jpg”, “pic2x.jpg”, or “pic4x.jpg”—depending on whether the quality of the user’s screen and perhaps other factors, such as the user’s bandwidth.

It will then load the chosen URL in the img child, as if it were specified in the img element’s own src attribute.

For backwards compatibility with older user agents that don’t yet understand the picture element, one of the URLs is duplicated in the img element’s src attribute. This is ignored in modern user agents, but will result in something useful (though perhaps lower-resolution than the user would like) being displayed even in older user agents.

Alternately, because this example uses only a single source element and none of the more advanced attributes, it can be rewritten to use only img:

<img alt="A rad wolf" srcset="pic1x.jpg 1x, pic2x.jpg 2x, pic4x.jpg 4x"
                      src="pix1x.jpg" width="500" height="500">
When doing responsive design, it is sometimes necessary to produce slightly different versions of an image depending on the user’s device. For example, on a small screen like a phone, the author might want to show a zoomed-in crop of an image, while on a larger screen the full image can be displayed instead. This can be accomplished with multiple source elements:
  <picture>
    <source media="(min-width: 45em)" srcset="large.jpg">
    <source media="(min-width: 18em)" srcset="med.jpg">
    <img src="small.jpg" alt="The president giving an award.">
  </picture>

In this example, depending on the user’s screen size, one of the URLs will be downloaded. For example, on a large desktop screen “large.jpg” will be displayed in the img element.

Of course, one can use both multiple source elements and multiple sources within a single source element together:
  <picture>
    <source media="(min-width: 45em)" srcset="large-1.jpg, large-2.jpg 2x">
    <source media="(min-width: 18em)" srcset="med-1.jpg, med-2.jpg 2x">
    <source srcset="small-1.jpg, small-2.jpg 2x">
    <img src="small-1.jpg" alt="The president giving an award.">
  </picture>

In this example, the user agent first chooses which set of sources to look at, depending on the size of the user’s screen. Then it chooses which of the different-density sources to load, based on information it knows about the user’s device.

If the final size of the img element isn’t known ahead of time, it can be difficult or impossible to specify a density descriptor for the image sources. For example, if an image is meant to take up the full width of the user’s screen, an image that is 600 image pixels wide will be approximately 2x on a small 320px-wide phone, approximately 1x on a larger tablet display, and less than 1x on a large desktop monitor.

To help with this, rather than specifying the densities of each image source, the sizes of each image source can be specified directly, along with the size of the img element. The user agent will then automatically calculate the effective pixel density of the image and choose which one to download accordingly.

  <picture>
    <source sizes="100vw" srcset="pic400.jpg 400w, pic800.jpg 800w, pic1600.jpg 1600w">
    <img src="pic400.jpg" alt="The president giving an award.">
  </picture>

In this example, the image source is provided at three sizes—400 pixels wide, 800 pixels wide, and 1600 pixels wide. As well, it declares that the img element is intrinsically sized to be as wide as the entire viewport. If the user’s device is 320px wide, this is equivalent to specifying pic400.jpg 1.25x, pic800.jpg 2.5x, pic1600.jpg 5x. On the other hand, if the user’s device is 1200px wide, this is equivalent to specifying pic400.jpg .33x, pic800.jpg .67x, pic1600.jpg 1.33x.

With this information, the user agent can choose the correct image source to download regardless of how large the user’s device is.

The previous example showed how to deal with the img element’s size being unpredictable because it was a fraction of the viewport’s size. Sometimes, though, the size of an image can also change based on a site’s layout breakpoints.

For example, say your site had three basic layouts:

Single-column (100%) on small screens, two-column (50%) on medium screens, and three-column (approximately 33%) on large screens.

Assuming that the same image is supposed to be used at all of these layouts (that is, you aren’t doing art direction cropping to optimize the display of the image for a given size), then all of these cases can be addressed by a handful of images at various sizes. In that case, the following code specifies the image’s display:

  <picture>
    <source sizes="(max-width: 30em) 100vw, (max-width: 50em) 50vw, calc(33vw - 100px)"
            srcset="pic100.jpg 100w, pic200.jpg 200w, pic400.jpg 400w,
                    pic800.jpg 800w, pic1600.jpg 1600w, pic3200.jpg 3200w">
    <img src="pic400.jpg" alt="The president giving an award.">
  </picture>

The sizes attribute sets up the layout breakpoints at 30em and 50em, and declares the image sizes between these breakpoints to be 100vw, 50vw, or calc(33vw - 100px).

The six image sources provided automatically cover every reasonable possibility. For small screens (phone size, or even smaller, like watches), anything from the 100 pixel wide image to the 800 pixel wide image may be downloaded, depending on screen size and density. For medium and large screens, anything from the 400 pixel wide image and up may be chosen. The author doesn’t have to do any math or complex figuring, just provide the image in enough sizes to cover everything they believe reasonable.

Specifying this information with multiple source elements, one for each of the media queries, is possible, but much more verbose, as the srcset attributes have to be duplicated for each of them.

This example shows how picture and other HTML elements can be used together.
  <figure>
      <picture>
        <source media="(min-width: 45em)" srcset="large-1.jpg, large-2.jpg 2x">
        <source media="(min-width: 18em)" srcset="med-1.jpg, med-2.jpg 2x">
        <img src="small-1.jpg" srcset="small-2.jpg 2x" alt="">
      </picture>
      <figcaption>A figure of a fox jumping over a lazy box.</figcaption>
  </figure>
The srcset and sizes attributes can also be used on the img element directly, removing the need for the picture and source elements in simpler cases.
  <img src="pic400.jpg" alt="The president giving an award." sizes="100vw"
      srcset="pic400.jpg 400w, pic800.jpg 800w, pic1600.jpg 1600w">

In this example, the sizes attribute could also be removed, since the default value is 100vw.

The type attribute can be used to serve images of different file types depending on user agent support.
    <picture>
      <source type="image/webp" srcset="dogs-1.webp, dogs-2.webp 2x">
      <source type="image/vnd.ms-photo" srcset="dogs-1.jxr, dogs-2.jxr 2x">
      <img src="dogs-1.jpg" srcset="dogs-2.jpg 2x" alt="Hundreds of hot dogs" width="600" height="200">
    </picture>

In this example, the user agent will choose the first source that has a type attribute with a supported MIME type. If the user agent supports WebP images, the first source element will be chosen. If not, but the user agent does support JPEG XR images, the second source element will be chosen. If neither of those formats are supported, the img element will be chosen. Then, the user agent chooses which of the different-density sources to load, based on information it knows about the user’s device.

1.3 Relationship to srcset

The srcset attribute allows authors to define various image resources and “hints” that assist a user agent in determining the most appropriate image source to display. Given a set of image resources, the user agent has the option of either following or overriding the author’s declarations to optimize the user experience based on criteria such as display density, network connection type, user preferences, and so on.

The picture element defines conditions under which the user agent needs to follow the author’s explicit instructions when selecting which resource to display. This includes image sources with inherent sizes designed to align with layout variations specified in CSS media queries (see: design breakpoints, and relative units) or content variations for increased clarity and focus (i.e., art direction) based on the client’s viewport.

The proposed solutions are not mutually exclusive. They work together to address the complete set of use cases and requirements for responsive images (see [respimg-usecases]).

Furthermore, this specification extends the srcset attribute to adequately address the viewport based selection use case.

2 Definitions

The following terms are used throughout this specification so they are gathered here for the readers convenience. The following list of terms is not exhaustive; other terms are defined throughout this specification.

The following terms are defined by the [HTML] specification: skip whitespace, collect a sequence of characters, space character, split a string on spaces, valid non-negative integer, rules for parsing non-negative integers, valid floating-point number, rules for parsing floating-point number values, valid non-empty URL, update the image data, valid media query, and current pixel density.

3 The picture Element

Name:picture
Categories:Flow content, Phrasing content, Embedded content, Palpable content
Contexts:Where embedded content is expected
Content model:Zero or more source elements, followed by one img element.
Attributes:Global attributes

The picture element is a container which provides multiples sources to its contained img element, allowing the displayed image to vary based on the density of the screen or other environmental factors expressed as media queries.

Note: picture is somewhat different from the similar-looking elements video and audio. While all of them contain source elements, the src attribute has no meaning when the element is nested within picture, and the resource selection algorithm is different. As well, the picture element itself does not display anything; it merely provides a context for its contained img that enables it to choose from multiple source urls.

An img element is associated with a source set.

A source set is a set of zero or more image sources, a source size, and optionally a media query.

An image source is a URL, and optionally either a density descriptor, or a width descriptor.

A source size is a <source-size-value>. When a source size has a unit relative to the viewport, it must be interpreted relative to the img element’s document’s viewport. Other units must be interpreted the same as in media queries. [MEDIAQ]

The relevant mutations for an img element are as follows:

When an img element has a current pixel density that is zero, the element’s intrinsic dimensions must be +Infinity CSS pixels by +Infinity CSS pixels.

Note: User agents are expected to have limits in how big images can be rendered, which is allowed by HTML’s hardware limitations clause. [HTML]

3.1 Sub-Algorithms

3.1.1 Selecting an Image Source

In the HTML spec, in the algorithm update the image data, replace the whole step that calls "processing the image candidates" with the following step:
  1. Let selected source and selected pixel density be the URL and pixel density that results from selecting an image source.

Also in the HTML spec, in the algorithm "The user agent may at any time run the following algorithm to update an img element’s image in order to react to changes in the environment.", make the following changes:

While at it, do the following changes elsewhere in the HTML spec:

When asked to select an image source for a given img element el, user agents must do the following:

  1. Update the source set for el.
  2. If el’s source set is empty, return null as the URL and undefined as the pixel density and abort these steps.
  3. Otherwise, take el’s source set and let it be source set.
  4. In a UA-specific manner, choose one image source from source set. Let this be selected source.
  5. Return selected source and its associated pixel density.

This just selects a single image and then sticks with it, unlike CSS’s image(), but similar to srcset. That okay?

3.1.2 Updating the Source Set

When asked to update the source set for a given img element el, user agents must do the following:

  1. Initially set el’s source set to an empty source set.
  2. If el has a parent node and that is a picture element, let elements be an array containing el’s parent node’s child elements, retaining relative order. Otherwise, let elements be array containing only el.
  3. Iterate through elements, doing the following for each item child:
    1. If child is el:
      1. If child has a srcset attribute, parse child’s srcset attribute and let the returned source set be source set. Otherwise, let source set be an empty source set.
      2. Parse child’s sizes attribute and and let source set’s source size be the returned value.
      3. If child has a src attribute and source set does not contain an image source with a density descriptor value of 1, and no image source with a width descriptor, append the child’s src attribute value to the source set.
      4. Set the returned source set to be el’s source set.
      5. Exit the "Updating the Source Set" algorithm.
    2. If child is not a source element, continue to the next child. Otherwise, child is a source element.
    3. If child does not have a srcset attribute, continue to the next child.
    4. Parse child’s srcset attribute and let the returned source set be source set.
    5. If source set has zero image sources, continue to the next child.
    6. If child has a media attribute, and its value is a valid media query which evaluates to true, attach the media query to source set. If its value is not a valid media query, or is a valid media query that evaluates to false, continue to the next child.
    7. Parse child’s sizes attribute and let source set’s source size be the returned value.
    8. If child has a type attribute, and its value is an unknown or unsupported MIME type, continue to the next child.
    9. Normalize the source densities of source set.
    10. Set source set to be el’s source set.
    11. Exit the "Updating the Source Set" algorithm.

I’d like to allow individual sources to specify a type as well with a type() function, overriding the default type specified by the type attribute, but I’m keeping things simple for now.

3.1.3 Parsing a srcset Attribute

When asked to parse a srcset attribute from an element, parse the value of the element’s srcset attribute as follows:

  1. Let input be the value passed to this algorithm.
  2. Let position be a pointer into input, initially pointing at the start of the string.
  3. Let raw candidates be an initially empty ordered list of URLs with associated unparsed descriptors. The order of entries in the list is the order in which entries are added to the list.
  4. Splitting loop: Skip whitespace.
  5. Collect a sequence of characters that are not space characters, and let that be url.
  6. If url ends with a U+002C COMMA character (,), remove that character from url and let descriptors be the empty string. Otherwise, follow these substeps:
    1. If url is empty, then jump to the step labeled descriptor parser.
    2. Collect a sequence of characters that are not U+002C COMMA characters (,), and let that be descriptors.
  7. Add url to raw candidates, associated with descriptors.
  8. If position is past the end of input, then jump to the step labeled descriptor parser.
  9. Advance position to the next character in input (skipping past the U+002C COMMA character (,) separating this candidate from the next).
  10. Return to the step labeled splitting loop.
  11. Descriptor parser: Let candidates be an initially empty source set. The order of entries in the list is the order in which entries are added to the list.
  12. For each entry in raw candidates with URL url associated with the unparsed descriptors unparsed descriptors, in the order they were originally added to the list, run these substeps:
    1. Let descriptor list be the result of splitting unparsed descriptors on spaces.
    2. Let error be no.
    3. Let width be absent.
    4. Let density be absent.
    5. For each token in descriptor list, run the appropriate set of steps from the following list:
      If the token consists of a valid non-negative integer followed by a U+0077 LATIN SMALL LETTER W character
      1. If width and density are not both absent, then let error be yes.
      2. Apply the rules for parsing non-negative integers to the token. Let width be the result.
      If the token consists of a valid floating-point number followed by a U+0078 LATIN SMALL LETTER X character
      1. If width and density are not both absent, then let error be yes.
      2. Apply the rules for parsing floating-point number values to the token. Let density be the result.
    6. If error is still no, then add a new image source to candidates whose URL is url, associated with a width width if not absent and a pixel density density if not absent.
  13. Return candidates.

An image candidate string consists of the following components, in order:

  1. Zero or more space characters.
  2. A valid non-empty URL that does not end with a U+002C COMMA character (,), referencing a non-interactive, optionally animated, image resource that is neither paged nor scripted.
  3. Zero or more space characters.
  4. Zero or one of the following:

  5. Zero or more space characters.

If the source or img element has a sizes attribute present, all image candidate strings for that element must have the width descriptor specified.

If an image candidate string for an source or img element has the width descriptor specified, all other image candidate strings for that element must also have the width descriptor specified.

The specified width in an image candidate string’s width descriptor must match the intrinsic width in the resource given by the image candidate string’s URL.

3.1.4 Parsing a sizes Attribute

When asked to parse a sizes attribute from an element, parse a comma-separated list of component values from the value of the element’s sizes attribute (or the empty string, if the attribute is absent), and let unparsed sizes list be the result.

Simplest algo

Attempt to parse each entry of unparsed sizes list as a <source-size> or <source-size-value>. For the first one that parses successfully, return the value of the <source-size-value> so parsed. If no entry parses successfully, return 100vw.

Slightly more complex algo

For each unparsed size in unparsed sizes list:

  1. Remove all consecutive <whitespace-token>s from the end of unparsed size. If this removes all the tokens from unparsed size, continue to the next iteration of this algorithm.
  2. If the last component value in unparsed size is a valid positive <source-size-value>, let size be its value and remove the component value from unparsed size. Otherwise, continue to the next iteration of this algorithm.
  3. If all remaining component values in unparsed size are <whitespace-token>s, return size and exit this algorithm.
  4. Parse the remaining component values in unparsed size as a <media-condition>. If it does not parse correctly, or it does parse correctly but the <media-condition> evaluates to false, continue to the next iteration of this algorithm.
  5. Return size and exit this algorithm.

If the above algorithm exhausts unparsed sizes list without returning a size value, return 100vw.

A valid source size list is a string that matches the following grammar: [CSS3VAL]

  <source-size-list> = <source-size>#+ [ , <source-size-value> ]? | <source-size-value>
  <source-size> = <media-condition> <source-size-value>
  <source-size-value> = <length>

A <source-size-value> must not be negative and must not be zero.

Note: While a valid source size list only contains a bare <source-size-value> (without an accompanying <media-condition>) as the last entry in the <source-size-list>, the parsing algorithm technically allows such at any point in the list, and will accept it immediately as the size if the preceding entries in the list weren’t used. This is to enable future extensions, and protect against simple author errors such as a final trailing comma.

3.1.5 Normalizing the Source Densities

An image source can have a density descriptor, a width descriptor, or no descriptor at all accompanying its URL. Normalizing a source set gives every image source a density descriptor.

When asked to normalize the source densities of a source set source set, user agents must do the following:

  1. Let source size be source set’s source size.
  2. For each image source in source set:
    1. If the image source has a density descriptor, continue to the next image source.
    2. Otherwise, if the image source has a width descriptor, replace the width descriptor with a density descriptor with a value of width descriptor / source size and a unit of x.
    3. Otherwise, give the image source a density descriptor of 1x.

3.2 Interaction with the Preload Scanner

Some UAs utilize a “preload scanner”, which is a simplified parser that runs ahead of the main HTML parser during the initial load of the page, looking for URLs that it can begin loading as quickly as possible.

The definition of the picture element is compatible with a preload scanner, at least for some media queries.

If a UA uses a preload scanner, it must recognize when a an img element is a child of a picture, and use the full algorithm to select an image source to determine which URL to pre-load, with the following caveats:

In some cases, the preload scanner or the parser cannot be aware of the final viewport’s dimensions when it encounters a picture or a img element. One example of such a case is when a @viewport definition is located in an external style-sheet. Another is when the preload scanner is operating on a iframe based document, which has its dimensions defined in an external style-sheet.

Authors should be aware of these limitations, and should follow the following best practices:

3.3 HTMLPictureElement interface

interface HTMLPictureElement : HTMLElement {
  };

The HTMLPictureElement serves as an extension point for future API extensions.

4 Changes to the source Element

The source element, if it has a picture element as its parent, must have the srcset attribute specified. The value must consist of one or more image candidate strings, each separated from the next by a U+002C COMMA character (,). If an image candidate string contains no descriptors and no space characters after the URL, the following image candidate string, if there is one, must begin with one or more space characters.

The source element, if it has a picture element as its parent, may also have the sizes attribute specified. If it is specified, the value must be a valid source size list.

The source element, if it has a picture element as its parent, may also have the media attributes specified. If it is specified, the value must contain a valid media query.

The source element, if it has a picture element as its parent, must not have the src attribute specified.

The type attribute, when specified on a source element that has a picture element as its parent, gives the type of the images in the source set, to allow the user agent to skip to the next source if it does not support the given type.

If the type attribute is not specified, the user agent will not select a different source element if it finds that it does not support the image format after fetching it.

The IDL attributes srcset, sizes and media must reflect the respective content attributes of the same name. [HTML]

partial interface HTMLSourceElement {
    attribute DOMString srcset;
    attribute DOMString sizes;
    attribute DOMString media;
  };

When a source element is a child of a picture element and has a following sibling source element or img element with a srcset attribute specified, it must have at least one of the following:

5 Changes to the img Element

The img element, if it has a srcset attribute, may have a sizes attribute specified. If it is specified, the value must be a valid source size list.

The IDL attribute sizes must reflect the sizes content attribute. [HTML]

partial interface HTMLImageElement {
    attribute DOMString sizes;
    readonly attribute DOMString? currentSrc;
  };

The IDL attribute currentSrc must initially be set to the value null. When asked to select an image source for the img element, if the algorithm returns a selected source, the currentSrc IDL attribute must be set to the serialization of that source’s URL; otherwise, it must be set to the value null.

6 Acknowledgements

A complete list of participants of the Responsive Images Community Group is available at the W3C Community Group Website.

Contributions also from: David Newton, Ilya Grigorik, John Schoenick and Leon de Rijke. Special thanks to Adrian Bateman for providing the group with guidance.

Conformance

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

References

Normative References

[CSS3VAL]
Håkon Wium Lie; Tab Atkins; Elika J. Etemad. CSS Values and Units Module Level 3. 30 July 2013. W3C Candidate Recommendation. (Work in progress.) URL: http://www.w3.org/TR/2013/CR-css3-values-20130730/
[HTML]
Ian Hickson. HTML. Living Standard. URL: http://whatwg.org/html
[MEDIAQ]
Florian Rivoal. Media Queries. 19 June 2012. W3C Recommendation. URL: http://www.w3.org/TR/2012/REC-css3-mediaqueries-20120619/
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. URL: http://www.ietf.org/rfc/rfc2119.txt
[respimg-usecases]
Marcos Cáceres; et al. Use Cases and Requirements for Standardizing Responsive Images. NOTE. URL: http://www.w3.org/TR/respimg-usecases/

Informative References

Index

Property index

No properties defined.

Issues Index

In the HTML spec, in the algorithm update the image data, replace the whole step that calls "processing the image candidates" with the following step:
  1. Let selected source and selected pixel density be the URL and pixel density that results from selecting an image source.

Also in the HTML spec, in the algorithm "The user agent may at any time run the following algorithm to update an img element’s image in order to react to changes in the environment.", make the following changes:

While at it, do the following changes elsewhere in the HTML spec:

This just selects a single image and then sticks with it, unlike CSS’s image(), but similar to srcset. That okay?
I’d like to allow individual sources to specify a type as well with a type() function, overriding the default type specified by the type attribute, but I’m keeping things simple for now.