Rehydration

This commit introduces a "rehydration" mode to the Glimmer VM.

-- What is Rehydration?

The rehydration feature means that the Glimmer VM can take a DOM tree
created using Server Side Rendering (SSR) and use it as the starting
point for the append pass.

Rehydration is optimized for server-rendered DOMs that will not need
many changes in the client, but it is also capable of making targeted
repairs to the DOM:

- if a single text node's value changes between the server and the
  client, the text node itself is repaired.
- if the contents of a `{{{}}}` (or SafeString) change, only the
  bounds of that fragment of HTML are blown away and recreated.
- if a mismatch is found, only the remainder of the current element
  is cleared.

What this means in practice is that rehydration repairs problems
in a relatively targeted way, and doesn't blows away more content
than the contents of the current element when a mismatch occurs.

Near-term work will narrow the scope of repairs further, so that
mismatches inside of blocks only clear out the remainder of the
content of the block.

-- What Changes Were Needed?

Previously, code in the append pass was permitted to make DOM
changes at any point, so long as they made the changes through
the stateless "DOM Helper" abstraction, which ensured that
code didn't inadvertantly rely upon environment-specific
behaviors. This includes both browser quirks (including the
most recent version of Safari), and restricting Glimmer to
the subset of DOM used in `SimpleDOM`, the library that is
responsible for emulating the DOM in our current Server
Side Rendering implementation.

Rehydration works by replacing attempts to create DOM nodes
with a check for whether the node that Glimmer is trying to
create is already present. It maintains a "cursor" against
the unhydrated DOM, and when Glimmer attempts to create an
element, it first checks to see whether the candidate node
matches the node that Glimmer is trying to create.

For example, if Glimmer wants to create a text node,
the rehydrator checks to see if the node under the cursor
is a text node. If it is a text node, it updates its
contents if necessary. If not, it begins a coarser-grained
repair (which, as described above, is never worse than
clearing out the rest of the DOM for the current element).

This requires that the core DOM operations not only go
through a stateless abstraction (DOM Helper), but also
have a choke-point through a stateful builder that can
maintain the candidate cursor and initiate repairs.

Most of this commit restructures code so that DOM operations during
the append pass go throug the central ElementBuilder in all cases.

Part of this process involved restructuring the code that manages
dynamic content and dynamic attributes to simplify them and make
them a better fit for ensuring that all DOM operations in the
append pass go through the ElementBuilder.

-- Serialize Mode

This commit also introduces a dedicated "serialize" mode that
server-side renderers should use to generate the DOM.

The serialize mode is reponsible for inserting additional markers
into the DOM to serialize boundaries that otherwise serialize
incorrectly.

For example, it inserts a comment node between two adjacent
text nodes so that they remain separated when rehydrating.

The serialize mode does not prescribe a DOM implementation;
any DOM that is compatible with the well-defined SimpleDOM
subset will work (including a real browser's DOM, JSDOM,
or SimpleDOM); it just ensures that the created DOM will
round-trip through a string of HTML.

Importantly, the rehydrator mode assumes that the server-
provided DOM was created in serialize mode.

-- Future Work: Attributes and Properties

Today, when you write something like `<div title={{value}}>`,
Glimmer attempts to use DOM properties if possible. This
works, more or less, because the DOM automatically reflects
properties to attributes in most cases.

This means that if you write `<img src={{url}} />` in a template,
we will set the `src` property, which the DOM automatically
reflects onto the `src` attribute.

Using properties where possible as opposed to attributes in
all cases addresses a number of use cases. For example,

- `<select value={{someValue}}>` will select the correct
  `<option>` automatically
- `<div onclick={{action foo}}>` "just works" to set
  event handlers.
- `<textarea value={{someValue}}>` updates the inside of
  the textarea instead of requiring you to deal with
  the text node contents of the textarea.

We intend to move to all-attributes-all-the-time in the future, but
we need to address these use-cases. There are various approaches
under consideration (including a few special-cases, element
modifiers, and an alternate sigil), but they are out of scope
for this PR.

In order to address this gap for now, this PR works around the slight
semantic inconsistency. When the rehydrator pushes an element, it
snapshots the list of all attributes that are already present on
the element as a list of candidates for attribute removal. When
Glimmer attempts to set an attribute **or property** on the element,
the rehydrator removes it from the list of candidates for attribute
removal.

When an element's attributes are "flushed" (a step in the Glimmer
VM), the rehydrator removes any attributes that it didn't see during
the append step.

This should work in the vast majority of cases because:

First, it doesn't stop properties from being set, so any case where
properties were set before continue to be set now.

Second, if an attribute was present and the same-named property was
set in the client, the attribute will remain in the DOM. This
can only go wrong if the application was relying on a property
being set that **didn't** set the same-named attribute. This
is very rare; one example might be if an app was relying on using
`form.reset()` to reset a field back to `''`, relying on the
fact that `<input value={{someValue}}>` doesn't **actually**
set the value attribute.

This is very unlikely because empirically very few people use
`.reset()` in Ember or Glimmer, which is **why using a property
instead of an attribute here worked in the first place**.

In short, if someone is relying on the fact that an element's
attribute actually sets a property also **relies on the fact that
it doesn't set an attribute,** the current workaround will repair
inconsistently. This should be very rare and something we can
address next.

Author: wycats

.vscode/settings.json

@@ -11,7 +11,14 @@
     "**/node_modules": true,
     "**/dist": true,
     "dist": true,
-    "tmp": true
+    "tmp/**": true
+  },
+  "files.watcherExclude": {
+    "**/.git/objects/**": true,
+    "**/.git/subtree-cache/**": true,
+    "**/node_modules/**": true,
+    "tmp": true,
+    "dist": true
   },
   "files.trimTrailingWhitespace": true,
   "editor.renderWhitespace": "boundary",

package.json

@@ -58,4 +58,4 @@
     "tslint": "^4.0.2",
     "typescript": "^2.2.0"
   }
-}
+}

packages/@glimmer/interfaces/lib/dom/simple.d.ts

@@ -10,18 +10,18 @@ export type Namespace =
   | "http://www.w3.org/2000/xmlns/";
 
 export enum NodeType {
-  Element,
-  Attribute,
-  Text,
-  CdataSection,
-  EntityReference,
-  Entity,
-  ProcessingInstruction,
-  Comment,
-  Document,
-  DocumentType,
-  DocumentFragment,
-  Notation
+  Element = 1,
+  Attribute = 2,
+  Text = 3,
+  CdataSection = 4,
+  EntityReference = 5,
+  Entity = 6,
+  ProcessingInstruction = 7,
+  Comment = 8,
+  Document = 9,
+  DocumentType = 10,
+  DocumentFragment = 11,
+  Notation = 12
 }
 
 // This is the subset of DOM used by the appending VM. It is
@@ -31,12 +31,18 @@ export interface Node {
   nextSibling: Option<Node>;
   previousSibling: Option<Node>;
   parentNode: Option<Node>;
-  nodeType: NodeType | number;
+  nodeType: NodeType;
   nodeValue: Option<string>;
   firstChild: Option<Node>;
+  lastChild: Option<Node>;
+}
+
+export interface DocumentFragment extends Node {
+  nodeType: NodeType.DocumentFragment;
 }
 
 export interface Document extends Node {
+  nodeType: NodeType.Document;
   createElement(tag: string): Element;
   createElementNS(namespace: Namespace, tag: string): Element;
   createTextNode(text: string): Text;
@@ -47,15 +53,38 @@ export interface CharacterData extends Node {
   data: string;
 }
 
-export interface Text extends CharacterData {}
+export interface TokenList {
+  [index: number]: string;
+  length: number;
+
+  add(s: string): void;
+  remove(s: string): void;
+  contains(s: string): boolean;
+}
+
+export interface Text extends CharacterData {
+  nodeType: NodeType.Text;
+}
+
+export interface Comment extends CharacterData {
+  nodeType: NodeType.Comment;
+}
 
-export interface Comment extends CharacterData {}
+export interface Attribute {
+  name: string;
+  value: string;
+}
+
+export interface Attributes {
+  [index: number]: Attribute;
+  length: number;
+}
 
 export interface Element extends Node {
+  nodeType: NodeType.Element;
   namespaceURI: Option<string>;
   tagName: string;
-  firstChild: Option<Node>;
-  lastChild: Option<Node>;
+  attributes: Attributes;
   removeAttribute(name: string): void;
   removeAttributeNS(namespaceURI: string, name: string): void;
   setAttribute(name: string, value: string): void;

packages/@glimmer/node/lib/node-dom-helper.ts

@@ -1,5 +1,6 @@
 import * as SimpleDOM from 'simple-dom';
-import { DOMTreeConstruction, Bounds, Simple, ConcreteBounds } from '@glimmer/runtime';
+import { DOMTreeConstruction, Bounds, ConcreteBounds } from '@glimmer/runtime';
+import { Simple } from '@glimmer/interfaces';
 
 export default class NodeDOMTreeConstruction extends DOMTreeConstruction {
   protected document: SimpleDOM.Document;

packages/@glimmer/node/test/node-test.ts

@@ -1,6 +1,7 @@
 import * as SimpleDOM from 'simple-dom';
 import { TestEnvironment, TestDynamicScope } from "@glimmer/test-helpers";
-import { Template, Simple } from '@glimmer/runtime';
+import { Template } from '@glimmer/runtime';
+import { Simple } from '@glimmer/interfaces';
 import { precompile } from '@glimmer/compiler';
 import { UpdatableReference } from '@glimmer/object-reference';
 import { NodeDOMTreeConstruction } from '@glimmer/node';
@@ -38,7 +39,7 @@ function commonSetup() {
 function render<T>(template: Template<T>, self: any) {
   let result;
   env.begin();
-  let templateIterator = template.render(new UpdatableReference(self), root, new TestDynamicScope());
+  let templateIterator = template.render({ self: new UpdatableReference(self), parentNode: root, dynamicScope: new TestDynamicScope() });
 
   do {
     result = templateIterator.next();

packages/@glimmer/runtime/index.ts

@@ -1,6 +1,6 @@
 import './lib/bootstrap';
 
-export { default as templateFactory, TemplateFactory, Template } from './lib/template';
+export { default as templateFactory, TemplateFactory, Template, TemplateIterator, RenderOptions } from './lib/template';
 
 export { NULL_REFERENCE, UNDEFINED_REFERENCE, PrimitiveReference, ConditionalReference } from './lib/references';
 
@@ -25,26 +25,11 @@ export {
   CompiledDynamicProgram
 } from './lib/compiled/blocks';
 
-export {
-  AttributeManager as IAttributeManager,
-  AttributeManager,
-  PropertyManager,
-  INPUT_VALUE_PROPERTY_MANAGER,
-  defaultManagers,
-  defaultAttributeManagers,
-  defaultPropertyManagers,
-  readDOMAttr
-} from './lib/dom/attribute-managers';
-
 export {
   Register,
   debugSlice
 } from './lib/opcodes';
 
-export {
-  normalizeTextValue
-} from './lib/compiled/opcodes/content';
-
 export {
   setDebuggerCallback,
   resetDebuggerCallback,
@@ -72,6 +57,12 @@ export {
 
 export { PublicVM as VM, UpdatingVM, RenderResult, IteratorResult } from './lib/vm';
 
+export {
+  SimpleDynamicAttribute,
+  DynamicAttributeFactory,
+  DynamicAttribute
+} from './lib/vm/attributes/dynamic';
+
 export {
   IArguments as Arguments,
   ICapturedArguments as CapturedArguments,
@@ -81,7 +72,7 @@ export {
   ICapturedNamedArguments as CapturedNamedArguments,
 } from './lib/vm/arguments';
 
-export { SafeString, isSafeString } from './lib/upsert';
+export { SafeString } from './lib/upsert';
 
 export {
   Scope,
@@ -109,8 +100,7 @@ export {
   ModifierManager
 } from './lib/modifier/interfaces';
 
-export { default as DOMChanges, DOMChanges as IDOMChanges, DOMTreeConstruction, isWhitespace, insertHTMLBefore } from './lib/dom/helper';
-import * as Simple from './lib/dom/interfaces';
-export { Simple };
-export { ElementStack, ElementOperations } from './lib/builder';
+export { default as DOMChanges, SVG_NAMESPACE, DOMChanges as IDOMChanges, DOMTreeConstruction, isWhitespace, insertHTMLBefore } from './lib/dom/helper';
+export { normalizeProperty } from './lib/dom/props';
+export { ElementBuilder, NewElementBuilder, ElementOperations } from './lib/vm/element-builder';
 export { default as Bounds, ConcreteBounds } from './lib/bounds';

packages/@glimmer/runtime/lib/bounds.ts

@@ -1,4 +1,4 @@
-import * as Simple from './dom/interfaces';
+import { Simple } from '@glimmer/interfaces';
 import { Option, Destroyable } from '@glimmer/util';
 
 export interface Bounds {
@@ -12,6 +12,16 @@ export class Cursor {
   constructor(public element: Simple.Element, public nextSibling: Option<Simple.Node>) {}
 }
 
+export function currentNode(cursor: Cursor): Option<Simple.Node> {
+  let { element, nextSibling } = cursor;
+
+  if (nextSibling === null) {
+    return element.lastChild;
+  } else {
+    return nextSibling.previousSibling;
+  }
+}
+
 export default Bounds;
 
 export interface DestroyableBounds extends Bounds, Destroyable {}
@@ -46,11 +56,11 @@ export class SingleNodeBounds implements Bounds {
   lastNode() { return this.node; }
 }
 
-export function bounds(parent: Simple.Element, first: Simple.Node, last: Simple.Node): Bounds {
+export function bounds(parent: Simple.Element, first: Simple.Node, last: Simple.Node): ConcreteBounds {
   return new ConcreteBounds(parent, first, last);
 }
 
-export function single(parent: Simple.Element, node: Simple.Node): Bounds {
+export function single(parent: Simple.Element, node: Simple.Node): SingleNodeBounds {
   return new SingleNodeBounds(parent, node);
 }