Part 1 | Part 2 | Part 3 | Part 4 | Part 5 |
Drupal is quickly approaching a critical inflection point in terms of its ability to adapt to and outperform other technologies in the web development space, particularly in the front end. Trends like decoupled Drupal, the subject of a well-received book and an annual conference in New York City, are rapidly gaining adoption in the Drupal community, but such architectural approaches do not resolve the issue of how Drupal's front end can contend with the increasing focus on popular front-end technologies like React and Vue.
Several months ago, Fabian Franz (Senior Technical Architect and Performance Lead at Tag1 Consulting) delivered a well-attended session about how Drupal's front end will need to adapt to the changes currently revolutionizing front-end development. For Fabian, Web Components are the bridge that can reconcile Drupal's front end and back end in order to provide for advantageous features like reactivity and real-time functionality that are essential to Drupal's ability to outcompete other players in the web development market.
In this multi-part blog series, we look into how Fabian's utopian future for the Drupal front end incorporates Web Components, virtual DOMs, and key ideas from other communities in order to furnish a stable future for Drupal for many years to come. In this third installment in the series, we explore the experimental notion of Twig components and how we can take inspiration from JSX and other similar approaches to implement a reactive front-end approach in Drupal.
Twig components
If you have not read the other blog posts in this series, I highly recommend you peruse both the first and second installments, as they contain crucial background about Drupal and JavaScript technologies that lay the foundation for the concepts we discuss in this third installment. Topics that should be familiar to you by this point include how Drupal's render pipeline has evolved until today and how Drupal's render tree is in fact quite similar to the core ideas behind virtual DOMs in JavaScript.
The Twig ecosystem
Are components possible in Twig? This is a question that has been asked frequently as of late not only in the larger Twig community but also in the Drupal community with increasing urgency. In its own right, Twig's ecosystem includes a variety of useful projects, including a server-based Twig.js library, a client-based Twig.js library, and Twing, a newer project that is responsible for running tests.
One of these projects, server-side Twig.js, provides PHP code on the server that can compile to JavaScript, and that JavaScript result is sent to the JavaScript client, which can then use the rendered data. As for Twing, it has a huge advantage in that it passes an entire Twig test suite and is therefore one of the most stable technologies available in the Twig ecosystem.
Rationalizing Twig components
For pure components, Twig can function perfectly normally, but because of Drupal's unique customizations of Twig, core Drupal Twig templates will encounter significant difficulties in enabling the componentized approach that Fabian envisions. After all, front-end responsibilities do not revolve solely around rendering, as developers must also rely on features that provide reactivity and virtual DOM diffing.
For instance, reactivity means that users can click on some portion of the web page and witness an immediate result that catalyzes a change somewhere else. One of the key questions Fabian asks in his presentation is whether we truly need to reinvent the wheel and create our own library for Drupal that allows for reactivity to take place in Drupal's own front-end user interfaces. His conclusion, as we will see shortly, is that there is a better and simpler way to tackle the problem.
A deep dive into virtual DOMs
Fabian notes that several years ago, he attempted to solve this very problem of reactivity in Drupal. Unfortunately, he admits that he failed at the time due to the fact that Twig renders to a string. At the time, he states that he felt this was a showstopper that would prevent any further advancements in Drupal's ability to provide reactivity. However, with the proliferation of React and Vue (both of which are covered at length in Decoupled Drupal in Practice), which render not to a string but to a virtual DOM (document object model), Fabian witnessed how such an approach could furnish an excellent experience for developers and users alike.
Virtual DOMs in React and Vue
In the virtual DOM that React and Vue implement, only what changes in the DOM is rerendered. This is highly practical because it allows for these technologies not to have to rerender all of the elements in a tree, therefore saving considerable time. Drupal's analogue to the virtual DOM, the replaceElement() function in Drupal's Ajax API, is ill-suited for this sort of approach, because it would replace other elements that do not in fact require it, thus negatively impacting performance.
In JSX, every instance of declarative rendering is in fact transpiled to a use of the createElement() function, as we can see below:
createElement('span', createTextElement('Hello world!');
To accomplish this, JSX creates an internal tree. This internal tree is one of the primary reasons, Fabian argues, why Angular and other frameworks utilize syntax that requires the addition of directives directly to DOM nodes with properties. Moreover, many of these frameworks only allow placeholders within properties, because fundamentally they are DOM languages:
<li v-for="item in items">
{{ item.message }}
</li>
Twig is ill-suited for virtual DOMs
Twig, on the other hand, is not a DOM language; it can output JavaScript code, CSS code, or SQL code arbitrarily, depending on what is required of it. As a result, Twig hinders our ability to provide a similar approach in Drupal, because Twig and virtual DOMs are concepts that are fundamentally opposed to one another, according to Fabian. As such, it is extremely difficult to transpile Twig output into a virtual DOM. Consider, for example, that the following is valid in Twig:
{% set tag = "<span" %}
{% set endtag = "</span" %}
{{ tag }}>Hello world!{{ endtag }}
You can try this example yourself and see that it works perfectly in Twig and renders valid HTML, but this is clearly impossible to transpile into a virtual DOM.
Lessons from Vue and tradeoffs in Twig
This brings us to some of the fundamental tradeoffs of Twig: In short, we have to perform parsing of the document object model (DOM), and we also need a component tree, because we need to know how to rerender the entire tree. However, the JavaScript projects that we have been examining up to this point can also offer insights in this regard.
For example, Vue, in addition to its virtual DOM, also has an inline template parser such that any HTML can be converted into a virtual DOM at runtime, which could help us solve the aforementioned problem. In addition, Vue does not care whether trees are first transformed into a string and then transformed into a virtual DOM later. It only needs the virtual DOM to discover the components contained within and does not depend on it to render the ultimate output.
As Fabian states, we can adopt a similar approach to this using Twig. With this exact same DOM parsing approach, we can create a Twig template that is aware of its context and convert that to a string of HTML and components, generating a virtual DOM in the process. Because the result is pure HTML, we can now input this into Vue, admittedly with a performance penalty, but this process could be more than adequate for the 95% case.
Given the choice between writing every template twice, with one for the server and one for the client, Fabian confesses, and only writing 5% of the templates that are most critical for our use cases, we can be confident that generating such output through Twig would probably be much faster than rendering Ajax returned by the Drupal server. And Fabian admits that he would readily accept this tradeoff in order to be able to benefit from the aforementioned 95% case.
Conclusion
In this third installment of our multi-part blog series about how Drupal's front end could improve thanks to innovative ideas borrowed from the JavaScript community, we dove into some of the most important takeaways from Fabian's examination of how the virtual DOM functions in React and Vue as well as how Twig challenges our desired approach to providing a virtual DOM in Drupal.
In the next installment of this multi-part blog series, we'll take a look at what Drupal needs for the ecosystem to realistically support this novel approach from Fabian's point of view as well as how Drupal can perform graceful server-side rendering with integration points for further client-side rendering thanks to Web Components and visionary concepts from the JavaScript landscape.
Special thanks to Fabian Franz and Michael Meyers for their feedback during the writing process.
Part 1 | Part 2 | Part 3 | Part 4 | Part 5 |
For more on Web Components and how they're being used, see Web Components.
Photo by Sam Goodgame on Unsplash