feat!: implement more expressive templates via liquidjs - #428
Open
pausan wants to merge 2 commits into
Open
Conversation
Templates are parsed once and rendered per recipient, so a basic
template can render at 140k renders/s whereas a richer one with
sevearl conditionals and loops renders at ~18k renders/s. 1M-contact
campaign spends under one minute of CPU time for rendering, which is
negligible against 1M SES calls.
The engine is sandboxed: {% include %}, {% render %} and {% layout %}
are rejected because they resolve partials off the file system, and
parse/render/memory are capped so a runaway loop can't stall a worker.
Rendering stays lenient, a template that fails to parse falls back to
plain {{variable}} substitution rather than failing the send. Syntax
errors are surfaced at write time instead: POST/PATCH of templates and
campaigns, and POST /v1/send with an inline subject/body, now return
400 with the line and column.
Adding multiple tests to assess the proper behaviour and to keep
some performance guarantees (for both memory & CPU).
Backwards compatible with existing template system for most scenarios.
See template-language.mdx Notes for existing templates for more info,
but basically: 0/false/NaN are now rendered while before they produced
empty string. Also {{ name ?? 'hi' }} produced 'hi' with quotes, which
seems like a bug, now emits hi with no quotes. Body copy containing
balanced {% %} markup is now executed rather than printed literally —
wrap it in {% raw %} to keep it as text.
verifyDomain and getDomainVerificationAttributes were already mocked, but deleteIdentity and disableFeedbackForwarding were not, so the suite issued live AWS SES requests on every run. DomainService catches and logs failures from both, so the calls stayed invisible in test output while still going over the network. With real SES credentials in .env they would have executed DeleteIdentity and SetIdentityFeedbackForwardingEnabled against a live AWS account. DomainService.test.ts drops from ~40s to 23s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implement liquidjs templating system as per #426
Tests and documentation have been updated. I manually ran tests locally.
Templates are parsed once and rendered per recipient, so a basic template can render at 140k renders/s whereas a much richer template with several conditionals and loops renders at ~18k renders/s (worse-case scenario).
Even on a 1M-contact campaign, it would take less than 1 minute of CPU time for rendering, which is still negligible against 1M SES calls. I mean, worse-case scenario less than a minute for 1M emails, whereas best-case scenario less than 8 seconds (note: tested on an AMD Ryzen AI 9 365).
The engine is sandboxed: {% include %}, {% render %} and {% layout %} are rejected because they resolve partials off the file system, and parse/render/memory are capped so a runaway loop can't stall a worker.
Rendering stays lenient, a template that fails to parse falls back to plain {{variable}} substitution rather than failing the send. Syntax errors are surfaced at write time instead: POST/PATCH of templates and
campaigns, and POST /v1/send with an inline subject/body, now return 400 with the line and column.
Adding multiple tests to assess the proper behaviour and to keep some performance guarantees (for both memory & CPU).
Backwards compatible with existing template system for most scenarios.
See template-language.mdx Notes for existing templates for more info (in the commited files), but basically:
I think is very unlikely these create issues with existing templates.
Type of Change
Marking as breaking change since it is not 100% backwards compatible, even though my personal guess is that it might stay compatible with 99% campaigns or even higher, but I have no data to back this up, thus, breaking change it is.
feat:New feature (MINOR version bump)fix:Bug fix (PATCH version bump)feat!:Breaking change - new feature (MAJOR version bump)fix!:Breaking change - bug fix (MAJOR version bump)docs:Documentation update (no version bump)chore:Maintenance/dependencies (no version bump)refactor:Code refactoring (no version bump)test:Adding tests (no version bump)perf:Performance improvement (PATCH version bump)PR Title Format
Testing
I created new automated tests both for template system itself and also for campaign service but also performance test to assess memory and CPU performance stays within reasonable boundaries.
The important caveat here is that I have not tested this in a live environment, given the change, it's totally worth to try it.
Most scenarios for the use of variables should be backwards compatible. See notes.
Checklist
Related Issues
Closes #426