{"id":6110,"date":"2022-01-24T20:55:05","date_gmt":"2022-01-24T20:55:05","guid":{"rendered":"https:\/\/mailinvest.blog\/index.php\/2022\/01\/24\/evolving-core-web-vitals-tactics-using-cloudflare-and-webpagetest\/"},"modified":"2022-01-24T20:55:05","modified_gmt":"2022-01-24T20:55:05","slug":"evolving-core-web-vitals-tactics-using-cloudflare-and-webpagetest","status":"publish","type":"post","link":"https:\/\/mailinvest.blog\/index.php\/2022\/01\/24\/evolving-core-web-vitals-tactics-using-cloudflare-and-webpagetest\/","title":{"rendered":"Evolving Core Web Vitals tactics using Cloudflare and WebpageTest"},"content":{"rendered":"

<\/a>\n<\/p>\n

\n

In our guide to Core Web Vitals tactics using Cloudflare and WebpageTest<\/a>, we outlined basic requirements for using Cloudflare as a reverse proxy for testing tactical HTML changes with WebpageTest. Our version of the test is simplified from Patrick Meenan\u2019s original concept, which uses HTMLRewriter()<\/a><\/code> to select an element and modify code.<\/p>\n

We\u2019re going in-depth with this tutorial, but if you\u2019re just looking for the Cloudflare Worker script, you can find it here<\/a>.<\/p>\n

Our first installment noted that it won\u2019t keep up with changes at Search Engine Land. The LCP was hard-coded and we would need it to interact with a dynamic page and its values. While WebpageTest has, at the time of publication, the most well-thought-out waterfall chart and more details than you can imagine, it isn\u2019t the fastest way to get results.<\/p>\n

Lighthouse from the Command Line<\/h2>\n

Running the Lighthouse CLI (Command Line Interpreter) program with --extra-headers<\/code> options needed for the test allows us to also simulate standard settings for Core Web Vitals the way we did with WebpageTest. You\u2019ll need to work from a terminal emulator. <\/p>\n

The easiest way to install Lighthouse is with NPM (Node Package Manager). Once installed, run the following statement:<\/p>\n

$ lighthouse https:\/\/sel.deckart.workers.dev \\<\/code>
--extra-headers \"{\\\"x-host\\\":\\\"searchengineland.com\\\", \\\"x-bypass-transform\\\":\\\"false\\\"}\" \\<\/code>
--form-factor=mobile \\<\/code>
--throttling.cpuSlowdownMultiplier=4 \\<\/code>
--only-categories=performance \\<\/code>
--view<\/code><\/p>\n

The evolution of our Testbed<\/h2>\n

Our aim is to demonstrate an evolution from an original concept for a testbed to a project suitable for our future events and articles. The testbed should not be confined to running performance evaluations; that\u2019s just where we\u2019ll start. But, it has to work fairly well for a number of situations with websites and this can prove pretty difficult. We\u2019ll supply methods to help.<\/p>\n

For example, sites often use relative paths to asset resources rather than absolute (with HTTP protocol and all). We\u2019ll supply a block to match these so HTML will generally work. After applying this, when things still don\u2019t work, switching troublesome references between the test and test subject hostnames often does the trick, even for CORS policy violations.<\/p>\n

That\u2019s where the beauty of Cloudflare\u2019s HTMLRewriter()<\/code> really shines. Site-wide assets are usually loaded as page HEAD child elements. With flexibility matching like jQuery, even similar syntax, we can select child elements of HEAD when necessary. You can use XPath selectors and regular expressions. Let\u2019s keep it simple and look for relative paths that start with \u201c\/\u201d for src<\/code> or href<\/code> attributes:<\/p>\n

return new HTMLRewriter()\n  .on('link', {\n    element: el => {\n      const link_href = el.getAttribute('href');\n      if (link_href && link_href.startsWith('\/')) {\n        el.setAttribute('href', 'https:\/\/' + host + link_href);\n      }\n    }\n  })\n  .on('script', {\n    element: el => {\n      const script_src = el.getAttribute('src');\n      if (script_src && script_src.startsWith('\/')) {\n        el.setAttribute('src', 'https:\/\/' + host + script_src);\n      }\n    }\n  })\n  .on('img', {\n    element: el => {\n      const img_src = el.getAttribute('src');\n      if (img_src && img_src.startsWith('\/')) {\n        el.setAttribute('src', 'https:\/\/' + host + img_src);\n      }\n    }\n  })<\/code><\/pre>\n
We\u2019re leveraging the power (and cost effectiveness) of Edge Computing to conduct seriously useful tests. Modify the x-host<\/code> request header to load different sites in the testbed and open DevTools. Transformations may not be needed, but your mileage will vary. Frontend experience gives you a feel for it.<\/p>\n
Comment blocks like switches will fail and require a little experimentation (which may be all you need). For example, some asset references may be spelled without HTTP colon. You would need to write another conditional to check for paths where href<\/code> or src<\/code> starts with \u201c\/\/\u201d and then modify the selected element value in the script. Try to end up with no console errors the actual site doesn\u2019t have.<\/p>\n
Lighthouse gives you LCP<\/h2>\n
It\u2019s relatively easy to retrieve LCP references using Lighthouse, PageSpeed Insights or WebpageTest. Presuming the LCP qualifies for preload, like when it\u2019s not a <div><\/code> or a <p><\/code>, and when it isn\u2019t already getting preloaded, provide our script the href<\/code> value by URL \u2018query param\u2019 structure (or return HTML with a form) to test for changes to a page\u2019s LCP timing with preload.<\/p>\n
Most technical SEO practitioners are handy at modifying request query parameters to process different things in server-side programs, like Google search results. Using the same interface, our script will preload the LCP using the path you apply in the \u201clcp\u201d parameter value and passes it to a function called addPreloadAfter()<\/code> for interpolating HTML for the test.<\/p>\n
async function handleRequest(request) {\n  const { searchParams } = new URL(request.url);\n  let lcpHref = searchParams.get(\"lcp\");\n\n  return new HTMLRewriter()\n    .on('title', addPreloadAfter(lcpHref))\n  .transform(newResponse);\n}<\/code><\/pre>\n
The addPreloadAfter()<\/code> function takes our \u201clcpHref\u201d value from searchParams.get()<\/code> and processes it as \u201chref\u201d to build HTML.<\/p>\n
const addPreloadAfter = (href) => ({\n  element: (el) => {\n    el.after(`<link rel=\"preload\" href=\"https:\/\/searchengineland.com\/${href}\" \/>`, { html: true });\n  }\n});<\/code><\/pre>\n
Notice the option \u201chtml: true\u201d? This is an option setting Cloudflare requires for safety when using Workers with HTMLRewriter()<\/code> API methods<\/a> that write HTML. You are going to want to learn its capabilities and constraints for coding your own tests.<\/p>\n
Cloudflare\u2019s KV<\/h2>\n
If we\u2019re ever going to do anything remotely interesting, we need a way to store persistent data between script executions. Luckily, Cloudflare also offers a neat little data storage mechanism called KV<\/a> that we can bind with our Workers to store a small data \u2018value<\/code>\u2018 field, accessible by its \u2018key<\/code>.\u2019 It\u2019s surprisingly easy to comprehend and implement. To demonstrate how to use it we\u2019ll write a quick little hit counter.<\/p>\n
const counter = parseInt(await KV.get('counter') || 0);\n\nif (!host || counter > 1000) {\n  return new Response('hit limit exceeded or x-host missing', {status: 403});\n} else {\n  await KV.put(\"counter\", counter + 1);\n}<\/code><\/pre>\n
Find the KV navigation menu item under Workers.<\/p>\n
\n
\"The