Salesforce, Python, SQL, & other ways to put your data where you need it

Need event music? 🎸

Live and recorded jazz, pop, and meditative music for your virtual conference / Zoom wedding / yoga class / private party with quality sound and a smooth technical experience

Frontend automated testing demystifier

07 Jul 2025 🔖 web development
💬 EN

Table of Contents

When it comes to authoring automated test code for web application frontends, it’s easy to get confused about what kinds of test go where in the “testing pyramid” and how to implement them – especially since all four levels of the “testing pyramid” potentially ask the question: “does this HTML/DOM element exist in this condition?”.

Here’re some examples, using a “book review” webapp (with an Angular frontend) where users can:

  1. view a list of books
  2. add a review to a book
  3. see each book’s average ratings


Unit testing

Principles

  • Purpose: pinpoint bugs inside a specific method, class, utility, helper, service, component, etc.
  • Mock dependencies, including child components, to isolate concerns and make it so that refactoring doesn’t mean a nightmare of writing unit tests. They have their own tests; presume in tests that “call” them that they “just work”! See the “examples” section below for clarification. You’ll want to mock:
    • return values for all methods called by the thing being tested, rather than letting them actually execute
      • only write test cases for whatever is unique to the thing currently being tested; don’t bother re-testing things that are part of the mocked dependency’s logic
      • (that said, confirming that the mocked method was attempted to be executed is something to test at the level of “the thing currently being tested”)
    • UI/DOM rendering for all child @Components included within the template of a @Component being tested, rather than letting the children actually render
      • only write test cases for whatever is unique to the parent currently being tested; don’t bother re-testing things that are part of the mocked child’s logic or template
      • (that said, confirming that the mocked rendering is included in the parent’s rendering is something to test at the parent level currently being tested)
  • Speed: each test should run very fast (hence using Karma instead of Playwright for @Component testing, though Playwright might eventually support testing individual Angular components)
  • Quantity: there should be a lot of these, if you’re being thorough. Maybe dozens of expect() calls per .spec.ts file.
  • Tip: If you’re using an LLM coding assistant to help you write unit tests, please make sure it understands that you want it to:
    • be very strict about mocking
    • be very thorough about inventing suites of test cases that hit all possible boundaries of potential inputs, nulls, misformed data, exceptions, etc.

Technologies

NPM modules used

  1. @angular/core/testing
  2. Jasmine (provides the describe(), it(), expect(), etc. syntax)
  3. Karma (if @Component-decorated code is being tested)

How it’s run

  • via: CI/CD pipeline
  • when: before npx ng build
  • command: npx ng test
  • tip: remember to have your CI/CD pipeline write out the results to your QA test-run tracking tool of choice

Examples


ReviewService: calculate average rating method

Does the calculateAverageRating() method in the ReviewService (/src/app/features/review/review.service.spec.ts) return the correct average when given a list of ratings?

File path

/src/app/features/review/review.service.spec.ts (alongside its corresponding .service.ts file)

Dependencies mocked

Literally everything that calculateAverageRating() calls.

For example:

  • If calculateAverageRating() calls getRatingsFromApi({ bookid: '12345'}) function, then:
  • Mock out getRatingsFromApi() to always return a fake answer of, say, [{ "user": "aaa", "rating": 5}, { "user": "bbb", "rating": 3}], that’s shaped the same way as data returned by your API.

(And then have the RatingService’s unit tests validate that calculateAverageRating() returns 4, given that fake API response.)

What it tests
  1. When book has no ratings
    • Fake getRatingsFromAPI() response returns [] (empty array)
    • Expected: returns null or 0 (depending on your business logic for when the book has no ratings; clarify with your team / QAs)
  2. When book has only 1 rating
    • Fake getRatingsFromAPI() response returns [{ "user": "aaa", "rating": 5}]
    • Expected: returns 5
  3. When book has multiple ratings
    • Fake getRatingsFromAPI() response returns [{ "user": "aaa", "rating": 5}, { "user": "bbb", "rating": 3}]
    • Expected: returns correct average: 4
  4. When API returns null
    • Fake getRatingsFromAPI() response returns null
    • Expected: returns null or 0, or throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  5. When API returns undefined
    • Fake getRatingsFromAPI() response returns undefined
    • Expected: returns null or 0, or throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  6. When API returns nonsense
    • Fake getRatingsFromAPI() response returns "hello world"
    • Expected: returns null or 0, or throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  7. When ratings array contains negative numbers
    • Fake getRatingsFromAPI() response returns [{ "user": "aaa", "rating": 5}, { "user": "bbb", "rating": -1}]
    • Expected: returns correct average: 2, or throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  8. When calculation returns something besides an integer
    • Fake getRatingsFromAPI() response returns [{ "user": "aaa", "rating": 5}, { "user": "bbb", "rating": 4}]
    • Expected: returns correct average: 4.5, or rounds up to 5 (depending on your business logic; clarify with your team / QAs)
    • Tip: Jasmine includes a .toBeCloseTo() function that’s great for numbers like “six and two thirds” that aren’t easy to type out in a .toBe() call. e.g.
        it('should handle non-integer calculations', () => {
            expect(
                service.calculateAverageRating(
                    { bookid: '12345'}
                )
            ).toBeCloseTo(6.67, 2);
        });
      
  9. When ratings array is incredibly long
    • Fake getRatingsFromAPI() response returns [{...rating #1...}, {...rating #2...}, {...rating #5,000,000...}]
    • Expected:
      • returns correct average: 2, and
      • calculateAverageRating() does not do any sort of “overflow,” and
      • calculateAverageRating() executes performantly (especially if the function could be called frequently)

BookRatingComponent: a book’s rating, wrapped in an HTML div tag

Does the BookRating component (/src/app/features/book/rating/bookrating.component.spec.ts) (selector: app-book-rating) return correct HTML elements in its rendered DOM, given the inputs passed to it?

File path

/src/app/features/book/rating/bookrating.component.spec.ts (alongside its corresponding .component.ts file)

Dependencies mocked

Literally everything that the @Component decorator on BookRating component results in being called.

For example:

  • If the BookRating component calls rvwSvc.calculateAverageRating({ bookid: this.bookid }), then:
  • Mock out the ReviewService’s calculateAverageRating() to always return a fake answer of, say, 3.

(And then have the BookRating component’s unit tests validate that the rendered component’s DOM contains <div data-testid="average-rating">Average Rating: 3</div>, given that fake calculateAverageRating() response.)

What it tests
  1. When average rating is null
    • Fake calculateAverageRating() response returns null
    • Expected: div content is No ratings yet, or component throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  2. When average rating is 0
    • Fake calculateAverageRating() response returns 0
    • Expected: div content is No ratings yet, or component throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  3. When the calculation throws an error
    • Fake calculateAverageRating() throws an error
    • Expected: div content is No ratings yet, or component throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  4. When average rating is a negative number
    • Fake calculateAverageRating() response returns -2
    • Expected: div content is No ratings yet, or component throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  5. When average rating is not even parseable into a number
    • Fake calculateAverageRating() response returns "hello world"
    • Expected: div content is No ratings yet, or component throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  6. When average rating is not a number, but is easily parseable into one
    • Fake calculateAverageRating() response returns "2" (a plaintext string saying “2,” instead of the number 2)
    • Expected: div content is Average Rating: 2, or component throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  7. When average rating is an integer
    • Fake calculateAverageRating() response returns 4
    • Expected: div content is Average Rating: 4
  8. When average rating is a non-integer
    • Fake calculateAverageRating() response returns 1.5
    • Expected: div content is Average Rating: 4.5, or content is rounded off to Average Rating: 5 (depending on your business logic; clarify with your team / QAs)
  9. When a change to the component’s input is detected
    • At first, a book ID is passed into the component’s inputs that results in a fake calculateAverageRating() response returning 4.
    • Then, the component’s ngOnChanges(), picking up a different book ID being passed into the component’s inputs instead, causes a fake calculateAverageRating() response to return again, this time with a value of 2.
    • Expected:
      • At first, div content is Average Rating: 4
      • After the component’s .ngOnChanges() fires, the div content is Average Rating: 2
    • (Note: I know this isn’t at all realistic – who would change which book we’re talking about in a real UI, dynamically? But I thought it was important to mention the concept, in general, of being sure not to forget about testing “before” & “after” when the component has ngOnChanges() behavior.)
  10. When the book ID passed to the BookRating component triggers `calculate
    • Tip: Fake calculateAverageRating() response returns 1.5
    • Expected: div content is Average Rating: 4.5, or content is rounded off to Average Rating: 5 (depending on your business logic; clarify with your team / QAs)

BookRatingComponent: a book’s rating, wrapped in an HTML div tag

Does the BookDetailPage component (/src/app/features/book/detailpage/bookdetailpage.component.spec.ts) (selector: app-book-detail) return correct HTML elements in its rendered DOM, given the inputs passed to it?

File path

/src/app/features/book/detailpage/bookdetailpage.component.spec.ts (alongside its corresponding .component.ts file)

Dependencies mocked

Literally everything that the @Component decorator on BookDetailPageComponent results in being called.

For example:

  • If the BookDetailPage component’s template includes an <app-book-rating/> element, then:
  • Mock out the BookRating to always return a fake rendering of, say:
      <div data-testid="book-rating">
          <div data-testid="average-rating">Average Rating: 3</div>
      </div>
    

(And then have the BookDetailPage component’s unit tests validate that the rendered component’s DOM contains a <div/> whose data-testid is book-rating, but don’t worry about what exactly is within that – no need to re-test for Average Rating: 3.)

What it tests
  1. When there is a title
    • Fake (mock out however BookDetailPage gets the book’s title) returns a valid response (whether that’s a string or whether we’re mocking out a child component – it depends on your implementation)
    • Expected: BookDetailPage component contains a <h1 data-testid="book-title">FAKE TITLE GOES HERE</h1> element
  2. When something goes wrong trying to get the title
    • Fake (mock out however BookDetailPage gets the book’s title) returns a bad response (whether that’s raw data or whether we’re mocking out a child component – it depends on your implementation – and actually, you may want to have several variants on bad responses as several test cases)
    • Expected: Unit test deliberately fails (or BookDetailPage throws a well-formatted error), because it is absolutely business-critical that every book detail page show the book’s title
  3. When there is a snopsis
    • Fake (mock out however BookDetailPage gets the book’s synopsis) returns a valid response (whether that’s a string or whether we’re mocking out a child component – it depends on your implementation)
    • Expected: BookDetailPage component contains a non-empty <article data-testid="book-synopsis">FAKE SYNOPSIS GOES HERE</article> element
  4. When something goes wrong with trying to get the synopsis
    • Fake (mock out however BookDetailPage gets the book’s synopsis) returns a bad response (whether that’s raw data or whether we’re mocking out a child component – it depends on your implementation – and actually, you may want to have several variants on bad responses as several test cases)
    • Expected: Placeholder “synopsis coming soon” text appears, or unit test deliberately fails, or BookDetailPage throws a well-formatted error (depending on your business logic; clarify with your team / QAs)
  5. When the synopsis includes an insanely long word
    • Fake (mock out however BookDetailPage gets the book’s synopsis) returns a plaintext string containing a word that is, say, 300 characters long (think “supercalifragilisticexpialidocious” but even longer).
    • Expected: The <article data-testid="book-synopsis"/> element does not overflow its allocated space (e.g. text wraps with auto-hyphenation or shrinks gracefully), even in a small allocated space (e.g. a small screen), and the “graceful” handling (e.g. shrinking the text) does not interfere with accessibility (e.g. making it too small to read anymore, especially on a small screen or for humans with vision challenges).
  6. When the BookRating child component returns a well-formatted HTML/DOM response
    • Fake BookRating rendering returns something short and sweet – e.g. an empty <div></div>.
    • Expected: BookDetailPage component’s rendering contains an <app-book-rating> element (that is, an @angular/core/testing ComponentFixture that meets an expect() of .toBeTruthy()), which is sandwiched between:
      • the <h1 data-testid="book-title"/> element, and
      • the <article data-testid="book-synopsis"/> element
    • Tip:
      • Yes, you still need to have a test case that verifies the inclusion/presence of the child element on the page
      • No, you don’t need to validate anything about the mocked child element’s details.
  7. When the BookRating child component throws an error
    • Fake BookRating rendering throws an error
    • Expected: BookDetailPage component includes nothing between the book title element and the book synopsis element, or component throws a well-formatted error (depending on your business logic; clarify with your team / QAs)


Integration testing

Principles

  • Purpose: pinpoint bugs that might only arise based on the way multiple units work with each other.
  • Mock: dependencies and child components as thoroughly as you would for unit testing. Only un-mock anything, bit by bit, as necessary to actually implement a meaningful test case. You might be surprised how few un-mocks you’ll do.
  • Speed: each test should run moderately fast, but will probably run slower t han unit tests (hence using Karma instead of Playwright for @Component testing, though Playwright might eventually support testing individual Angular components)
  • Quantity: there should still be quite a few these, if you’re being thorough. Maybe dozens of expect() calls per .spec.ts file.
  • Tip: If you’re using an LLM coding assistant to help you write integration tests, please make sure it understands that you want it to:
    • be very strict about mocking (yes, even though this is an integration test)
    • be very thorough about inventing suites of test cases that hit all possible boundaries of potential inputs, nulls, misformed data, exceptions, etc.

Technologies

NPM modules used

  1. @angular/core/testing
  2. Jasmine (provides the describe(), it(), expect(), etc. syntax)
  3. Karma (if @Component-decorated code is being tested)
  4. @rxjs (if interactions between child @Component-decorated code are being tested within a parent component)

How it’s run

  • via: CI/CD pipeline
  • when: before npx ng build
  • command: npx ng test
  • tip: remember to have your CI/CD pipeline write out the results to your QA test-run tracking tool of choice

Examples


BookReviewSubmission and BookRating via BookDetailPage

Imagine a BookDetailPage component whose template includes element that invoke the rendering of:

  1. a BookRating child component up top (that dynamically displays the book’s latest average rating)
  2. a BookReviewSubmission child component down bottom (that includes a submit button for your opinion about the book)

Given realistic mocked data simulating interactions between the child components, does BookDetailPage coordinate its child components correctly?

File path

There are a number of great options:

  • /src/app/features/book/detailpage/bookdetailpage.component.spec.ts (same as before, just add more tests – perhaps in a describe())
  • /src/app/features/book/detailpage/bookdetailpage.integrationtest.ts
  • /src/app/features/book/detailpage/updaterating.integrationtest.ts
  • /src/app/features/book/updaterating.integrationtest.ts (perhaps does a fleet of similar-to-each-other tests with a lot of different components that have BookReviewSubmission and BookRating child components on the same page)
  • /tests/integration/features/detailpage/bookdetailpage.component.spec.ts
  • /tests/integration/features/detailpage/updaterating.component.spec.ts
  • /tests/integration/features/book/updaterating.integrationtest.ts

Clarify with your team to come up with a file-naming pattern that works for you.

If you’re using an LLM coding assistant, be sure to make sure it knows what your team came up with, too.

Dependencies mocked

TODO

What it tests
  1. When the mocked BookReviewSubmission child component emits (@angular/core’s EventEmitter’s .emit() method)
    • Fake BookRating child component renderings return different responses at different times, using the Jasmine spy object’s .and.returnValues() method:
      • <div>BEFORE_TESTING_HELLO_ABC_123</div> before the review submission event.
      • <div>AFTER_TESTING_WORLD_ZYX_987</div> after the review submission event.
    • Tip: note that the mock BookRating child component shouldn’t even need to mock out a call to rvwSvc.calculateAverageRating({ bookid: this.bookid }).
      • It just returns a rendered value that, quite honestly, has absolutely nothing to do with what the real BookRating’s rendered template would look like, but that’s okay.
      • All we care about right now is just making sure that “mocked rendering #1” shows up before the review submission event, and that “mocked rendering #2” shows up after the review submission event.
      • We don’t actually need to validate that computing averages works right inside of BookRating; we did that when we wrote thorough unit tests for the BookRating component itself.
      • For this integration test, it’s best if we make it crystal clear to our future selves, and to our colleagues, that we’re just black-box testing the nuances of the interactions amongst various components boundaries and input/output contracts.
      • It’s actually better for maintainability if we make the mocked BookRating child component return utter nonsense, rather than letting it try to do something that looks like it has anything to do with average ratings.
      • That way, we can feel confident that we won’t need to refactor this test, if the inner workings of the real BookRating or BookReviewSubmission ever changed. As long as the real BookReviewSubmission still does an .emit(), and as long as the real BookRating still has a .refresh() method, this integration test can stay exactly as it is, without having to be rewritten.
    • Expected:
      • Before the review submission event:
        • The mocked BookRating child component’s .refresh() method has been called 0 time (Jasmine’s .toHaveBeenCalledTimes(0)).
        • The BookDetailPage component’s rendering includes the phrase BEFORE_TESTING_HELLO_ABC_123.
        • The BookDetailPage component’s rendering does not include the phrase AFTER_TESTING_WORLD_ZYX_987.
      • After the review submission event:
        • The mocked BookRating child component’s .refresh() method has been called 1 time (Jasmine’s .toHaveBeenCalledTimes(1)).
        • The BookDetailPage component’s rendering includes the phrase AFTER_TESTING_WORLD_ZYX_987.
        • The BookDetailPage component’s rendering does not include the phrase BEFORE_TESTING_HELLO_ABC_123.


End-to-end testing

Commmon abbreviation for “end-to-end”: “E2E.”

Principles

  • Purpose: simulate real user flows through the whole webapp, as a user would experience it; catch issues missed by lower-level tests (though once caught, adding more tests to the lower-level tests to catch the bug is a great idea)
  • Mock: if login is involved, be sure to create and use a dedicated fake user account that does not have the ability to do any meaningful damage, even if someone malicious were to log into your webapp as that fake user account
  • Speed: think about designing your test cases for high parallelism, because each test will run a lot slower than your other tests
    • Low coupling: even for a single real-world “flow,” are there ways you could break it up into smaller parts that would still prove that things “work” just as effectively? (Clarify with your team / QAs.)
  • Quantity: you’ll have a lot of these if you haven’t gotten around to breaking them up into unit and integration tests; you won’t have all that many of them if you have.
  • Tip: If you’re using an LLM coding assistant to help you write end-to-end tests, please remember to feed it business-level artifacts such as:
    • your QAs’ manual testing test cases
    • your product’s FAQ and help docs aimed at users
    • the initial project’s requirements docs
    • requirements / reproduce steps for the feature or bugfix you’re working on
    • etc.

Technologies

NPM modules used

  1. @playwright/test & Playwright, for example (be sure to set it to run against a variety of browsers and screen sizes)
  2. often, some sort of 3rd-party service capable of running tests, in parallel, written in your testing framework of choice, against fleets of different kinds of devices (e.g. desktops, phones, tablets) – but for small test suites, the operating system of the CI/CD runtime might suffice

How it’s run

  • via: CI/CD pipeline
  • when:
    • either before npx ng build if you can run it effectively against the CI/CD runtimes’s localhost, or
    • after the CI/CD pipeline does the deploy step; point the baseUrl for the Playwright tests at the URL that the built frontend was just deployed to
  • command: npx playwright test, for example
  • tip: remember to have your CI/CD pipeline write out the results to your QA test-run tracking tool of choice

Examples


Log in, add a review to a book, and see the updated average rating on the book’s page

File path
  • /tests/e2e/user-flows/add-review.e2e.ts

Clarify with your team to come up with a file-naming pattern that works for you.

If you’re using an LLM coding assistant, be sure to make sure it knows what your team came up with, too.

Use fake accounts

You probably won’t be “mocking dependencies,” per se, when writing E2E tests.

However, if login is involved, be sure to create and use a dedicated fake user account that does not have the ability to do any meaningful damage, even if someone malicious were to log into your webapp as that fake user account.

If you can’t do that, you need to refactor your application (perhaps even in the database or backend) so that it can support working meaningfully when a logged-in user doesn’t have adequate permissions to do anything important.

Particularly if you’re testing against any sort of long-lived system (rather than one that will cease to exist when the CI/CD pipeline stops running), you should probably play it safe and architect your entire workload so that if a test credential were to be leaked on Pastebin, it wouldn’t matter (other than that maybe you’d be having some spam logins to deal with high volume of).

Presume your test accounts are going to be taken over by hackers and limit their “blast zone” accordingly.

(Hey! Here’s a great idea! Include end-to-end tests that make sure you get errors if you try to do things you thought you locked test accounts out of being able to do! 😅)

What it tests
  1. That login works
  2. That the /books URL displays a list of books
  3. That a given book’s detail page (perhaps navigated to by clicking a book in the aforementioned list) – let’s call it the /books/12345 URL – displays details about the book
  4. That:
    • Capturing the app-book-rating element’s average-rating sub-element’s text content into a variable for later comparison,
    • Filling in the rating input in the app-book-review-submission element with a value that is wildly different than the current average rating,
    • Capturing the app-book-rating element’s average-rating sub-element’s new text content into another variable, and
    • Comparing the two variable values…
    • …results in the two values being different (note: this might not work so well if you’re testing against a long-lived system where there are so many ratings that it’s hard to budge the average – see tip below).
    • …also results in there being yet another review in in the list, if you also submitted a textual review, not just a rating, earlier on when submitting your review.
  5. etc.

Note: some of these are is a destructive test if you are running them against a real, long-lived system.

You could end up with a lot of junk data that isn’t actually what any real people think about a book. Yikes!

This is often a problem with end-to-end testing.

Again, please design your application around supporting test IDs if you’re going to have to test against real, long-lived systems.

For example, maybe you leave a book called “Just Testing – the art of end-to-end testing – qwertyuiop1234567890” sitting around in your database that only your test account has read or write access to – never real users.

And maybe you prevent your test account from being able to interact with any real books that are visible to real users.

Tip: Also, maybe you’ve got some sort of automation running every so often to re-set the ratings of your fake book so that there are only two of them (a 0 and a 1, for an average of 0.5), so that by the time your end-to-end tests run and add a third rating of 5, you’ll actually notice the average go up noticeably when you run your test.



Synthetic testing

Also known as “synthetic monitoring”

Principles

  • Purpose: constantly (e.g. every few minutes/hours/days) re-validate that critical-path user flows through the production instance of the webapp seem to be up and running performantly – collect uptime stats, alert on failures
  • Quantity: fewer than E2E; you’re just spot-checking the apparent availability of a few mission-critical things (e.g. did items suddenly stop being able to be added to carts, in an e-commerce app?)

Technologies

Platforms used

Typically some sort of dedicated synthetic monitoring service. e.g.

  • Splunk Observability Cloud Synthetic Tests

How it’s run

  • via: the dedicated service
  • when: on a schedule (typically every few minutes, hours, days, etc. depending on your app)

Examples


Log in, add a review to a book, and see the updated average rating on the book’s page

File path
  • /tests/synthetic/splunk-config-terraform-files/user-flows/add-review.tf

Clarify with your team to come up with a file-naming pattern that works for you.

If you’re using an LLM coding assistant, be sure to make sure it knows what your team came up with, too.

Use fake accounts

The same issues apply as for fake accounts with E2E testing.

Tip: make them different fake accounts than you use in E2E testing, so you can tell if one leaks, acts up, etc.

This account is meant to have access to your production webapp – often “write” access.

Please play it safe and architect your entire workload so that if a synthetic monitoring “test user” credential were to be leaked on Pastebin, it wouldn’t matter (other than that maybe you’d be having some spam logins to deal with high volume of).

Please presume your test accounts, used against production, are going to be taken over by hackers and limit their “blast zone” accordingly. Think “when,” not “if,” and design defense in depth.

What it tests
  1. That login works (presuming that your team / QAs agree this is mission-critical)
  2. That a given book’s detail page loads promptly (presuming that your team / QAs agree this is mission-critical)
  3. That submitting a review for a book changes its average rating, within a reasonable delay (presuming that your team / QAs agree this is mission-critical)
    • See E2E notes above: this is a destructive test, so please consider having some fake books in the system that normal users can’t see, and prohibiting your synthetic monitoring user from ever dealing with real books.
    • See E2E tip above: also consider resetting the value of the fake books every so often, so that the change in average will actually be noticeable. That, or use something else as a way of validating whether the average was “likely” updated – e.g. validating that a certain JavaScript function got executed within the browser’s runtime.
What it doesn’t test
  1. That the /books URL displays a list of books (presuming that, for whatever reason, there’s not a lot of business value in detecting this proactively rather than just waiting for a user to open a support ticket saying it’s broken; clarify with your team / QAs)

Note that this doesn’t mean you don’t test this at all – it’s just more the concern of pre- and post-release E2E tests than of ongoing synthetic monitoring.

Typically:

  • a synthetic monitoring test is going to make sure that adding a product to a shopping cart results in a non-zero subtotal dollar value being displayed in the sidebar in a reasonable amount of time, whereas
  • an end-to-end test is going to make sure that the subtotal dollar value displayed in the sidebar actually is the sum of the cost of the items in the cart
--- ---