Setting up for Drupal's Functional JavaScript tests

Richard A. Allen — Sat, 08 Jun 2024 06:06:43 +0000

Resources

PHPUnit Browser test tutorial, Drupal.org, 26 January 2024
Running PHPUnit JavaScript tests, Drupal.org, 8 June 2024
PHPUnit JavaScript test writing tutorial, Drupal.org, 24 July 2022
Mink documentation, Behat.org, latest version
Inspiration for custom Dockerfile I'm using in Lando, Github, 2023
D.O. issue #3405976: Transaction autocommit during shutdown relies on unreliable object destruction order (xdebug 3.3+ enabled), Drupal.org, created 2023, updated 3 June 2024.

About you

You are a Drupal, Symfony, or any kind of PHP dev and are looking for help with a peculiar aspect of unit testing (see title above), a tale from the crypts, or nap time story that will put you to sleep. Probably a combination of all of those. Otherwise you should probably move along ;)

The context

I'm testing a OAuth implementation between Drupal and SoundCloud and using Drupal's functional suite to do so. The rest of my module is using either kernel or regular unit tests. But for OAuth it's either a mock (see tests for Martin1982\OAuth2 on Packagist), or a live call.

I already looked at some mocking methods and I don't like them, too fake; I know, hot take. The SoundCloud service decommissions your access if you do not use it. Baking a live OAuth call into the testing helps keep the access alive (otherwise you'd have to wait months for re-authorization).

I'm aiming to TDD the heck out of a contrib module I'm doing for fun, and so leaving stones unturned (in testing) is not an option. This is our definition of fun in the development department.

Not trying to say one way (mocking) is better than the other (live call), or that there is a right answer. I'm not going to Joel Spolsky this post either. I'm just going with my gut feeling and documenting the process.

Why this post

I already lost the opportunity to document other fixes/discoveries/tips I've done so far with regards to "functional javascript" unit testing today, and so that's why I am starting this post.

I know that not in a week, but rather by tomorrow I'll forget the details because they're too many to keep track of. Not a bug of this business, just a feature. Documenting stuff somewhere is how we deal with this ... loss of context.

Also the sooner I can start dumping my browser tabs somewhere else other than a session saver or OneNote, the faster that I can close them up. Marie Kondo would not approve of the amount of tabs I have open, and neither do I! The thing I hate about dumping an entire browser session is that I'll barely revisit those while hunting for a resource. And OneNote just gets out of control, too... many... notes

Needless to say, you should know this is a raw post. There is no editing. There is no shiny merchandising material, alt coins or artificial intelligence involved.

Anyways.

Before PHPUnit was having trouble connecting to Chrome. A quick telnet in the Lando appserver container confirmed that Chrome was indeed up and reachable by the PHP container:

And before that I dealt with



/app/docroot/web/core/tests/Drupal/FunctionalJavascriptTests/WebDriverTestBase.php:146
The "chromeOptions" array key is deprecated in drupal:10.3.0 and is removed from drupal:11.0.0. Use "goog:chromeOptions instead.

The link (with fix) to that notice is https://www.drupal.org/node/3422624.

I prefer to use environment variables to configure the unit testing because they're easier to access than phpunit.xml. While I've set them before in the Lando config file, setting things in Lando means that you have to rebuild and restart the whole stack for environmental changes to take effect. Time, it just adds up, man.

Instead I put it in a bash helper called by Lando tooling. This is the env var:



# The "chromeOptions" array key is deprecated in drupal:10.3.0 and is removed from drupal:11.0.0.
# Use "goog:chromeOptions instead. See https://www.drupal.org/node/3422624
export MINK_DRIVER_ARGS_WEBDRIVER='["chrome", {"browserName": "chrome", "goog:chromeOptions": {"args": ["--disable-gpu","--headless", "--no-sandbox", "--disable-dev-shm-usage"]}}, "http://chrome:9515"]'

Fixing the (second) connectivity issue

My Lando chrome spec needed to be updated to add the allowed origins flag to the chromedriver:



  chrome:
    type: compose
    services:
      image: drupalci/webdriver-chromedriver:production
      command: chromedriver --log-path=/tmp/chromedriver.log --verbose --allowed-origins=* --whitelisted-ips=

Note that there is a myriad chromedriver images out there, and that the drupalci happens to be the one mentioned by either Lando or Drupal.org documentation (maybe).

Credit for that fix is in Github, thanks to user @Niklan.

Access is denied and invalid cookie domain

Mink is reporting Failed to read the 'sessionStorage' property from 'Window': Access is denied for this document.

PHPUnit is reporting Test Skipped (Drupal\Tests\musica\FunctionalJavascript\FooTest::testMyFirstJavasScriptTest) An unexpected error occurred while starting Mink: invalid cookie domain

T.B.C., 6/7/24

I still have 30 or so browser tabs open (lol) and have to hunt this one down.

Love spending time on testing infrastructure instead of "actual code" (sarcasm).

It's well past beyond midnight and tomorrow is another day. I'll come back to update this post as I move along.

T.B.C. ...

Invalid cookie domain, 6/8/24

This seems to be highly correlated to env vars SIMPLETEST_BASE_URL and BROWSERTEST_OUTPUT_BASE_URL. I changed the first from secure https to plain-text http. The URL belongs to the Lando project hosting the Docker and Drupal stack.

Here is what my environment variables for PHPUnit are looking like now:



# Variables by integration tests.
export SIMPLETEST_BASE_URL="http://d10ee.lndo.site"
export SIMPLETEST_DB="mysql://drupalX:drupalX@database/drupal10_simpletest"
export BROWSERTEST_OUTPUT_DIRECTORY="$SITES_PATH/simpletest/browser_output"
export BROWSERTEST_OUTPUT_BASE_URL="http://d10ee.lndo.site"

This feels good, this is a great start to my weekend. The Chrome webdriver logs went from displaying nada to basically a million lines of verbose output? It's doing a lot of something!

Another good thing that happened is that the invalid cookie domain error in the PHPUnit logs went away (when run with the --debug flag), and I got a hit in the breakpoint for one of the tests. Both dummy tests I have in place were getting skipped before and so no test function breakpoints were getting hit.

Now here's an interesting error that I've never seen before while doing Unit or Kernel tests in Drupal:

PHPUnit\Framework\Exception: PHP Fatal error: Uncaught AssertionError: Transaction $stack was not empty.

All I have for my dummy test is the following:



  /**
   * Stub.
   */
  public function testCancelExpressionInRule(): void {
    $page = $this->getSession()->getPage();

    $this->assertTrue(TRUE, 'dumb assertion');

    $test = NULL;
  }

I'll be interesting to see where that error is coming from. I've never done this particular kind of test in Drupal (Functional JavaScript through PHPUnit), so I'll be learning something new and sharing it here.

T.B.C., see you later today!

Test Class Structure, 6/8/24 1:44pm

I want to make a brief note about how the Functional JavaScript test is structured.

When I declare my test, the class looks something like this:



#[CoversNothing]
#[Group('javascript')]
class FooTest extends WebDriverTestBase {

The covers nothing group is used because as far as I understand, Functional and Functional JavaScript tests are not able to cover anything (hence they cover nothing). The group is arbitrary and I use it to fine-tune which tests get run.

Note the extension of WebDriverTestBase. This is an abstract class in Drupal core, which itself extends BrowserTestBase.



/**
 * Runs a browser test using a driver that supports JavaScript.
 *
 * Base class for testing browser interaction implemented in JavaScript.
 *
 * @ingroup testing
 */
abstract class WebDriverTestBase extends BrowserTestBase {

WebDriverTestBase is located at core/tests/Drupal/FunctionalJavascriptTests/WebDriverTestBase.php.

BrowserTestBase itself extends PHPUnit\Framework\TestCase, which as the namespace indicates comes from PHPUnit itself.

So the inheritance hierarchy looks something like this:

PHPUnit\Framework\TestCase -> BrowserTestBase -> WebDriverTestBase -> YourFooBarTest

The reason why I bring this up is because the BrowserTestBase->setUp() method is interesting:



  /**
   * {@inheritdoc}
   */
  protected function setUp(): void {
    parent::setUp();

    $this->setUpAppRoot();
    chdir($this->root);

    // Allow tests to compare MarkupInterface objects via assertEquals().
    $this->registerComparator(new MarkupInterfaceComparator());

    $this->setupBaseUrl();

    // Install Drupal test site.
    $this->prepareEnvironment();
    $this->installDrupal();

    // Setup Mink. Register Mink exceptions to cause test failures instead of
    // errors.
    $this->registerFailureType(MinkException::class);
    $this->initMink();

    // Set up the browser test output file.
    $this->initBrowserOutputFile();

    // Ensure that the test is not marked as risky because of no assertions. In
    // PHPUnit 6 tests that only make assertions using $this->assertSession()
    // can be marked as risky.
    $this->addToAssertionCount(1);
  }

In particular,



    // Install Drupal test site.
    $this->prepareEnvironment();
    $this->installDrupal();

Why is this interesting? Well, on the Chrome driver logs I'm seeing the following:



[1717864444.039][DEBUG]: DevTools WebSocket Response: Page.getFrameTree (id=30) (session_id=DFB171623C6AD0FD50AE0D2C08195426) 0A012DB02731872ED0B91131E95AB5F4 {
   "frameTree": {
      "frame": {
         "adFrameStatus": {
            "adFrameType": "none"
         },
         "crossOriginIsolatedContextType": "NotIsolated",
         "domainAndRegistry": "lndo.site",
         "gatedAPIFeatures": [  ],
         "id": "0A012DB02731872ED0B91131E95AB5F4",
         "loaderId": "D4966004B7AA6D4BA7B903A2FC1BE098",
         "mimeType": "text/html",
         "secureContextType": "InsecureScheme",
         "securityOrigin": "http://d10ee.lndo.site",
         "url": "http://d10ee.lndo.site/core/install.php"
      }
   }
}

This is odd because as I highlighted above, the Functional JavaScript test, by capacity of it's inheritance model is supposed to $this->installDrupal();.

Basically PHPUnit installs it's own database scheme, separate from whatever the default Lando/DDev/Docker database is for your Drupal instance. And it does it for each test suite, at the bare minimum. Not 100% sure if it does it for each single test case, but it works along those broad outlines. The finer-grained details of how many times the database is installed is not relevant here, just the fact that PHPUnit installs the database at least once is.

When I see install.php it means that something is still probably amiss in the PHPUnit configuration... because the test cases should be visiting an already-installed Drupal instance, not a yet-to-be installed instance.

Digging into it.

C.U. later

Wholly molly: Transaction $stack was not empty

Well I didn't see this one coming, certainly out of leftfield it came.

The full error (minus stack trace) is: PHPUnit\Framework\Exception: PHP Fatal error: Uncaught AssertionError: Transaction $stack was not empty. Active stack: 666487f8d33255.42467631\drupal_transaction in /app/docroot/web/core/lib/Drupal/Core/Database/Transaction/TransactionManagerBase.php:99

Luckily the first Google result is for D.O. issue #3405976: Transaction autocommit during shutdown relies on unreliable object destruction order (xdebug 3.3+ enabled)

Scrolling through the comments I see Mondrake mention that a fix is to disable xdebug's develop mode.

Let's see, what does my Lando say:



  appserver:

    # Lies, deception: tell Lando it's 8.2 instead of 8.3.
    # https://github.com/lando/php/issues/77.
    type: php:8.2

    # https://docs.lando.dev/config/php.html#configuration
    xdebug: "debug,develop,coverage"
    config:
      php: ../lando/resources/php.ini

Container:



www-data@31a2e4846210:/app/docroot$ echo $XDEBUG_MODE
debug,develop,coverage

I'm definitely not gonna spend a dollar or two in electricity rebuilding the Docker stack, times are tight!

I go to my trusty Bash test runner for PHPUnit and change XDEBUG_MODE:



# From this :
# export XDEBUG_MODE="debug,develop,coverage"

# To this :
export XDEBUG_MODE="debug"

Coverage hasn't been working at all for me lately, so I'll leave it disabled until I get to that particular bone another day.

Fixing XDebug fatal error

Before:



PHPUnit 10.5.20 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.7
Configuration: /app/docroot/web/core/phpunit.xml.dist

E

Time: 00:37.061, Memory: 12.00 MB

Foo (Drupal\Tests\musica\FunctionalJavascript\Foo)
 ✘ Cancel expression in rule
   ┐
   ├ PHPUnit\Framework\Exception: PHP Fatal error:  Uncaught AssertionError: Transaction $stack was not empty. Active stack: 666487f8d33255.42467631\drupal_transaction in /app/docroot/web/core/lib/Drupal/Core/Database/Transaction/TransactionManagerBase.php:99
   ├ Stack trace:
...
ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

After



PHPUnit 10.5.20 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.7
Configuration: /app/docroot/web/core/phpunit.xml.dist

.F                                                                  2 / 2 (100%)

Time: 01:03.459, Memory: 12.00 MB

Foo (Drupal\Tests\musica\FunctionalJavascript\Foo)
 ✔ Cancel expression in rule
 ✘ My first javas script test
   ┐
   ├ Behat\Mink\Exception\UnsupportedDriverActionException: Status code is not available from Drupal\FunctionalJavascriptTests\DrupalSelenium2Driver
...
FAILURES!
Tests: 2, Assertions: 5, Failures: 1.

Notice the test result goes from E to .F, which in PHPUnit lingo means the first unit test ran! I must say I was surprised when I saw that dot "pop up".

If you're wondering what version of Xdebug you're running, just do php -v, like so:



www-data@31a2e4846210:/app/docroot$ php -v
PHP 8.3.7 (cli) (built: Jun  6 2024 01:39:14) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.3.7, Copyright (c) Zend Technologies
    with Zend OPcache v8.3.7, Copyright (c), by Zend Technologies
    with Xdebug v3.4.0alpha2-dev, Copyright (c) 2002-2024, by Derick Rethans

The D.O. issue mentions Xdebug 3.3+, and I'm on 3.4.x, so the version matches the description, along with the database-related fatal error stack trace. Another win for today.

Conclusion

With the last fix for Xdebug addressed, and all the kinks of getting Chrome up and running for the Drupal Functional JavaScript tests (I really wish they changed the name to something shorter), I am going to wrap up this post.

With this part of the testing infrastructure fully operational I can now finally go around my business of actually writing software as opposed to infrastructure as software.

I'm sure these errors will pop up someday again and I'll be glad I posted this here for my future self.

If you have any questions or feedback, make sure to use the comments below!

I will leave here the bash test runner I've been referencing through the post:

Here is the current Landofile .lando.yml I am using. Notice I am not using Lando's default PHP image. This is because my current version of Lando does not support the latest Lando PHP recipe, so I had to cook my own PHP image in order to get the latest PHP.

If you really want or need to build your own Docker container like I'm doing, just check out the Lando issue I'm referencing at https://github.com/lando/php/issues/77. It contains pretty much the Dockerfile I'm using in the build parameter. I'm not pushing it (the image is massive), so you'll literally have to build instead of pulling if you go that route.

For the Lando tooling in .lando.base.yml, the relevant config is:



  # Call PHPUnit from the directory that contains Drupal core /vendor.
  # It's where bootstrap.php expects to be called from.
  test:
    description: Debug PHPUnit
    dir: /app/docroot
    cmd:
      - appserver: ./test-lando

That's it for today, Brooklyn sends its regards!

DEV Community: Richard A. Allen