Playground CLI: Mount /wordpress, /internal, and /tmp dirs within real temporary dir #2446

brandonpayton · 2025-07-31T06:33:14Z

Motivation for the change, related issues

There are two reasons to mount a real directory as /internal/shared:

We intend /internal/shared to be truly shared between all PHP instances. Issue:
Playground CLI: Make /internal/shared truly shared in a multi-worker environment #2301.
Zipping the primary worker /internal dir and unzipping for secondary workers is making multi-worker CLI startup quite slow. Issue:
Playground CLI: Starting additional workers delays readiness of Playground CLI server #2419.

Once we looked at doing the above, we saw that it would be good for workers to share the same real FS by default. It made it easier for users to start a multi-worker Playground server, and those workers would all naturally share the same underlying FS state.

Implementation details

When launching, Playground CLI:
- Creates a native temp dir that is configured to be cleaned up before the application exits.
- Launches a non-blocking cleanup of any unused Playground temp dirs that are:
  - Older than two days
  - No longer associated with a running process
We create the following subdirectories in the temp dir:
- /internal
- /wordpress
- /home - We don't use this AFAIK, but since it is assumed to exist in the emscripten-generated php-wasm JS, it seemed good to make sure it is also backed by a native dir.
- /tmp
The real path for the /internal dir is passed to php-wasm init where it handles mounting and creation of internal subdirs.
If the user does not provide mounts for /wordpress, /home, or /tmp, we mount the temp subdirs in their place.

Testing Instructions (or ideally a Blueprint)

CI

brandonpayton · 2025-07-31T07:16:12Z

With --enableMultiWorker=2 and a native dir mounted as /internal/shared, Playground CLI dies in the second call to bootWordPress(). We haven't ever had two workers that truly shared a live /internal/shared dir, so this isn't too surprising.

Planning to look at this more tomorrow.

brandonpayton · 2025-07-31T07:21:44Z

Currently, the Blueprints v1 worker's bootAsSecondaryWorker() method just calls bootAsPrimaryWorker(). We might be running into conflicts with attempting to recreate existing files or something like that.

brandonpayton · 2025-07-31T07:29:38Z

Currently, the Blueprints v1 worker's bootAsSecondaryWorker() method just calls bootAsPrimaryWorker(). We might be running into conflicts with attempting to recreate existing files or something like that.

Yeah, the errno is 20 which maps to EEXIST in Emscripten libs.

adamziel · 2025-07-31T07:52:06Z

Yeah we might need to rewire some of the /internal initialization logic for a secondary worker when using Blueprints v1 to support this.

brandonpayton · 2025-07-31T18:08:47Z

This is a functional experiment now, and Playground CLI multi-worker boot feels nearly instantaneous on my laptop.

It requires rebuilding php-wasm/node, which I plan to do after lunch.

NOTE:
I did see one random error where Playground CLI was redirecting homepage requests to an error page like "expected WebSocket request" or something like that. But the error disappeared after restarting the CLI, and I haven't been able to reproduce it again.

brandonpayton · 2025-07-31T18:10:33Z

Yeah we might need to rewire some of the /internal initialization logic for a secondary worker when using Blueprints v1 to support this.

@adamziel, the only issue appears to have been creating the initial dirs under /internal/shared. By switching those lines from FS.mkdir() to FS.mkdirTree() and making one file creation conditional, the problem was fixed.

brandonpayton · 2025-08-02T04:26:25Z

Some things that need done:

Stress test
Add support for temp /internal/shared dir for Blueprints v2
Add auto-cleanup when Playground CLI is killed
Confirm that we are OK with the temp dir location and default permissions in case temp dir cleanup fails

brandonpayton · 2025-08-02T04:27:02Z

This isn't ready to merge, but it is ready for more feedback.

packages/php-wasm/compile/php/phpwasm-emscripten-library.js

adamziel · 2025-08-04T15:50:20Z

This is simple and effective, I like it. Thank you @brandonpayton!

Add auto-cleanup when Playground CLI is killed

This is a good idea. There will be some cases where that auto-cleanup won't run, e.g. the process gets killed without waiting, there's a power outage etc. I wonder what are some things we could do to maximize our chances of actually cleaning up stale directories. Perhaps we could use a naming pattern including PID and then sweep stale directories on startup? There could be some risk with short-lived processes so maybe a playground-{pid}-{timestamp} could help resolve them, as in we'd only sweep stale directories that are at least 1 hour old?

adamziel · 2025-08-04T16:50:28Z

Poking around more, we'll need to address all these paths:

wordpress-playground/packages/playground/wordpress/src/boot.ts

Lines 232 to 236 in 89a7f30

    
           proxyFileSystem(await requestHandler.getPrimaryPhp(), php, [ 
        
           	'/tmp', 
        
           	requestHandler.documentRoot, 
        
           	'/internal/shared', 
        
           ]);

Ideally, we can either get rid of proxyFileSystem() or find a way to contextualize what it does, as in uses PROXYFS, NODEFS, or other synchronization means in the future.

Also, we need to take this bit of the PHP class into account (in hotSwapPHPRuntime) – I think it already skips non-MEMFS nodes, but I'll still surface it here just in case:

wordpress-playground/packages/php-wasm/universal/src/lib/php.ts

Line 1279 in 89a7f30

// Copy the old /internal directory to the new filesystem

adamziel · 2025-08-04T16:55:29Z

Noodling on this more, perhaps a useful API would be something like markPathAsShared(/internal/shared, secondaryWorker, primaryWorker)? 🤔 The workers could then decide what to do based on how they were initialized, e.g.

Ignore the call in a primary worker and mount a primaryWorker path PROXYFS in a secondary worker
Mount a NODEFS directory in all the workers
(future) Mount a SharedArrayBuffer Filesystem

In any case, I've instrumented the Blueprints v2 worker code and the workers seem to boot fairly quickly. I don't have any numbers to quote but it feels fast!

adamziel · 2025-08-04T17:01:09Z

packages/php-wasm/node/asyncify/php_8_0.js

@@ -3914,6 +3912,8 @@ export function init(RuntimeName, PHPLoader) {
 				node = lookup.node;

 				if (FS.isMountpoint(node)) {
+					console.log({ mountpoint });


I accidentally reformatted the file. My changes here are all about this and the following two console.logs.

brandonpayton · 2025-08-07T21:11:48Z

This comment responds to multiple comments, not necessarily in the order they were made.

In any case, I've instrumented the Blueprints v2 worker code and the workers seem to boot fairly quickly. I don't have any numbers to quote but it feels fast!

Nice! Thanks for adding that. Yeah, that's been my experience as well.

Noodling on this more, perhaps a useful API would be something like markPathAsShared(/internal/shared, secondaryWorker, primaryWorker)? 🤔 The workers could then decide what to do based on how they were initialized,

@adamziel, what if we invert this? It seems like we want all workers to respond to requests based on the same state. What FS paths do we want to be private/separate per worker?

Poking around more, we'll need to address all these paths:
and
Also, we need to take this bit of the PHP class into account (in hotSwapPHPRuntime) – I think it already skips non-MEMFS nodes, but I'll still surface it here just in case:

Ah, good points! It's obvious we'd want to stop proxying already-shared filesystems (and same for copying from old to new PHP instances during hot swap), but somehow I was forgetting that.

adamziel · 2025-08-07T22:13:22Z

what if we invert this? It seems like we want all workers to respond to requests based on the same state. What FS paths do we want to be private/separate per worker?

Oh I like it! Off the top of my head, it's just the input/output devices. It seems like we could share everything else 🤔

So these paths would become worker-specific:

/internal/stdout
/internal/stderr
/internal/headers

We could move them to /internal/isolated (as opposed to /internal/shared to keep the /internal/shared path working for existing API consumers) and share the rest of the filesystem.

brandonpayton · 2025-08-08T03:47:41Z

Oh I like it! Off the top of my head, it's just the input/output devices. It seems like we could share everything else 🤔

So these paths would become worker-specific:
* /internal/stdout
* /internal/stderr
* /internal/headers
We could move them to /internal/isolated (as opposed to /internal/shared to keep the /internal/shared path working for existing API consumers) and share the rest of the filesystem.

Sweet. I'm looking at doing this.

Using /internal/isolated seems a bit tricky to do with Emscripten mounting because it doesn't look like we can easily mount a MEMFS dir inside a NODEFS dir. At least Google suggests we probably cannot easily do this because it would mean mixing contents of a real NODEFS dir with artificial mounts in that dir. Based on my recent looks into emscripten FS implementations, I found this reasoning compelling, but I haven't tested it.

If that is truly an issue, we will likely run into trouble if trying to mount a NODEFS dir as a subdir of our default/temp NODEFS dir. Then again, how does wp-now do this today for automounted subdirs of /wordpress ?

Will test and follow up here.

brandonpayton · 2025-08-08T03:53:49Z

Using /internal/isolated seems a bit tricky to do with Emscripten mounting because it doesn't look like we can easily mount a MEMFS dir inside a NODEFS dir. At least Google suggests we probably cannot easily do this because it would mean mixing contents of a real NODEFS dir with artificial mounts in that dir. Based on my recent looks into emscripten FS implementations, I found this reasoning compelling, but I haven't tested it.

OK, I guess we can forget that. In testing, it seems totally possible. I can mount a real dir as /wordpress and then mount another as /wordpress/wp-content on top of that.

brandonpayton · 2025-08-08T04:26:04Z

OK, I guess we can forget that. In testing, it seems totally possible. I can mount a real dir as /wordpress and then mount another as /wordpress/wp-content on top of that.

Ah, that wasn't mounting a MEMFS dir within a NODEFS dir, but I just tested that by editing a php_8_3.js file and mounting a MEMFS dir inside NODEFS like:

if (phpWasmInitOptions?.nativeInternalDirPath) {
	FS.mount(
		FS.filesystems.NODEFS,
		{ root: phpWasmInitOptions.nativeInternalDirPath },
		'/internal/shared'
	);
	FS.mkdir('/internal/shared/something');
	FS.mount(
		FS.filesystems.MEMFS,
		{ root: phpWasmInitOptions.nativeInternalDirPath },
		'/internal/shared/something'
	);
}

And there are no errors during boot. So this should all be doable. 👍

brandonpayton · 2025-08-08T04:29:40Z

Random thing, when I was thinking of adjusting the directory trees to workaround possibly mount challenges, I was thinking something like:

/wordpress - for WordPress
/internal - for files shared by entire php-wasm server
/request - for state that is unique to the current request and the single php-wasm instance that is handling it. stdout, stderr, and headers would go here.

@adamziel how does this sound to you? I kinda like it, at least compared to nesting non-shared things in a shared dir.

brandonpayton · 2025-08-08T04:30:10Z

I'm out of time today and plan to continue tomorrow.

brandonpayton · 2025-08-08T19:00:23Z

Also, we need to take this bit of the PHP class into account (in hotSwapPHPRuntime) – I think it already skips non-MEMFS nodes, but I'll still surface it here just in case:

wordpress-playground/packages/php-wasm/universal/src/lib/php.ts

Line 1279 in 89a7f30

// Copy the old /internal directory to the new filesystem

Cool. It looks like we already skip any FS node that is not detected as MEMFS:

wordpress-playground/packages/php-wasm/universal/src/lib/php.ts

Lines 1457 to 1461 in 533d676

    
           // MEMFS nodes have a `contents` property. NODEFS nodes don't. 
        
           // We only want to copy MEMFS nodes here. 
        
           if (!('contents' in oldNode.node)) { 
        
           	return; 
        
           }

adamziel · 2025-08-08T23:30:45Z

Random thing, when I was thinking of adjusting the directory trees to workaround possibly mount challenges, I was thinking something like:
/wordpress - for WordPress
/internal - for files shared by entire php-wasm server
/request - for state that is unique to the current request and the single php-wasm instance that is handling it. stdout, stderr, and headers would go here.
@adamziel how does this sound to you? I kinda like it, at least compared to nesting non-shared things in a shared dir.

Sounds good to me! I want to preserve the directory structure inside the /internal directory as I know at least Studio creates and reads files from there. stdout, stderr, and headers are not regular files, though. They're devices, and they're off limits – folks are not supposed to use those files directly. We can move them somewhere else just fine.

adamziel · 2025-09-15T23:12:14Z

This is looking really good! I've left some notes, a few of which are blocking, but nothing major. Thank you for figuring this one out @brandonpayton!

brandonpayton · 2025-09-16T04:53:36Z

This is looking really good! I've left some notes, a few of which are blocking, but nothing major. Thank you for figuring this one out @brandonpayton!

@adamziel! It's good to have your review. I wanted to self-review and get some of the mess cleaned up before you looked, but it sounds like it mostly wasn't too bad. :) Thanks! I plan to follow up on your comments tomorrow.

mho22 · 2025-09-16T09:53:36Z

🚀

brandonpayton

Thanks for the review! I went over the comments, made changes, and left some questions.

A couple of things remain that we could do here or in a follow-up PR:

Find a way to avoid Playground CLI instances launching conflicting stale-directory cleanup operations. It shouldn't break anything but could cause some confusing error messages to be printed.
Add logic to remount under /internal/symlinks if the mount target changes from a file to a directory or vice versa.

packages/php-wasm/compile/php/phpwasm-emscripten-library.js

brandonpayton · 2025-09-17T02:22:32Z

packages/php-wasm/node/src/lib/load-runtime.ts

+							symlinkMountNode.mount.mountpoint !==
+							symlinkMountPath
+						) {
+							phpRuntime.FS.mount(


This area has been tricky for me to think through, but I think it is OK, as long as user PHP doesn't directly manipulate /internal/symlinks descendants. Here's why:

/internal/symlinks contains the paths to symlink targets, not the symlinks themselves. If a PHP script changes a symlink (outside of /internal/symlinks), all that will happen is that our realpath() override will create another mount under /internal/symlinks/the/absolute/path/of/new/symlink/target.

Therefore, if a symlink changes its target, it shouldn't have any effect on /internal/symlinks.

Does that makes sense?

brandonpayton · 2025-09-17T02:27:28Z

packages/php-wasm/node/src/lib/load-runtime.ts

+							symlinkMountNode.mount.mountpoint !==
+							symlinkMountPath
+						) {
+							phpRuntime.FS.mount(


One thing that does look like a "gotcha" is if someone switches the type of mount from "dir" to "file" or vice versa. Maybe we can handle this in a follow-up PR as it is an existing limitation. Sound OK?

packages/php-wasm/universal/src/lib/php.ts

packages/playground/cli/src/run-cli.ts

packages/playground/cli/src/blueprints-v2/worker-thread-v2.ts

brandonpayton · 2025-09-17T04:10:42Z

packages/playground/cli/src/blueprints-v2/worker-thread-v2.ts

+			},
+			onPHPInstanceCreated: async (php: PHP) => {
+				await mountResources(php, args['mount-before-install'] || []);
+				if (this.blueprintTargetResolved) {


It feels a bit off to have to track the pre-install PHP instances like this, but I like how explicit the current code is. This way, we are able to see all waiting instances in one place. In addition, we can remove a PHP instance from the wait list if it exits before the blueprint target is resolved.

AFAICT, if we awaited a promise for blueprintTargetResolved, dead PHP instances would not be able to be GC'd until the blueprint target was resolved, and we'd have to devise another way to avoid trying to apply post-install mounts to each dead PHP instance.

Does that sound reasonable to you?

brandonpayton · 2025-09-17T04:12:23Z

packages/playground/cli/src/run-cli.ts

+
+			// NOTE: This is an async operation, but we do not care to block on it.
+			// Let's let the cleanup happen as the main thread has time.
+			cleanupStalePlaygroundTempDirs(


Let's make sure we're not running multiple simultaneous cleanups in two processes if the developer runs npx @wp-playground/cli twice in two different terminals.

@adamziel, I didn't get to this today but should be able to look at it tomorrow. Or we could address as a follow-up.

…of workers and php instances

adamziel · 2025-09-17T12:34:35Z

I've patched the last few remaining things I was nervous about. This one seems good to go – let's do it! 🤞 🤞 🤞

brandonpayton · 2025-09-17T14:45:46Z

packages/php-wasm/universal/src/lib/php.ts

-			) {
+			// The /request directory holds per-request state that is isolated to a
+			// single PHP instance. Let's not copy it.
+			if (path && path !== '/request') {


@adamziel, great catch! Thank you! We don't want to be duplicating request state.

brandonpayton · 2025-09-17T15:05:27Z

@adamziel, thanks for your help! It's great to be getting this in.

@brandonpayton

## Motivation for the change, related issues #2446 assumes nested mounts are supported by Playground CLI. That wasn't always the case. This PR adds an explicit regression test to ensure they work and will keep working. cc @brandonpayton

brandonpayton requested a review from a team July 31, 2025 06:33

brandonpayton self-assigned this Jul 31, 2025

brandonpayton added [Type] Exploration An exploration that may or may not result in mergable code [Package][@php-wasm] Node [Package][@wp-playground] CLI labels Jul 31, 2025

brandonpayton mentioned this pull request Aug 2, 2025

Playground CLI: Make additional worker initialization lazy #2420

Closed

brandonpayton marked this pull request as ready for review August 2, 2025 04:26

adamziel reviewed Aug 4, 2025

View reviewed changes

packages/php-wasm/compile/php/phpwasm-emscripten-library.js Show resolved Hide resolved

adamziel reviewed Aug 4, 2025

View reviewed changes

packages/php-wasm/compile/php/phpwasm-emscripten-library.js Show resolved Hide resolved

adamziel reviewed Aug 4, 2025

View reviewed changes

adamziel mentioned this pull request Sep 16, 2025

Function mismatch fix in GD jpeg extension for PHP below 8.0 #2634

Open

brandonpayton added 9 commits September 16, 2025 22:08

Correct mistaken comment

ff18b47

Try to clarify is-symlink-mounted check

898f619

Attempt to undo unnecessary changes to php-process-manager

3fb05fd

On copy /internal during hot swap if on non-shared FS

cbed3d1

Disambiguate names for php versions and instances

3ab8f8c

Remove stale TODO

6d64a98

Make PHPInstanceCreated callback type more readable

59f9ed5

Clarify when certain PHP instances get post-WP-install mounts

ae186cc

Clarify subdir-has-a-mount checks

a7ef33e

brandonpayton commented Sep 17, 2025

View reviewed changes

brandonpayton and others added 6 commits September 17, 2025 00:24

Fix boolean expression for isSymlinkMounted

da64b2e

Attempt to address runtime rotation test failures

8fb5625

Remove unused import

c7f54ec

Redo propagating directories on runtime rotation

9a40c60

rename copyFS to copyMEMFSNodes

ac30770

Clean up copyMEMFSNodes

070d55f

adamziel mentioned this pull request Sep 17, 2025

[PHP] Add unit tests for nested mounts #2639

Merged

adamziel added 2 commits September 17, 2025 13:42

Only create the initial files on boot once, regardless of the number …

3d08016

…of workers and php instances

Rollback unrelated change

cc89bd0

adamziel merged commit 4a35b82 into trunk Sep 17, 2025
26 checks passed

adamziel deleted the playground-cli/try-making-internal-real-temp-dir branch September 17, 2025 12:34

brandonpayton commented Sep 17, 2025

View reviewed changes

adamziel mentioned this pull request Sep 17, 2025

[CLI] php command to run PHP scripts #2641

Open

Playground CLI: Mount /wordpress, /internal, and /tmp dirs within real temporary dir #2446

Playground CLI: Mount /wordpress, /internal, and /tmp dirs within real temporary dir #2446

Uh oh!

Conversation

brandonpayton commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for the change, related issues

Implementation details

Testing Instructions (or ideally a Blueprint)

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

adamziel commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Jul 31, 2025

Uh oh!

brandonpayton commented Aug 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Aug 2, 2025

Uh oh!

Uh oh!

Uh oh!

adamziel commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamziel commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamziel commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamziel Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

brandonpayton commented Aug 7, 2025

Uh oh!

adamziel commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

brandonpayton commented Aug 8, 2025

Uh oh!

adamziel commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamziel commented Sep 15, 2025

Uh oh!

brandonpayton commented Sep 16, 2025

Uh oh!

mho22 commented Sep 16, 2025

Uh oh!

brandonpayton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

brandonpayton Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

brandonpayton Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

brandonpayton commented Jul 31, 2025 •

edited

Loading

adamziel commented Jul 31, 2025 •

edited

Loading

brandonpayton commented Aug 2, 2025 •

edited

Loading

adamziel commented Aug 4, 2025 •

edited

Loading

adamziel commented Aug 4, 2025 •

edited

Loading

adamziel commented Aug 4, 2025 •

edited

Loading

adamziel commented Aug 7, 2025 •

edited

Loading

adamziel commented Aug 8, 2025 •

edited

Loading