Questions on Packagist package updates and tag handling

[hirasso]

Hello! I have a few questions about how Packagist interacts with GitHub repositories, and I haven’t been able to find documentation that fully clarifies them.

1. When does Packagist pull updates from a GitHub repo?

• Does Packagist pull updates every time a tag is created, regardless of the branch?
• Is there a way to specify a tag naming pattern that Packagist should expect for a certain package (e.g., tags starting with v)?

2. Managing build artifacts and vendor folder with Packagist

My goal is to scope my dependencies with php-scoper. In the future, I may also need to include pre-processed JavaScript files in my package. Ideally, I’d like to avoid committing these build artifacts to my main repository just to have them appear on Packagist.

Current plan: I’m considering maintaining a separate dist branch where I would, on each merge from main:

1. Commit the prefixed vendor directory.
2. Create a tag so that packagist picks it up.

Would this approach ensure that only the necessary files are visible on Packagist?

Or is there a better method to include the vendor directory in my package—something more similar to the npm publish process, where I could manually push releases with a version?

Thanks for your help!

[Seldaek]

What package is this for? Why do you feel like including all deps prefixed is a good idea for you while 99.99% of packages do not do this?

[hirasso]

Thank you @Seldaek, for getting back to me! I'll try to explain my perspective as clearly as possible:

My Package

The package I'm investigating this for is a WordPress plugin in very early alpha stage. It can be found here: https://github.com/hirasso/placeholders. It's a simple plugin with only a few non-dev composer dependencies, but I’m planning to release other plugins with more dependencies in the future. Additonaly to the official WP.org plugin repository, I'd also love to be able to distribute my plugins via Packagist. I usually use a fully composer-managed WordPress boileerplate, roots/bedrock, so being able to manage my plugins via Packagist would be nice and I could skip the WP-org bottleneck (SVN, plugin review process, ...).

The plugin ecosystem problem

In WordPress (or any other ecosystem like for example Statamic or Kirby CMS), it’s not uncommon for plugin dependencies to raise conflicts between each other at some point. Tools like php-scoper, mozart or strauss were created to solve this problem.

For example, I recently encountered an issue where I couldn't install a composer package due to a strict dependency on psr/log, which conflicted with the version required by my already-installed version of monolog/logger. When conflicts like this occur, users depend on package maintainers to quickly release a fix (Maybe I'm wrong with this assumption!). Otherwise, they may need to maintain their own fork, which will lead to ecosystem fragmentation — something I'd like to avoid.

...and I just found out about phpstan/phpstan and phpstan/phpstan-src – where the first is used to distribute to packagist and the second is used to develop the package. For similar reasons: To avoid dependency conflicts.

WordPress

Quite a few of the more successful WordPress plugins have encountered dependency issues in the past and implemented custom solutions. Here is an example of one vendor talking about the issue and their solution to it. Sadly no info in that post about if and how they publish their scoped version to packagist.

Another large plugin vendor, Yoast, provides the most popular WordPress SEO plugin, which can also be found on Packagist. They are going to great lengths to prefix and ship their non-dev dependencies and include them in their Packagist distribution. I haven't yet figured out exactly how they do it (the vendor_prefixed folder in their repository is empty). However, when you run composer require yoast/seo, that folder is filled with guzzlehttp, league/oauth2-client, and a few other packages, all prefixed with the namespace YoastSEO_Vendor\. These dependencies are not listed in the require object in their composer.json, to prevent conflict errors during composer install.

Other thoughts/questions

• As opposed to frontend javascript bundles, where file size matters, PHP files only lives on the server, so file size can't be the argument for having only one version of a package installed at the same time (Maybe I'm wrong!)
• With a composer dump-autoload --classmap-authorative, performance should also not be a huge issue with multiple composer autoloaders in a project (Maybe I'm wrong!)
• When I run composer why psr/log in my project, I'm seeing this:

composer/xdebug-handler 3.0.5 requires psr/log (^1 || ^2 || ^3) 
monolog/monolog         3.7.0 requires psr/log (^2.0 || ^3.0)  

How will this look like in a few years? Like this?

composer/xdebug-handler 3.0.5 requires psr/log (^1 || ^2 || ^3 || ^4 || ^5 || ^6 || ^7 || ^8 || ^9 || ^10) .....
monolog/monolog         3.7.0 requires psr/log (^2.0 || ^3.0 || ^4.0 || ^5.0 || ^6.0 || ^7.0 || ^8.0 || ^9.0 || ^10.0) .....  

That covers the PHP side of my thoughts. The other part concerns frontend assets with a build step:

Frontend assets with a build step

Ideally, I wouldn’t want to commit my built frontend assets (scripts and styles) to my repos just to publish them with a Packagist package. Statamic, for example, compiles its frontend assets in a GitHub release action and attaches them to the release, and downloads them "manually" later via yet another Composer dependency/plugin.

All of this would be so much simpler if there were an "official" way to distribute packages with a build step to Packagist. Similar to npm publish for the JavaScript world. My dream-come-true would be a composer publish that would push my latest version to packagist. The second best would be any official way to publish something else then my GitHub main branch.

It's entirely possible that I’m making wrong assumptions here. But I'm certainly not the only one 😄 ...so I'd be extremely grateful for any guidance. Or even an open discussion about ways to improve the overall experience of composer and packagist.

[stof]

the reason for having a single version of a package installed at a time is because PHP does not work with a concept of modules exporting things. Instead, all classes are defined in the same naming space, so we cannot have 2 different versions of the class Monolog\Logger loaded in the process. That's the reason why Composer forbids installing the same package multiple times (while npm does not).

All of this would be so much simpler if there were an "official" way to distribute packages with a build step to Packagist. Similar to npm publish for the JavaScript world. My dream-come-true would be a composer publish that would push my latest version to packagist. The second best would be any official way to publish something else then my GitHub main branch.

Packagist does not store any code. It only stores metadata about packages. So there is no way to push code to Packagist (btw, this is the main point making it possible for Jordi and Niels to operate packagist.org, compared to the infrastructure of npmjs.com which is backed by Microsoft).

What you can do is something similar to what phpstan does: use one GitHub repository for your source code where you develop things (ignoring anything you want to ignore) and a second repository in which the CI would automatically push the build output. That second repository would be the one registered in packagist.
Note that this is also how frameworks using mono-repos (Symfony, Laravel, etc...) are working: they have some automated tooling splitting each package into a separate GitHub repository and those repositories are registered in packagist.org.

[Seldaek]

Yeah.. WP is a problem because it doesn't embrace Composer and then you end up with these plugins conflicting with each other.. If you had one composer.json for all plugins and their deps (like roots/bedrock does) this would not happen, but alas..

So right now yeah there are a few hacky ways to do this, and we are well aware of the problem and considering ways to fix it by hosting archives ourselves, but I cannot tell you when we get there.. So if you need a solution now then go the phpstan route or something else. And hopefully at some point we'll have better archive support.

That being said, with scoped dependencies while you're right that disk size is not a huge problem, you do have much higher memory usage if you use the same library scoped into 5 namespaces. Classes will be kept in opcache 5 times, etc. It solves some problem for WP, but it's really not the solution, and packages having a simple git = zip relationship are much better IMO both for efficiency and from a code review / security perspective.

[hirasso]

Thank you @stof and @Seldaek for taking the time to answer! Much appreciated. And a big thank you to Jordi and Niels for maintaining Packagist. It's an amazing service to the whole community!! 🙏

Things I learned from your replies

• The recommended way for releasing a package with a build step is to maintain two repos – one for development, the other for distribution
• Having multiple versions of the same package in one project is slower due to more more memory consumption

Things I'm still asking myself

• When exactly does Packagist consider something a new release? Anytime I tag a commit, no matter on what branch?
• Does my idea with a dist branch for Packagist in the same repo (as opposed to two separate repos) make any sense at all?

Regarding the dependency problem: WP is not really the issue

If you had one composer.json for all plugins and their deps (like roots/bedrock does) this would not happen, but alas..

I'm not sure I agree here, or I'm missing something. With one composer.json, dependency conflicts are also very real, as described in my original post. The following happened in a completely composer-managed WordPress project:

For example, I recently encountered an issue where I couldn't install a package due to a strict dependency on psr/log, which conflicted with the version required by my already-installed version of monolog/logger. When conflicts like this occur, users depend on package maintainers to quickly release a fix (Maybe I'm wrong with this assumption!). Otherwise, they may need to maintain their own fork, which will lead to ecosystem fragmentation — something I'd like to avoid.

So a fully composer.json-managed project doesn't solve dependency conflicts – it just catches them earlier, by simply not being able to install a plugin/package. This is of course better than throwing errors at runtime – but it's still functionality I cannot use then. This one is not WordPress specific. It will also happen in ecosystems like Statamic or Kirby. If I need to run a different major version of the same library in the same project, there is no way around scoping that I'm aware of. Poorer performance or not...

God I would love to meet in a call sometime for a coffee and talk about these things. Writing it all out is exhausting, especially as I'm not a native English speaker 😅

[stof]

Composer (and by extension Packagist) has 2 different kinds of versions for packages:

• released versions, corresponding to git tags (and as such expected to be immutable), which can map to several stability levels (stable, RC, beta or alpha)
• dev versions, corresponding to git branches, which are moving targets and are always mapped to the dev stability.

If I need to run a different major version of the same library in the same project, there is no way around scoping that I'm aware of.

As I said previously, this is not a restriction added by Composer itself. PHP makes it impossible to have 2 different definitions for a given class/interface. If you were installing psr/log twice in different version, the first load of the LoggerInterface would then be used by all code using that interface, ignoring the second definition of the interface being present (or triggering a fatal error due to duplicate class name if you try to force loading the second definition).

[hirasso]

Thank you @stof . Seems like I really need to rtfm for composer thoroughly 😅

[Seldaek]

So a fully composer.json-managed project doesn't solve dependency conflicts – it just catches them earlier, by simply not being able to install a plugin/package.

Yes, you get a resolution conflict sometimes, and then you have to maybe settle for an older version of some software, or nag someone to update their requirements so that things become compatible. But then in the end composer resolves all packages to a single version that is compatible with all other installed libraries, and that is much better than scoping everything IMO. And it does seem to work for everyone using Composer, and it causes way less pain than the wordpress model of every plugin installing their own stuff but then trying to coexist in a single process.

[hirasso]

Thanks for taking the time and explaining your perspective!