[core] Move the internal packages from docs/packages

[michaldudak]

This is a follow-up to #35244 (comment)

Yarn docs explicitly state that nested workspaces are not supported. This PR moves all the packages from /docs/packages to the /packages directory. It also moves the language configuration to docs/config to be better accessible.

[mui-bot]

	Messages
📖	Netlify deploy preview: https://deploy-preview-35305--material-ui.netlify.app/

No bundle size changes

Generated by 🚫 dangerJS against e1c82b3

[mnajdova]

The new structure makes sense to me 👍 I would leave to the infra team to do the final review :)

[michaldudak]

@oliviertassinari @mbrookes As the location of the feedback package will change, are there any changes required to the deployment of the lambda on AWS? Or is it done manually?

[mbrookes]

@michaldudak It's deployed from the local machine, so the package location doesn't matter.

[Janpot]

I notice this is cjs so it can be used in next.config.js but also imported in application code as ESM. We could convert to next.config.mjs and author this module in ESM

[michaldudak]

I haven't yet converted the scripts that use it to ESM. It is indeed imported in few places but only in TypeScript files that are transpiled to CJS. After I convert the scripts to ESM, I can update this config file as well.

[oliviertassinari]

I'm late to the party, but it makes sense. Keeping things flat, with an appropriate prefix convention, sounds easier to reason with. e.g. we plan to move https://github.com/mui/mui-x/tree/ac685cda3ed6ba19557246b4d580b60e34bc32dc/packages/grid one level up. These two packages were introduced in #22885, and #26774, nothing to report there.

---

One thing I wonder about is the private vs. public aspect. The packages folder could suggest that these folders are meant for npm packages (public), but there are a good number that aren't meant to be public npm packages:

• api-docs-builder
• docs-utilities
• feedback
• markdown
• markdownlint-rule-mui
• mui-docs
• netlify-plugin-cache-docs
• typescript-to-proptypes

I suspect that it could be great to:

1. Clarity. Move these folders to a different place. It could help not confuse the developers that browse the source of public npm packages. The alternative is to have a naming convention like prefixing them with private-.
2. Reduce degrees of freedom. Have as many as possible of dependencies in the root package.json to minimize version duplication. I mean, as long as we can get away with it, it's usually obvious when it breaks down, e.g. the pain around the need for a progressive version upgrade. I fail to see how the "lodash": "^4.17.21" dependency inside packages/markdown makes sense.
3. KISS. Remove the package.json in all the folders where there is no need for custom dependencies or scripts.
4. Reduce degrees of freedom. Merge as many as possible of the folders together, up until the point where we see a concrete problem with bundling too much. For example, maybe modules/waterfall, api-docs-builder, docs-utilities, mui-docs, typescript-to-proptypes are meant to be in the same place, as a generic internal source folder, everything flat.

[michaldudak]

Clarity. Move these folders to a different place. It could help not confuse the developers that browse the source of public npm packages. The alternative is to have a naming convention like prefixing them with private-.

This is a good point. Having internal packages separately could help the readers know in which context they are.

Have as many as possible of dependencies in the root package.json to minimize version duplication

I was actually thinking about the very opposite - to remove as many dependencies from the root as possible (even to the point of having all dependencies within yarn workspaces). This approach is recommended by Yarn (it warns when trying to install dependencies globally). It also promotes building more modular packages and would help if we ever wanted to move packages between repos (see https://yarnpkg.com/features/workspaces#what-does-it-mean-to-be-a-workspace).

Merge as many as possible of the folders together, up until the point where we see a concrete problem with bundling too much

My personal preference is also the opposite. I very much prefer smaller packages with a single responsibility and well-defined exports.

[oliviertassinari]

to remove as many dependencies from the root as possible (even to the point of having all dependencies within yarn workspaces)

@michaldudak Do you have a specific problem in mind to solve?

It feels a bit like the monolith vs. micro-services tradeoff. What I learned trying this move to micro-services in the past (at a small scale ~10 services) is that the default should be a monolith, and nothing should be split until there is a clear problem. By "clear problem", I mean for instance hardware constraint on how much memory there is to load for a dependency. The rest, like coding style, are subjective that can already be solved with clear folders and files organization.

This approach is recommended by Yarn

This raises the question of what problem did we try to solve when we introduced yarn workspace? It seems to have started with fca1061 but I can't remember. The problems yarn workspace seems to solve today:

1. when running the docs, use the dependencies of the range specified in the npm packages that we publish. It makes the localhost docs experience closer to what developers experience.
2. being able to have scripts per folder, simpler
3. when doing heavy package upgrades, being able to do it more iteratively
4. ? I can't think of anything else

if we ever wanted to move packages between repos

It's probably a rare event. I think that it's better to not anticipate this.

[michaldudak]

Do you have a specific problem in mind to solve?

Reduce the cognitive load by having small, self-contained packages.

the default should be a monolith, and nothing should be split until there is a clear problem

From my experience, the default should be a modular monolith, which is a single deployment unit built with logically separate components. That's because when you actually need to split, breaking up a tangled monolith is usually a hard and expensive exercise. We see this problem today when other teams try to use the tools that are tied to the Core repo.

can already be solved with clear folders and files organization.

Having small encapsulated packages encourages having a clear organization of folders and makes it harder to import arbitrary stuff from files across the whole repo, as was the case with our docs generation scripts.

This raises the question of what problem did we try to solve when we introduced yarn workspace?

I don't know the reasons you had back then, but it seems that workspaces (or similar concepts in other package managers) became a de facto standard for working with monorepos.

[Janpot]

This raises the question of what problem did we try to solve when we introduced yarn workspace?

For me it's usually to link packages together locally. And to have a basic task runner that is aware of dependency topology (though it's a bit limited sometimes in that regard)