Setup CI infra to run DevTools tests against multiple React versions #19371

bvaughn · 2020-07-15T15:26:15Z

PR #19108 caused some Suspense-related DevTools regressions (more info available on #19368) which we did not catch because of the fact that DevTools tests are only run against the version of React in master.

We should follow the precedent of the regression fixtures tests and have CI run DevTools tests against multiple React versions, including v15, all v16 minors, and the current HEAD of master branch.

Setting this up will involve several things:

Infra to checkout older React packages and run tests against them.
Some form of gating so that we can account for expected differences in Store snapshots between React versions.
Some form of gating so that we can avoid running tests against invalid combinations of features and versions (e.g. don't test for Suspense in a version of React that didn't include that component yet).

The text was updated successfully, but these errors were encountered:

RitikPandey1 · 2020-09-19T07:24:22Z

I would like to take this as my first issue , please guide me from where I can start

bvaughn · 2020-09-21T03:15:12Z

@RitikPandey1 You are welcome to work on this issue if you would be interested!

Unfortunately, it would have to be self-guided work. I don't have any more of an outline for what needs to be done than what is written in the issue.

RitikPandey1 · 2020-09-21T03:18:48Z

Okay, @bvaughn I will try to figure this out .

MoSattler · 2020-09-26T15:01:00Z

@bvaughn, not sure if this is easily done with checking out older code.
I tried it here, and I keep running into issues where code is either coupled to newer dependencies, or the newer tests are coupled to the newer versions of the code to be tested. See test results here.

Another issue is, that older checkouts have too restrictive devEngines requirements to be installed with modern node, see this.

MoSattler · 2020-09-26T15:06:03Z

Also tried it by just installing the old dependencies via npm, with similar results. Maybe some of the tests are testing internals, and hence are coupled to internals of the latest code?

bvaughn · 2020-09-28T13:23:16Z

The build-for-devtools command builds an experimental build (RELEASE_CHANNEL=experimental) of React to be bundled along with the DevTools extension. Generally this build always comes from the master branch, so DevTools is built with the latest version of React.

This task is about using the DevTools to connect to applications written with multiple (usually older) versions of React. Replacing the build-for-devtools command doesn't seem like the right approach here, since that would also be changing which version of React the DevTools is actually running with.

The precedent here would be our "legacy" tests:
https://github.com/facebook/react/tree/master/packages/react-devtools-shared/src/__tests__/legacy

These tests using NPM aliasing to run an older version of React with DevTools (which is still built with the latest experimental build):

react/packages/react-devtools-shared/src/__tests__/legacy/inspectElement-test.js

Lines 76 to 84 in 480626a

    
           // Redirect all React/ReactDOM requires to the v15 UMD. 
        
           // We use the UMD because Jest doesn't enable us to mock deep imports (e.g. "react/lib/Something"). 
        
           jest.mock('react', () => jest.requireActual('react-15/dist/react.js')); 
        
           jest.mock('react-dom', () => 
        
             jest.requireActual('react-dom-15/dist/react-dom.js'), 
        
           ); 
        
           React = require('react'); 
        
           ReactDOM = require('react-dom');

MoSattler · 2020-09-30T11:50:21Z

Thanks for the elaboration, I will check it out!

ktfth · 2021-03-30T19:51:45Z

@MoSattler can i help you with that task?

bvaughn · 2021-03-30T19:52:58Z

This task is still up for grabs, if you're interested, @ktfth

ktfth · 2021-03-30T19:56:20Z

@bvaughn I'm totally interested on that task, can you update me with the needs to complete them?

bvaughn · 2021-03-30T19:59:50Z

Please feel free to work on it then!

Unfortunately other than my comment above (#19371 (comment)) this is a self-guided task. I don't have the bandwidth to provide much support for it.

ktfth · 2021-03-30T20:24:51Z

I'm gonna work to get all information needed. Thank you

ktfth · 2021-03-30T21:23:31Z

@bvaughn I have some question after investigate the process mentioned on #19371 (comment), the test need to be runned just on legacy code and for each version of react?

bvaughn · 2021-03-31T00:47:12Z

I don't know what you mean by "just legacy code"

Ideally probably at least each minor version of React

ktfth · 2021-03-31T16:19:15Z

The legacy test code showed by you on comment, do you remember?

ktfth · 2021-03-31T16:44:46Z

@bvaughn Maybe is not clear what is need to do on that task, can elaborate more about the case is need to cover with the setup?

bvaughn · 2021-03-31T17:09:21Z

Was referring to my comment above: #19371 (comment)

bvaughn · 2021-03-31T17:31:38Z

I don't have a lot of time to go into much detail here, but the idea is that we currently run automated DevTools tests against two flavors of React:

The latest (head of master branch)
The latest release of React 15 (legacy tests)

Ideally we would also run against every minor release of 16. There are a couple of challenges here:

How to install and run the tests against each release
How to gate which tests don't apply to all v16 releases (e.g. b'c it's a feature not added until a later version).

ktfth · 2021-03-31T18:08:54Z

@bvaughn Great this clarify the work, thank you

ktfth · 2021-03-31T20:43:00Z

I have made some tests here to create a case on CI there is the result first change, second change and the pipeline

ktfth · 2021-03-31T20:45:17Z

I'm gonna update here to you @bvaughn can review my changes and get the job done

bvaughn · 2021-03-31T20:47:33Z

Okay. Keep me posted when your PR is ready for a look 👍🏼

ktfth · 2021-03-31T21:09:53Z

Now i need to reproduce the same tests of legacy for other versions like 16 for every minor?

ktfth · 2021-03-31T21:28:29Z

Sorry about a lot of questions, but you have some example to give on that part?

bvaughn · 2021-04-01T00:59:47Z

Sorry about a lot of questions, but you have some example to give on that part?

No, sorry.

ktfth · 2021-04-01T18:27:29Z

Some updates on config based on @MoSattler work to test every version of react with the result on ci. The limits of service was reached. @bvaughn how can we continue?

bvaughn · 2021-04-01T18:31:44Z

Sorry @ktfth, but I don't have much bandwidth to support this task. Unfortunately it has to be self-guided 😦 I'll make time to review a PR but I don't think I can be involved in the troubleshooting phase at the moment.

ktfth · 2021-04-01T19:24:05Z

@bvaughn I'm gonna make a PR, thank you so much

ktfth · 2021-04-01T19:47:40Z

@bvaughn The PR is #21166 for this issue

hardikshah197 · 2021-05-04T10:45:17Z

Can I take up this issue? If it isn't fixed up yet @bvaughn

bvaughn · 2021-05-04T13:25:31Z

@hardikshah197 If you'd like to take a look at #21166 and maybe lend a hand?

hardikshah197 · 2021-05-04T17:25:53Z

Sure, I'll give it a look on and try it out. Thanks

bvaughn added Component: Build Infrastructure Component: Developer Tools labels Jul 15, 2020

bvaughn assigned bvaughn and acdlite Jul 18, 2020

bvaughn added the good first issue label Sep 3, 2020

MoSattler mentioned this issue Sep 26, 2020

Running devtool tests against multiple versions of react #19912

Closed

bvaughn unassigned bvaughn and acdlite May 5, 2021

facebook / react

Setup CI infra to run DevTools tests against multiple React versions #19371

Setup CI infra to run DevTools tests against multiple React versions #19371

bvaughn commented Jul 15, 2020

RitikPandey1 commented Sep 19, 2020

bvaughn commented Sep 21, 2020

RitikPandey1 commented Sep 21, 2020

MoSattler commented Sep 26, 2020 •

edited

MoSattler commented Sep 26, 2020 •

edited

bvaughn commented Sep 28, 2020

MoSattler commented Sep 30, 2020 •

edited

ktfth commented Mar 30, 2021

bvaughn commented Mar 30, 2021

ktfth commented Mar 30, 2021

bvaughn commented Mar 30, 2021

ktfth commented Mar 30, 2021

ktfth commented Mar 30, 2021

bvaughn commented Mar 31, 2021

ktfth commented Mar 31, 2021 •

edited

ktfth commented Mar 31, 2021

bvaughn commented Mar 31, 2021

bvaughn commented Mar 31, 2021

ktfth commented Mar 31, 2021

ktfth commented Mar 31, 2021 •

edited

ktfth commented Mar 31, 2021

bvaughn commented Mar 31, 2021

ktfth commented Mar 31, 2021

ktfth commented Mar 31, 2021

bvaughn commented Apr 1, 2021

ktfth commented Apr 1, 2021

bvaughn commented Apr 1, 2021 •

edited

ktfth commented Apr 1, 2021

ktfth commented Apr 1, 2021 •

edited

hardikshah197 commented May 4, 2021 •

edited

bvaughn commented May 4, 2021

hardikshah197 commented May 4, 2021

facebook / react

Setup CI infra to run DevTools tests against multiple React versions #19371

Setup CI infra to run DevTools tests against multiple React versions #19371

Comments

bvaughn commented Jul 15, 2020

RitikPandey1 commented Sep 19, 2020

bvaughn commented Sep 21, 2020

RitikPandey1 commented Sep 21, 2020

MoSattler commented Sep 26, 2020 • edited

MoSattler commented Sep 26, 2020 • edited

bvaughn commented Sep 28, 2020

MoSattler commented Sep 30, 2020 • edited

ktfth commented Mar 30, 2021

bvaughn commented Mar 30, 2021

ktfth commented Mar 30, 2021

bvaughn commented Mar 30, 2021

ktfth commented Mar 30, 2021

ktfth commented Mar 30, 2021

bvaughn commented Mar 31, 2021

ktfth commented Mar 31, 2021 • edited

ktfth commented Mar 31, 2021

bvaughn commented Mar 31, 2021

bvaughn commented Mar 31, 2021

ktfth commented Mar 31, 2021

ktfth commented Mar 31, 2021 • edited

ktfth commented Mar 31, 2021

bvaughn commented Mar 31, 2021

ktfth commented Mar 31, 2021

ktfth commented Mar 31, 2021

bvaughn commented Apr 1, 2021

ktfth commented Apr 1, 2021

bvaughn commented Apr 1, 2021 • edited

ktfth commented Apr 1, 2021

ktfth commented Apr 1, 2021 • edited

hardikshah197 commented May 4, 2021 • edited

bvaughn commented May 4, 2021

hardikshah197 commented May 4, 2021

MoSattler commented Sep 26, 2020 •

edited

MoSattler commented Sep 26, 2020 •

edited

MoSattler commented Sep 30, 2020 •

edited

ktfth commented Mar 31, 2021 •

edited

ktfth commented Mar 31, 2021 •

edited

bvaughn commented Apr 1, 2021 •

edited

ktfth commented Apr 1, 2021 •

edited

hardikshah197 commented May 4, 2021 •

edited