Skip to content

Comments

[SPARK-37757][BUILD] Enable Spark test scheduled job on ARM runner#35049

Closed
Yikun wants to merge 1 commit intoapache:masterfrom
Yikun:SPARK-37757
Closed

[SPARK-37757][BUILD] Enable Spark test scheduled job on ARM runner#35049
Yikun wants to merge 1 commit intoapache:masterfrom
Yikun:SPARK-37757

Conversation

@Yikun
Copy link
Member

@Yikun Yikun commented Dec 29, 2021

What changes were proposed in this pull request?

This patch adds the scheduled job on ARM runner.

Why are the changes needed?

Migrate Spark Arm Job from Jenkins to GitHub Actions.

Does this PR introduce any user-facing change?

NO

How was this patch tested?

Current runner version: '2.285.1'
Runner name: 'ubuntu-20.04-arm64'
Runner group name: 'Default'
Machine name: 'yikun-arm'
  • Trigger original x86 test in this patch

@srowen
Copy link
Member

srowen commented Dec 31, 2021

Looks good if it works

@Yikun
Copy link
Member Author

Yikun commented Jan 1, 2022

@srowen Thanks for review. And I am going to add self hosted runner to apache/spark first and then mark this as ready for review. Here is the arm64 result: Yikun#51 .

${{ matrix.java }}-${{ matrix.hadoop }}-coursier-
- name: Install Java ${{ matrix.java }}
uses: actions/setup-java@v1
# actions/setup-java doesn'ts suppport arm64, it had been installed in self-hosted runner.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# actions/setup-java doesn'ts suppport arm64, it had been installed in self-hosted runner.
# actions/setup-java doesn'ts suppport arm64, JDK 8 is pre-installed in the self-hosted runner.

# - SQL component also has Python related tests, for example, IntegratedUDFTestUtils.
# - Yarn has a Python specific test too, for example, YarnClusterSuite.
if: contains(matrix.modules, 'yarn') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-'))
# * We should install one Python that is higher then 3+ for SQL and Yarn because:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/then/than/

# * We should install one Python that is higher then 3+ for SQL and Yarn because:
# - SQL component also has Python related tests, for example, IntegratedUDFTestUtils.
# - Yarn has a Python specific test too, for example, YarnClusterSuite.
# * actions/setup-python doesn't support arm64, it had been installed in self-hosted runner.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# * actions/setup-python doesn't support arm64, it had been installed in self-hosted runner.
# * actions/setup-python doesn't support arm64, Python 3.8 is pre-installed in the self-hosted runner.

@github-actions
Copy link

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

@github-actions github-actions bot added the Stale label May 19, 2022
@github-actions github-actions bot closed this May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants