Rename the TPUSpawnStrategy to XLAStrategy #16781

awaelchli · 2023-02-16T11:33:52Z

What does this PR do?

Renames the TPUSpawnStrategy to XLAStrategy
Renames the strategy nickname from tpu_spawn to xla

The name was never accurate from the start. XLA has always used fork as the default launcher. This was already done in Fabric and now brings the two in line.

This change will be invisible for most users because it is never necessary to select the strategy explicitly when choosing accelerator=tpu.

Part of the Lightning 2.0 clean up.
Note: This will also make sense in the context of #16130

cc @Borda @JackCaoG @steventk-g @Liyang90 @justusschock @awaelchli @carmocca

github-actions · 2023-02-16T11:34:26Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.11)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.11)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py, src/lightning/pytorch/strategies/__init__.py, src/lightning/pytorch/strategies/launchers/xla.py, src/lightning/pytorch/strategies/xla.py, src/lightning/pytorch/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/accelerators/test_tpu.py, tests/tests_pytorch/conftest.py, tests/tests_pytorch/models/test_tpu.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_xla.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/properties/test_estimated_stepping_batches.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs)	success	✅

These checks are required after the changes to src/lightning/pytorch/strategies/__init__.py, src/lightning/pytorch/strategies/launchers/xla.py, src/lightning/pytorch/strategies/xla.py, src/lightning/pytorch/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/accelerators/test_tpu.py, tests/tests_pytorch/conftest.py, tests/tests_pytorch/models/test_tpu.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_xla.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/properties/test_estimated_stepping_batches.py, src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py.

🟢 pytorch_lightning: Azure HPU

Check ID	Status
pytorch-lightning (HPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py, src/lightning/pytorch/strategies/__init__.py, src/lightning/pytorch/strategies/launchers/xla.py, src/lightning/pytorch/strategies/xla.py, src/lightning/pytorch/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/accelerators/test_tpu.py, tests/tests_pytorch/conftest.py, tests/tests_pytorch/models/test_tpu.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_xla.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/properties/test_estimated_stepping_batches.py.

🟢 pytorch_lightning: Azure IPU

Check ID	Status
pytorch-lightning (IPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py, src/lightning/pytorch/strategies/__init__.py, src/lightning/pytorch/strategies/launchers/xla.py, src/lightning/pytorch/strategies/xla.py, src/lightning/pytorch/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/accelerators/test_tpu.py, tests/tests_pytorch/conftest.py, tests/tests_pytorch/models/test_tpu.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_xla.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/properties/test_estimated_stepping_batches.py.

🟢 pytorch_lightning: Docs

Check ID	Status
make-doctest (pytorch)	success	✅
make-html (pytorch)	success	✅

These checks are required after the changes to src/lightning/pytorch/strategies/__init__.py, src/lightning/pytorch/strategies/launchers/xla.py, src/lightning/pytorch/strategies/xla.py, src/lightning/pytorch/trainer/connectors/accelerator_connector.py, docs/source-pytorch/accelerators/tpu_faq.rst, docs/source-pytorch/advanced/strategy_registry.rst, docs/source-pytorch/api_references.rst, docs/source-pytorch/extensions/strategy.rst.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.11)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.12)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.9, 1.11)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.12)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13)	success	✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py, tests/tests_fabric/strategies/test_registry.py.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py, tests/tests_fabric/strategies/test_registry.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py, src/lightning/pytorch/strategies/__init__.py, src/lightning/pytorch/strategies/launchers/xla.py, src/lightning/pytorch/strategies/xla.py, src/lightning/pytorch/trainer/connectors/accelerator_connector.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.10)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.10)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.10)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.10)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.10)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.10)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.10)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.10)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.10)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.10)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.10)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.10)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.10)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.10)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.10)	success	✅

These checks are required after the changes to src/lightning/fabric/connector.py, src/lightning/fabric/strategies/xla.py, src/lightning/pytorch/strategies/__init__.py, src/lightning/pytorch/strategies/launchers/xla.py, src/lightning/pytorch/strategies/xla.py, src/lightning/pytorch/trainer/connectors/accelerator_connector.py.

🟢 link-check

Check ID	Status
markdown-link-check	success	✅

These checks are required after the changes to src/lightning/fabric/CHANGELOG.md, src/lightning/pytorch/CHANGELOG.md.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

for more information, see https://pre-commit.ci

…eature/rename-tpu-spawn

for more information, see https://pre-commit.ci

…eature/rename-tpu-spawn

for more information, see https://pre-commit.ci

codecov · 2023-02-17T00:13:52Z

Codecov Report

Merging #16781 (d6fd689) into master (57c1138) will decrease coverage by 22%.
The diff coverage is 100%.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #16781     +/-   ##
=========================================
- Coverage      82%      59%    -22%     
=========================================
  Files         441      416     -25     
  Lines       31660    31363    -297     
=========================================
- Hits        25888    18603   -7285     
- Misses       5772    12760   +6988

JackCaoG · 2023-02-17T02:09:12Z

Nice! Some one was confused for this naming in https://discuss.pytorch.org/t/why-use-cuda-must-be-0-when-xla-cuda-1/172692 so I think this is a great move!

awaelchli · 2023-02-17T10:23:31Z

Thanks @JackCaoG. With #16130 we should then be able to support accelerator="gpu" + strategy="xla" which would then fully clear up the confusion.

rename the TPUSpawnStrategy

0a138f2

awaelchli requested review from carmocca, justusschock, williamFalcon, edenlightning, lantiga and Borda as code owners February 16, 2023 11:33

github-actions bot added fabric lightning.fabric.Fabric pl Generic label for PyTorch Lightning package labels Feb 16, 2023

awaelchli added accelerator: tpu Tensor Processing Unit refactor breaking change Includes a breaking change and removed fabric lightning.fabric.Fabric pl Generic label for PyTorch Lightning package labels Feb 16, 2023

awaelchli added this to the 2.0 milestone Feb 16, 2023

[pre-commit.ci] auto fixes from pre-commit.com hooks

3bfc4c8

for more information, see https://pre-commit.ci

github-actions bot added fabric lightning.fabric.Fabric pl Generic label for PyTorch Lightning package labels Feb 16, 2023

awaelchli added 3 commits February 16, 2023 12:39

changelog

4525e1d

Merge remote-tracking branch 'origin/feature/rename-tpu-spawn' into f…

0e11076

…eature/rename-tpu-spawn

registry

0b4f7be

awaelchli self-assigned this Feb 16, 2023

fix

8676f60

carmocca approved these changes Feb 16, 2023

View reviewed changes

mergify bot added the has conflicts label Feb 16, 2023

Merge branch 'master' into feature/rename-tpu-spawn

e023493

mergify bot removed the has conflicts label Feb 16, 2023

awaelchli and others added 2 commits February 17, 2023 00:53

merge errors

03fcab9

[pre-commit.ci] auto fixes from pre-commit.com hooks

2134bb1

for more information, see https://pre-commit.ci

awaelchli and others added 3 commits February 17, 2023 00:54

merge errors

774ea34

Merge remote-tracking branch 'origin/feature/rename-tpu-spawn' into f…

e2a1d85

…eature/rename-tpu-spawn

[pre-commit.ci] auto fixes from pre-commit.com hooks

d6fd689

for more information, see https://pre-commit.ci

Borda approved these changes Feb 17, 2023

View reviewed changes

Borda enabled auto-merge (squash) February 17, 2023 01:50

mergify bot added the ready PRs ready to be merged label Feb 17, 2023

Borda merged commit 91e692c into master Feb 17, 2023

Borda deleted the feature/rename-tpu-spawn branch February 17, 2023 02:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Rename the TPUSpawnStrategy to XLAStrategy #16781

Rename the TPUSpawnStrategy to XLAStrategy #16781

Uh oh!

awaelchli commented Feb 16, 2023 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Feb 16, 2023 •

edited

Loading

Uh oh!

codecov bot commented Feb 17, 2023 •

edited

Loading

Uh oh!

JackCaoG commented Feb 17, 2023

Uh oh!

awaelchli commented Feb 17, 2023

Uh oh!

Uh oh!

Rename the TPUSpawnStrategy to XLAStrategy #16781

Rename the TPUSpawnStrategy to XLAStrategy #16781

Uh oh!

Conversation

awaelchli commented Feb 16, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

github-actions bot commented Feb 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚡ Required checks status: All passing 🟢

Groups summary

Uh oh!

codecov bot commented Feb 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

JackCaoG commented Feb 17, 2023

Uh oh!

awaelchli commented Feb 17, 2023

Uh oh!

Uh oh!

awaelchli commented Feb 16, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Feb 16, 2023 •

edited

Loading

codecov bot commented Feb 17, 2023 •

edited

Loading