Add name and version #18796

tchaton · 2023-10-13T17:52:27Z

What does this PR do?

This PR adds name and version to the Cache and add a new Dataset. Plus, some folder are being renamed.

Fixes #<issue_number>

Before submitting

Was this discussed/agreed via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--18796.org.readthedocs.build/en/18796/

cc @Borda

…into processor

github-actions · 2023-10-13T17:54:33Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 lightning_data: CPU workflow

Check ID	Status
data-cpu (macOS-11, lightning, 3.10, 2.1)	success	✅
data-cpu (ubuntu-20.04, lightning, 3.10, 2.1)	success	✅
data-cpu (windows-2022, lightning, 3.10, 2.1)	success	✅

These checks are required after the changes to src/lightning/data/__init__.py, src/lightning/data/streaming/__init__.py, src/lightning/data/streaming/cache.py, src/lightning/data/streaming/compression.py, src/lightning/data/streaming/config.py, src/lightning/data/streaming/constants.py, src/lightning/data/streaming/dataloader.py, src/lightning/data/streaming/dataset.py, src/lightning/data/streaming/dataset_optimizer.py, src/lightning/data/streaming/downloader.py, src/lightning/data/streaming/reader.py, src/lightning/data/streaming/sampler.py, src/lightning/data/streaming/serializers.py, src/lightning/data/streaming/writer.py, tests/tests_data/cache/test_cache.py, tests/tests_data/cache/test_data_optimizer.py, tests/tests_data/cache/test_sampler.py, tests/tests_data/cache/test_serializer.py, tests/tests_data/cache/test_writer.py.

🟢 lightning_app: Tests workflow

Check ID	Status
app-pytest (macOS-11, lightning, 3.8, latest)	success	✅
app-pytest (macOS-11, lightning, 3.8, oldest)	success	✅
app-pytest (macOS-11, app, 3.9, latest)	success	✅
app-pytest (macOS-12, app, 3.11, latest)	success	✅
app-pytest (ubuntu-20.04, lightning, 3.8, latest)	success	✅
app-pytest (ubuntu-20.04, lightning, 3.8, oldest)	success	✅
app-pytest (ubuntu-20.04, app, 3.9, latest)	success	✅
app-pytest (ubuntu-22.04, app, 3.11, latest)	success	✅
app-pytest (windows-2022, lightning, 3.8, latest)	success	✅
app-pytest (windows-2022, lightning, 3.8, oldest)	success	✅
app-pytest (windows-2022, app, 3.8, latest)	success	✅
app-pytest (windows-2022, app, 3.11, latest)	success	✅

These checks are required after the changes to requirements/app/app.txt.

🟢 lightning_app: Examples

Check ID	Status
app-examples (macOS-11, lightning, 3.9, latest)	success	✅
app-examples (macOS-11, lightning, 3.9, oldest)	success	✅
app-examples (macOS-11, app, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, oldest)	success	✅
app-examples (ubuntu-20.04, app, 3.9, latest)	success	✅
app-examples (windows-2022, lightning, 3.9, latest)	success	✅
app-examples (windows-2022, lightning, 3.9, oldest)	success	✅
app-examples (windows-2022, app, 3.9, latest)	success	✅

These checks are required after the changes to requirements/app/app.txt.

🟢 lightning_app: Docs

Check ID	Status
docs-make (app, doctest)	success	✅
docs-make (app, html)	success	✅

These checks are required after the changes to requirements/app/app.txt.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to requirements/app/app.txt, src/lightning/data/__init__.py, src/lightning/data/streaming/__init__.py, src/lightning/data/streaming/cache.py, src/lightning/data/streaming/compression.py, src/lightning/data/streaming/config.py, src/lightning/data/streaming/constants.py, src/lightning/data/streaming/dataloader.py, src/lightning/data/streaming/dataset.py, src/lightning/data/streaming/dataset_optimizer.py, src/lightning/data/streaming/downloader.py, src/lightning/data/streaming/reader.py, src/lightning/data/streaming/sampler.py, src/lightning/data/streaming/serializers.py, src/lightning/data/streaming/writer.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.11)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.11)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.11)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.11)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.11)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.11)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.11)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.11)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.11)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.11)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.11)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.11)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.11)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.11)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.11)	success	✅

These checks are required after the changes to src/lightning/data/__init__.py, src/lightning/data/streaming/__init__.py, src/lightning/data/streaming/cache.py, src/lightning/data/streaming/compression.py, src/lightning/data/streaming/config.py, src/lightning/data/streaming/constants.py, src/lightning/data/streaming/dataloader.py, src/lightning/data/streaming/dataset.py, src/lightning/data/streaming/dataset_optimizer.py, src/lightning/data/streaming/downloader.py, src/lightning/data/streaming/reader.py, src/lightning/data/streaming/sampler.py, src/lightning/data/streaming/serializers.py, src/lightning/data/streaming/writer.py, requirements/app/app.txt.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

codecov · 2023-10-13T21:15:32Z

Codecov Report

Merging #18796 (890b2be) into master (2306429) will decrease coverage by 33%.
The diff coverage is 86%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #18796      +/-   ##
==========================================
- Coverage      83%      50%     -33%     
==========================================
  Files         440      435       -5     
  Lines       34975    34886      -89     
==========================================
- Hits        29057    17445   -11612     
- Misses       5918    17441   +11523

src/lightning/data/streaming/cache.py

Co-authored-by: thomas <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 3f86ad7)

thomas and others added 30 commits October 9, 2023 18:05

update

8b440e4

update

aa96493

update

eb4a80d

update

eb2a90e

update

e745c85

update

1be2a9e

Merge branch 'processor' of https://github.com/Lightning-AI/lightning …

63ae4dd

…into processor

update

fe2ee8f

update

d1cfd5c

update

a34b8eb

update

5e5ac9f

Merge branch 'processor' of https://github.com/Lightning-AI/lightning …

194adb5

…into processor

update

44ccd47

update

030feff

update

ddd4d8f

update

b987d77

update

60adfd1

update

2b9f010

update

2b3c483

update

46caaab

update

83a7715

update

e2eba01

update

2ede0e9

update

4064a31

update

d10eadd

Merge branch 'processor' of https://github.com/Lightning-AI/lightning …

76af00e

…into processor

update

5625a6b

update

7bdd2ef

update

86430b9

update

e9a7d26

thomas added 5 commits October 13, 2023 17:27

update

812233c

update

1fc4222

update

5c968c9

update

c7f722b

update

d43fc07

tchaton requested review from lantiga, awaelchli, ethanwharris, nohalon, justusschock and Borda as code owners October 13, 2023 17:52

github-actions bot added app (removed) Generic label for Lightning App package dependencies Pull requests that update a dependency file labels Oct 13, 2023

mergify bot added the has conflicts label Oct 13, 2023

update

9354727

mergify bot removed the has conflicts label Oct 13, 2023

thomas added 2 commits October 13, 2023 20:59

update

1c44719

update

ac18d05

awaelchli approved these changes Oct 14, 2023

View reviewed changes

src/lightning/data/streaming/cache.py Show resolved Hide resolved

ethanwharris approved these changes Oct 14, 2023

View reviewed changes

mergify bot added the ready PRs ready to be merged label Oct 14, 2023

update

890b2be

justusschock approved these changes Oct 16, 2023

View reviewed changes

src/lightning/data/streaming/cache.py Show resolved Hide resolved

tchaton merged commit 3f86ad7 into master Oct 16, 2023

tchaton deleted the add_name_and_version branch October 16, 2023 13:33

Borda pushed a commit that referenced this pull request Nov 2, 2023

Add name and version (#18796)

5d9b25b

Co-authored-by: thomas <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 3f86ad7)

awaelchli added this to the 2.1.x milestone Nov 3, 2023

lantiga pushed a commit that referenced this pull request Nov 6, 2023

Add name and version (#18796)

0c8a1f5

Co-authored-by: thomas <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 3f86ad7)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add name and version #18796

Add name and version #18796

Uh oh!

tchaton commented Oct 13, 2023 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Oct 13, 2023 •

edited

Loading

Uh oh!

codecov bot commented Oct 13, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Add name and version #18796

Add name and version #18796

Uh oh!

Conversation

tchaton commented Oct 13, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

PR review

Uh oh!

github-actions bot commented Oct 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚡ Required checks status: All passing 🟢

Groups summary

Uh oh!

codecov bot commented Oct 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tchaton commented Oct 13, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Oct 13, 2023 •

edited

Loading

codecov bot commented Oct 13, 2023 •

edited

Loading