You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix: engine initializes optimizer attributes at the beginning (#7410)
As in `destroy`, `self.optimizer` is called, but the error out calling
to `destroy` can happen in `__init__`, even before optimizer and
scheduler is configured. So we need to move `self.optimizer` to the top
to avoid triggering another exception.
e.g.:
```logs
File "deepspeed/runtime/engine.py", line 453, in _configure_tensor_parallel_states
assert self.zero_optimization_stage(
AssertionError: Currently, the compatibility between 'autotp' and 'zero_stage = 3' has not been validated
Exception ignored in: <function DeepSpeedEngine.__del__ at 0x1516c0610820>
Traceback (most recent call last):
File "deepspeed/runtime/engine.py", line 509, in __del__
self.destroy()
File "deepspeed/runtime/engine.py", line 512, in destroy
if self.optimizer is not None and hasattr(self.optimizer, 'destroy'):
File "deepspeed/runtime/engine.py", line 621, in __getattr__
raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'DeepSpeedEngine' object has no attribute 'optimizer'
```
Signed-off-by: Hollow Man <[email protected]>
0 commit comments