Refactor for Import Util Functions

[yewentao256]

Nit/future work: I think with these new import checks and the above torch version check, it would be nice to pull these into a separate import_utils.py file like Transformers to make it clear for new libraries to add there https://github.com/huggingface/transformers/blob/main/src/transformers/utils/import_utils.py#L386

Originally posted by @mgoin in #20187 (comment)

Thanks @mgoin Sounds a good idea, I will have a pr for this, including all of current import utils we have.