Add torch script backend and model specific features
#1450
Conversation
…es manifested by weights provider
PawelPeczek-Roboflow
requested review from
grzegorz-roboflow,
yeldarby,
probicheaux and
hansent
as code owners
| @@ -12,7 +13,15 @@ | |||
| EMBEDDING_TASK = "embedding" | |||
|
|
|||
|
|
|||
| REGISTERED_MODELS: Dict[Tuple[ModelArchitecture, TaskType, BackendType], LazyClass] = { | |||
| @dataclass(frozen=True) | |||
| class RegistryEntry: | |||
There was a problem hiding this comment.
would it simplify by having it be just RegistryEntry? with e.g. dataclass:
- module_name
- class_name
- Optional[model_feature]
then below in the list contributors don't have to choose or wonder about difference between using LazyClass and RegistryEntry. could instantiate LazyClass or resolve directly in resolve_model_class if we don't do it elsewhere?
inference_experimental/inference_exp/models/auto_loaders/models_registry.py
Show resolved
Hide resolved
| @@ -49,8 +50,10 @@ def rank_model_packages( | |||
| # discarded in the previous stage. | |||
There was a problem hiding this comment.
re-reading the whole comment her made me think maybe at some point might consider giving a mechanism to configure ranking preference. Maybe configuring explicitly on AutoModel by passing constraints is already enough (but only gives control over filtering, not ranking). Just a thought not something we need to worry about now.
…ences to be selected
|
@PawelPeczek-Roboflow Are these changes strictly tied to new version of registry service? wondering if we can merge / merge partially to avoid merge conflicts. |
… rid of old (environment.json) based pre-processing for all models
…ne inputs and outputs(when applicable)
…n and add segmentation-models-pytorch to dependencies
Description
inference-expsupport for recent extensions introduced to RF API weights-provider:torch-scriptbackendmodelFeaturesmanifested for model packages to denote special (and optional) features of the model that this package bring - implemented in forward compatible mode - staleinference-expshould automatically reject the model packages for a specific backend which was registered with features newer than known in the time wheninference-expmodel implementation was created - this should improve stability of packages selected while auto-negotiation.Type of change
Please delete options that are not relevant.
How has this change been tested, please provide a testcase or example of how you tested the change?
Any specific deployment considerations
For example, documentation changes, usability, usage/costs, secrets, etc.
Docs