Skip to search

InferencePool

inference.networking.k8s.io / v1

apiVersion: inference.networking.k8s.io/v1 kind: InferencePool metadata: name: example
View raw schema
apiVersion string
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
kind string
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
metadata object
spec object required
Spec defines the desired state of the InferencePool.
endpointPickerRef object required
EndpointPickerRef is a reference to the Endpoint Picker extension and its associated configuration.
failureMode string
FailureMode configures how the parent handles the case when the Endpoint Picker extension is non-responsive. When unspecified, defaults to "FailClose".
enum: FailOpen, FailClose
group string
Group is the group of the referent API object. When unspecified, the default value is "", representing the Core API group.
pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
minLength: 0
maxLength: 253
kind string
Kind is the Kubernetes resource kind of the referent. Required if the referent is ambiguous, e.g. service with multiple ports. Defaults to "Service" when not specified. ExternalName services can refer to CNAME DNS records that may live outside of the cluster and as such are difficult to reason about in terms of conformance. They also may not be safe to forward to (see CVE-2021-25740 for more information). Implementations MUST NOT support ExternalName Services.
pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$
minLength: 1
maxLength: 63
name string required
Name is the name of the referent API object.
minLength: 1
maxLength: 253
port object
Port is the port of the Endpoint Picker extension service. Port is required when the referent is a Kubernetes Service. In this case, the port number is the service port number, not the target port. For other resources, destination port might be derived from the referent resource or this field.
number integer required
Number defines the port number to access the selected model server Pods. The number must be in the range 1 to 65535.
format: int32
minimum: 1
maximum: 65535
selector object required
Selector determines which Pods are members of this inference pool. It matches Pods by their labels only within the same namespace; cross-namespace selection is not supported. The structure of this LabelSelector is intentionally simple to be compatible with Kubernetes Service selectors, as some implementations may translate this configuration into a Service resource.
matchLabels object required
MatchLabels contains a set of required {key,value} pairs. An object must match every label in this map to be selected. The matching logic is an AND operation on all entries.
targetPorts []object required
TargetPorts defines a list of ports that are exposed by this InferencePool. Every port will be treated as a distinctive endpoint by EPP, addressable as a 'podIP:portNumber' combination.
minItems: 1
maxItems: 8
number integer required
Number defines the port number to access the selected model server Pods. The number must be in the range 1 to 65535.
format: int32
minimum: 1
maximum: 65535
status object
Status defines the observed state of the InferencePool.
parents []object
Parents is a list of parent resources, typically Gateways, that are associated with the InferencePool, and the status of the InferencePool with respect to each parent. A controller that manages the InferencePool, must add an entry for each parent it manages and remove the parent entry when the controller no longer considers the InferencePool to be associated with that parent. A maximum of 32 parents will be represented in this list. When the list is empty, it indicates that the InferencePool is not associated with any parents.
maxItems: 32
conditions []object
Conditions is a list of status conditions that provide information about the observed state of the InferencePool. This field is required to be set by the controller that manages the InferencePool. Supported condition types are: * "Accepted" * "ResolvedRefs"
maxItems: 8
lastTransitionTime string required
lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
message string required
message is a human readable message indicating details about the transition. This may be an empty string.
maxLength: 32768
observedGeneration integer
observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.
format: int64
minimum: 0
reason string required
reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
minLength: 1
maxLength: 1024
status string required
status of the condition, one of True, False, Unknown.
enum: True, False, Unknown
type string required
type of condition in CamelCase or in foo.example.com/CamelCase.
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
maxLength: 316
parentRef object required
ParentRef is used to identify the parent resource that this status is associated with. It is used to match the InferencePool with the parent resource, such as a Gateway.
group string
Group is the group of the referent API object. When unspecified, the referent is assumed to be in the "gateway.networking.k8s.io" API group.
pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
minLength: 0
maxLength: 253
kind string
Kind is the kind of the referent API object. When unspecified, the referent is assumed to be a "Gateway" kind.
pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$
minLength: 1
maxLength: 63
name string required
Name is the name of the referent API object.
minLength: 1
maxLength: 253
namespace string
Namespace is the namespace of the referenced object. When unspecified, the local namespace is inferred. Note that when a namespace different than the local namespace is specified, a ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the ReferenceGrant documentation for details: https://gateway-api.sigs.k8s.io/api-types/referencegrant/
pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
minLength: 1
maxLength: 63

No matches. Try .spec.endpointPickerRef for an exact path