Skip to main content

​
Object Detection Model Overview

Object detection models identify and locate objects within images or videos by assigning them specific bounding boxes. Applicable Metrics: Accuracy, Euclidian Distance (embeddings) Click here for all valid model types and metric combinations.

​
Object Detection Code Example

Example Row
image_vectorimage_linkprediction_bboxesactual_bboxesprediction_categoriesactual_categoriesactual_super_categoriesprediction_scoresTimestamp
[0.24713118374347687, 0.7061651349067688, 1.12...```javascript
”https://link-to-my-image.png”
```[[50.43, 109.49, 538.21...[[55.39, 107.72, 539.25, 362.9], [554.41, 194....[bus][bus, person, person][vehicle, person, person][0.9997552]```
1618590882

```python
embedding_feature_column_names={
    "image_embedding": EmbeddingColumnNames(
        vector_column_name="image_vector",
        link_to_data_column_name="url"
    )
}
object_detection_prediction_column_names=ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name="prediction_bboxes",
    categories_column_name="prediction_categories",
    scores_column_name="prediction_scores"
)
object_detection_actual_column_names=ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name="actual_bboxes",
    categories_column_name="actual_categories",
)

# feature & tag columns can be optionally defined with typing:
tag_columns = TypedColumns(
    inferred=["name"],
    to_int=["zip_code", "age"]
)

# Defina the Schema, including embedding information
schema = Schema(
    prediction_id_column_name="prediction_id",
    timestamp_column_name="prediction_ts",
    tag_column_names=tag_columns,
    embedding_feature_column_names=embedding_feature_column_names,
    object_detection_prediction_column_names=object_detection_prediction_column_names,
    object_detection_actual_column_names=object_detection_actual_column_names,
)

response = arize_client.log(
    model_id= "CV-object-detection",
    model_version= "v1",
    model_type=ModelTypes.OBJECT_DETECTION,
    environment=Environments.PRODUCTION,
    dataframe=df,
    schema=schema
)
https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/gc.ico

Google Colaboratory
For more details on Python Batch API Reference, visit here:

​
Object Detection Prediction & Actual Values

Arize supports logging object detection prediction and actual values using the ObjectDetectionColumnNames object, which can be assigned to the prediction/actual schema parameters, object_detection_prediction_column_names and object_detection_actual_column_names.Object prediction or actual declaration is required to use the object detection model type in Arize.
class ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name: str
    categories_column_name: str
    scores_column_name: Optional[str] = None # actual ground truth labels wont have scores
)

​
Embedding Features

In addition to object detection prediction and actual values, Arize supports logging the embedding features associated with the images in an object detection model using the EmbeddingColumnNames object.
  • The vector_column_name should be the name of the column where the embedding vectors are stored. The embedding vector is the dense vector representation of the unstructured input. ⚠️ Note: embedding features are not sparse vectors.
  • The link_to_data_column_name should be the name of the column where the URL links to the source images are stored. 
{ 
    "embedding_display_name": EmbeddingColumnNames(
        vector_column_name="image_vector", 
        link_to_data_column_name="image_link" 
    ) 
}
See here for more information on embeddings and options for generating them.