Worked example: CV for product-scale data collection

One member’s draft working taxonomy of computer-vision tasks for product-scale data collection at end-of-life: a worked example of how one corner of the applications map can be mapped in depth. Treat it as a starting point, not settled work.

Taxonomy at a glance

Two axes: physical scale (product, component, material) and information type (identity, quantity, structure, condition). Structure means the bill of components (BoC): which parts a product has and how they are joined. Each filled cell is a CV task with a maturity grade. The grades are mostly negative: no cell reaches strong when the photos are taken at end-of-life. The useful information is in the grades below that. Each section below says what its level covers, gives examples with their papers, and links a demo where one exists.

Scale ↓ / Info type →	Identity	Quantity	Structure (BoC)	Condition
Product	Label OCR + retrieval (P)	Depth / multi-view geometry (P)	Not sensed	Wear / damage grading (E)
Component	Marking OCR + open-vocab naming (P)	Per-part geometry, occluded (P)	Detection (P) / attachment relations (E)	Connector-state grading (U)
Material	Surface BoM classification (P); full BoM an open gap	Derived (geometry × density)	Not sensed	Corrosion / contamination (U)

Maturity grades, from highest to lowest:

• Strong: works across many kinds of product on a benchmark, works on field photos, and is used in practice. No cell is here yet.
• P, partial: works across many kinds of product on a benchmark, but fails on field photos or is not yet used in practice.
• E, emerging-but-narrow: works only for one narrow kind of product.
• U, plausible-but-unvalidated: shown only in principle or in a neighbouring field.
• Not sensed or Derived: no method reads it from a photo; at best it is computed from other cells.

A personal working taxonomy, still in draft, scoped to CV for product-scale data collection. Verdicts and coverage will shift as it develops.

Product-level tasks

What it covers: data recoverable from an image of the whole product.

Examples:

• Identity (partial). OCR metadata extraction from labels (text detection, recognition, then fuzzy match to a reference database), with retrieval-based classification and VLM scene-text reading as complements. Field-tested on washing machines: the pipeline read model numbers from 95.7% of label photos taken in professional repair, but only 39.7% of those taken at a recycler (Sterkens et al., 2023). Failure mode: label degradation at EoL, plus a database-coverage limit downstream.
• Quantity (partial, approximate). Monocular metric depth (Metric3D v2, Depth Pro, Depth Anything v2), feed-forward multi-view geometry (VGGT, MapAnything), and photogrammetry / NeRF / 3D Gaussian splatting recover error-bounded geometry. Single-image mass estimation exists but is coarse. Visible surfaces only; reflective, transparent, and thin parts are the limiting cases.
• Condition (emerging but narrow). Wear, damage, and corrosion recognition feeding repair / reuse / recycle triage. Per-unit grading works for single product classes on fixed camera rigs, such as phone buyback kiosks and laptop-cover inspection. None is validated across product classes or on field photos. Zero-shot anomaly detection (WinCLIP, AnomalyCLIP) is more general, but it flags that something is wrong, not how worn a unit is.

Demo: the zero-shot identification demo runs the open-vocabulary part of this cell in your browser.

Component-level tasks

What it covers: data about the individual components inside an opened product.

Examples:

• Structure / BoC (partial detection, emerging-but-narrow relations). The most populated cell. Detecting, segmenting, and counting components is accurate on the benchmarks a model was tuned for: about 98.8% on a laptop-teardown set, in mAP50, the standard detection score (Zhang et al., 2026). It is untested on field photos. Foundation segmenters (SAM, SAM 2, SAM 3, GroundingDINO, Grounded-SAM) need an added class-assignment layer. Inferring component-to-component attachment relations is the weakest cell. Single-image methods (SINGAPO, URDFormer) exist only for a few articulated household categories and collapse out of distribution (~82% → ~39%). They also recover kinematic joints rather than the fastener relations (screwed, glued, clipped) that a disassembly BoC needs.
• Identity (partial). OCR on component markings plus one-shot / retrieval recognition and open-vocabulary naming from a text prompt. Fine-grained brand- or part-specific identity is not reliably recovered.
• Quantity (partial, approximate). Inherits the product-level geometry methods, aggravated by occlusion within the assembled product.
• Condition (plausible but unvalidated). Connector-state recognition (screws, clips, solder joints) for disassembly planning. Published work shows fastener detection, not per-unit condition grading.

Material-level tasks

What it covers: the material composition (bill of materials) of the product and its parts.

Examples:

• Identity / BoM (partial on visible surfaces, open gap for full BoM). Surface material classification and segmentation of visible regions (MINC’s 23-class in-the-wild baseline; one/few-shot similarity given a region mask; CLIP-based recognition, fragile under domain shift and multi-material compositeness). One setting delivers image-based composition: an industrial sorter tested at pilot scale on shredded WEEE, using YOLO (mAP50 ≈ 0.99 on recorded data) (Vogelgesang et al., 2025). But it characterizes an aggregate flow at a central plant, not the per-component composition of an intact product. Occluded interior composition is an open gap.
• Quantity (derived, not sensed). Per-material mass or volume would come from component geometry × material class × a density prior. No dedicated method exists, and the estimate compounds the uncertainty of both inputs.
• Condition (plausible but unvalidated). Corrosion, degradation, and contamination. Surface segmentation can isolate regions, but per-region grading on arbitrary products is not validated.