Architecture testing notes

This file collects testing-related facts implied by the architecture diagram and project context.

Timing requirement

• Analysis + result description must fit within 15 seconds per image.
• Measure end-to-end time: trigger → inference → DB write → UI update.

Test data sets

Initial dataset

• Real photos of polyurethane soles.
• Used for the first training run.

Learning dataset

• Combined from:
  • initial dataset;
  • artificially generated / augmented images;
  • expert-verified production data.

Separate test set

• Must not overlap with training data.
• Size:
  • 20 soles without defects;
  • 3 soles with defects.

Artificial disturbances

Use the artificial image generator to simulate production conditions:

• lighting effects;
• PNG pattern overlays (dust, lens dirt, other obstacles);
• rotation (±5° to match allowed positioning tolerance);
• noise generator for dust emulation.

Position tolerance

• Rotation: ±5° relative to camera.
• Displacement: up to 10% of frame.
• Test inference quality across the full tolerance range.

Camera replacement study

• Compare recognition quality when swapping camera models.
• Document changes in preprocessing parameters needed after replacement.

Testing environment

• Run under conditions close to real production.
• Initial development can happen at home / lab using fake input data and artificial disturbances.
• Physical dimensions of soles must stabilize before testing (fixed time after casting).

Expert feedback loop

• Every production event should be reviewable by an operator / expert.
• Expert verdicts (correct / incorrect) feed the learning dataset.
• Retraining can be filtered by date to avoid including low-quality old data.

Model versioning

• Each trained / retrained model must be versioned.
• Keep metrics for each version so performance can be compared.
• Allow rollback to a previous model if retraining degrades quality.