feat: implement review items 1-7

Fork: 0

root / vmk-360-data_collector

Browse code feat: implement review items 1-7 - Decompose PropertyPipeline into 8 explicit stages with PipelineContext - Add tenacity retry (3 attempts, exponential backoff) to OllamaClient and ImageDownloader - Add simple in-memory circuit breaker for Ollama calls - Resize images to 1024px before base64 encoding for vision model - Add /health endpoint (DB, Ollama, disk checks) - Add DB performance indexes + alembic migration - Classify AI errors: OllamaRetryableError vs OllamaFatalError - Add strict PayloadSchema validation for ingest endpoint Co-Authored-By: Claude <noreply@anthropic.com> master
1 parent 9eb6d3e commit 82fa839b90e00798c2f854f6c43ba24f77b90112 Eugene Sukhodolskiy authored 1 day ago

Browse code

- Decompose PropertyPipeline into 8 explicit stages with PipelineContext
- Add tenacity retry (3 attempts, exponential backoff) to OllamaClient and ImageDownloader
- Add simple in-memory circuit breaker for Ollama calls
- Resize images to 1024px before base64 encoding for vision model
- Add /health endpoint (DB, Ollama, disk checks)
- Add DB performance indexes + alembic migration
- Classify AI errors: OllamaRetryableError vs OllamaFatalError
- Add strict PayloadSchema validation for ingest endpoint

Co-Authored-By: Claude <noreply@anthropic.com>

master

1 parent 9eb6d3e commit 82fa839b90e00798c2f854f6c43ba24f77b90112

Eugene Sukhodolskiy authored 1 day ago

Patch

Unified Split

Showing 22 changed files

Ignore Space Show notes View alembic/versions/ce83ed173113_add_performance_indexes.py 0 → 100644

Ignore Space Show notes View docs/REVIEW_FOLLOWUP.md 0 → 100644

Ignore Space Show notes View src/vmk_data_collector/api/deps.py

Ignore Space Show notes View src/vmk_data_collector/api/v1/router_health.py 0 → 100644

Ignore Space Show notes View src/vmk_data_collector/api/v1/router_properties.py

Ignore Space Show notes View src/vmk_data_collector/core/circuit_breaker.py 0 → 100644

Ignore Space Show notes View src/vmk_data_collector/core/exceptions.py

Ignore Space Show notes View src/vmk_data_collector/db/repositories/property.py

Ignore Space Show notes View src/vmk_data_collector/main.py

Ignore Space Show notes View src/vmk_data_collector/models/ai_enrichment.py

Ignore Space Show notes View src/vmk_data_collector/models/property_custom_field.py

Ignore Space Show notes View src/vmk_data_collector/models/property_image.py

Ignore Space Show notes View src/vmk_data_collector/models/property_listing.py

Ignore Space Show notes View src/vmk_data_collector/models/property_snapshot.py

Ignore Space Show notes View src/vmk_data_collector/models/raw_parsing_data.py

Ignore Space Show notes View src/vmk_data_collector/schemas/raw_data.py

Ignore Space Show notes View src/vmk_data_collector/services/ai_enricher.py

Ignore Space Show notes View src/vmk_data_collector/services/ai_image_analyzer.py

Ignore Space Show notes View src/vmk_data_collector/services/ai_normalizer.py

Ignore Space Show notes View src/vmk_data_collector/services/image_downloader.py

Ignore Space Show notes View src/vmk_data_collector/services/ollama_client.py

Ignore Space Show notes View src/vmk_data_collector/services/property_pipeline.py

Show line notes below