feat: download images locally and ingest via multipart /with-images endpoint

Fork: 0

root / vmk-360-domria_parser

Browse code feat: download images locally and ingest via multipart /with-images endpoint - Update collector to support ingest_with_images using CurlMime multipart - Crawler now downloads listing photos in parallel (5 workers) to temp files - Fix photo URL builder: use photosnew CDN + beautifulUrl (legacy file paths are 415) - Add fallback title generation in normalizer for PayloadSchema validation - Increase ingest timeout to 120s for inline pipeline processing Closes: photo ingestion via binary upload master
1 parent f85730b commit edffe4b3aa1219256e82ebf2f13609e9331e985d Eugene Sukhodolskiy authored 1 day ago

Browse code

- Update collector to support ingest_with_images using CurlMime multipart
- Crawler now downloads listing photos in parallel (5 workers) to temp files
- Fix photo URL builder: use photosnew CDN + beautifulUrl (legacy file paths are 415)
- Add fallback title generation in normalizer for PayloadSchema validation
- Increase ingest timeout to 120s for inline pipeline processing

Closes: photo ingestion via binary upload

master

1 parent f85730b commit edffe4b3aa1219256e82ebf2f13609e9331e985d

Eugene Sukhodolskiy authored 1 day ago

Patch

Unified Split

Showing 4 changed files

Ignore Space Show notes View src/collector.py

Ignore Space Show notes View src/config.py

Ignore Space Show notes View src/crawler.py

Ignore Space Show notes View src/normalizer.py

Show line notes below