| 2026-06-12 |
feat: proper Ukrainian FTS with stop-words filtering
Eugene Sukhodolskiy
committed
1 day ago
|
feat: switch FTS to ukrainian text-search config
Eugene Sukhodolskiy
committed
1 day ago
|
feat: add PostgreSQL full-text search
...
- Add generated search_vector tsvector column with immutable wrapper
to_tsvector_simple() for mixed ru/ua text
- Add GIN index ix_property_listings_search_vector_gin
- Add PropertyRepository.search_fulltext() using plainto_tsquery(simple)
and ts_rank_cd() with optional filters
- Add POST /api/v1/search/fulltext endpoint (120/min rate limit)
- Add FulltextSearchRequest/Result/Response schemas
- Update alembic.ini to use Docker PostgreSQL on port 5433
Co-Authored-By: Claude <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
1 day ago
|
feat: add pgvector semantic search
...
- Add pgvector dependency and Alembic migration (vector extension, embedding
column, HNSW index with cosine ops)
- Add nomic-embed-text embedding model to config
- Add OllamaClient.embed() method for /api/embed endpoint
- Add embedding generation stage to PropertyPipeline (_stage_embed)
- Add PropertyRepository.update_embedding() and search_similar() with
cosine distance + optional filters (deal_type, city, price range)
- Add POST /api/v1/search/similar endpoint with query embedding + filters
- Add SimilarSearchRequest/Response schemas
- Add backfill script for existing listings
- Update docker-compose.yml to pgvector/pgvector:pg16 image
- Update .env to use Docker PostgreSQL on port 5433
Co-Authored-By: Claude <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
1 day ago
|
feat: dockerize app, add structured logging, fix rate limiter
...
Changes:
- Add Dockerfile with python:3.12-slim, alembic migrations, uvicorn
- Add .dockerignore
- Update docker-compose.yml with app service, port 8020, external Ollama
- Configure alembic/env.py to read DATABASE_URL from env
- Update .env.example with port 8020, Ollama host 192.168.1.75, gemma4 model
- Fix slowapi rate limiter: sync key_func instead of async
- Add structured JSON logging (structlog) to ingest endpoint, pipeline stages
- Fix logging output via logging.basicConfig for Docker stdout
Co-Authored-By: Claude <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
1 day ago
|
| 2026-06-11 |
feat: implement review items 8-14
...
- Soft-delete/archive for listings: archived_at column + archive-check endpoint
- Rate limiting on /ingest: slowapi with 60/minute per source_slug
- Prometheus metrics: /metrics endpoint + custom counters/histograms
- Graceful shutdown: track active jobs in app.state, wait up to 30s
- Prompt injection protection: wrap user data in <user_data> XML tags
- Image download size limit: 50MB max with httpx streaming
- Raw data cleanup: admin endpoint to delete completed raw data older than N days
Co-Authored-By: Claude <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
1 day ago
|
feat: implement review items 1-7
...
- Decompose PropertyPipeline into 8 explicit stages with PipelineContext
- Add tenacity retry (3 attempts, exponential backoff) to OllamaClient and ImageDownloader
- Add simple in-memory circuit breaker for Ollama calls
- Resize images to 1024px before base64 encoding for vision model
- Add /health endpoint (DB, Ollama, disk checks)
- Add DB performance indexes + alembic migration
- Classify AI errors: OllamaRetryableError vs OllamaFatalError
- Add strict PayloadSchema validation for ingest endpoint
Co-Authored-By: Claude <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
1 day ago
|
feat: core pipeline + FastAPI API (Phases 0-6)
...
Implemented the full VMK Data Collector foundation and processing pipeline:
- Config, logging, exception hierarchy
- DB models (listings, raw data, images, snapshots, enrichments, custom fields)
- Alembic async migrations
- Repositories with upsert/snapshot support
- Domain entities and Pydantic schemas
- Ollama AI client with mock mode
- AI normalizer, image analyzer, enricher
- Image downloader with SHA-256 dedup
- PropertyPipeline: raw -> AI validate -> upsert/snapshot -> images -> enrich
- FastAPI app with /api/v1/ingest endpoint
Co-Authored-By: Claude <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
1 day ago
|