▪ claudecodedaily

I'm Building Substack Newsletter Search Engine, problem it solves semantic search with RAG and no need to juggle to read newsletters on particular topic of your favourite author. Challenging part: 1. Some newsletters are only for paid subscribers only and if we ingest those articles into source database then it pollute your search results as "This post is for paying subscribers only" and waste embeddings tokens. 2. Database Model Dual identifier system means internal id as incremental big int and external ids as uuid, it solves problem of data redundancy and query optimisation. I have used supabase-postgresql as data source, qdrant as vector store, opik for tracing LLM call and cost/token, gcp/aws for deployment. #AI #RAG #data