Skip to content

feat: add real A101 scraper for fruit and vegetables#51

Open
saba-github wants to merge 15 commits intomainfrom
feature/a101-real-scraper
Open

feat: add real A101 scraper for fruit and vegetables#51
saba-github wants to merge 15 commits intomainfrom
feature/a101-real-scraper

Conversation

@saba-github
Copy link
Copy Markdown
Owner

What changed

  • Added a real A101 scraper for fruit_veg

  • Integrated A101 into the existing raw → staging → fact pipeline

  • Split A101 fruit and vegetable scraping into:

    • meyve-sebze/meyve
    • meyve-sebze/sebze
    • meyve-sebze/yesillik
  • Fixed A101 section parsing by extracting products from body text sections

  • Added fallback unit normalization from product names

  • Enabled A101 records to flow into fact_price_observations

Result

Latest successful A101 run loaded:

  • 38 scraped records
  • 38 raw records
  • 38 staging records
  • 38 fact records

Category distribution:

  • Meyve: 16
  • Sebze: 22

Notes

  • Yeşillik parsing still needs improvement in a future iteration
  • Current implementation is stable enough to merge and continue product-level development

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant