CroEx, exchange rates from 17 Croatian banks
A scraper, a read-only API and a public website that compare what Croatian banks charge for foreign currency, with every bank's history since the euro arrived.
- Role
- Personal project, designed and built end to end
- Outcome
- About 2,300 lines of copied parser code replaced by one shared scraper; most banks are now around fifteen lines of configuration.
- Stack
- Python
- Playwright
- PostgreSQL
- Java 25
- Spring Boot
- Next.js
- Docker
Problem
Every Croatian bank publishes its own exchange rate list, on its own page, in its own format. To see which bank gives the best rate for a currency today, or how that rate has moved, you have to visit each one.
The first version of this project, an older Java scraper, had 19 parser classes, each with its own copy of the same four methods: about 2,300 lines. Banks were registered in a hand-written list, and two finished parsers were never added to it.
Approach
CroEx is three applications around one PostgreSQL database:
bank pages and archives
│
▼
croex-scrapware Python, Playwright · runs once a day
│ writes
▼
PostgreSQL
▲ reads, read-only
│
croex-api Spring Boot · private
▲
│ server-side calls only
croex-ui Next.js · public
▲
│
browser
The scraper runs once and exits, so any scheduler can drive it. A few design choices carry most of the weight:
- One algorithm, many banks. Scraping is shared code. A bank is a small class that names its URL, selectors and columns, and most of them are about fifteen lines.
- No hand-written list. Bank classes are discovered automatically, and startup fails if one is misconfigured or two share a name.
- One failure boundary per run. Seventeen websites outside my control means something breaks most days. A failing bank is logged in one line and the run continues; the exit status reports it.
- Safe to rerun. Each bank gets one snapshot per day, replaced as a whole, so a retry never duplicates rows.
- A real browser only where needed. Some banks build their list with JavaScript, so the daily run uses headless Chromium. The banks’ archives answer plain HTTP, so the history backfill needs no browser.
The history goes back to 1 January 2023, the euro’s first day in Croatia, collected from each bank’s own archive. Archives disagree with their own pages in ways worth knowing: number formats that changed over time, rates stored as 32-bit floats, lists published twice for the same day. Each case is handled in that bank’s source and documented. The live tests scrape today’s page and ask the archive for the same day, and require the same rates.
The API is a read-only Spring Boot service on Java 25. The scraper owns the schema; the API only validates it at startup and opens read-only connections. Errors follow RFC 9457 problem details, and history is paged by calendar month.
The website is Next.js with server components. Pages call the API on the server only, so the browser never learns its address, and every bank name or currency code from a URL is checked against the API’s own lists before it becomes part of a request. Page state lives in the URL, so every view can be shared.
Result
- 17 banks scraped daily, with each bank’s history since 1 January 2023 shipped as a database dump.
- 2,300 lines of copied parser code replaced by one shared algorithm and small bank configurations.
- The database records which periods it already holds, so a history run fetches only what is new instead of about 19,000 requests, and a full run takes under twenty minutes.
- A public site: best-rate finder, per-bank pages, multi-bank charts and an ATM map from OpenStreetMap data.
What I’d change
The website’s Content Security Policy still allows inline scripts, because Next.js needs them for hydration. Moving to per-request nonces is the next hardening step.