Scraping Bank of Ireland Transaction Data

TLDR

Sick of complaining about the lack of functionality to export bank statements to CSV files (for use in Spreadsheets) in Business Banking Online I decided to figure out how to scrape the site for my transaction data.

If you want to skip the rant click here for the approach I used.

Rant

The old BOI-BOL (Bank Of Ireland - Business Online) system broke my heart many a time. I use a Mac, so in order to use this antiquated online banking system I had to setup a virtual machine to run Windows XP with a copy of IE7 just to do my banking.

The old system was a real dog to use, and the roundabout way of accessing it meant that I became slower at a number of things: checking new transactions to process invoices, paying bills as soon as I got them, etc etc.

But above all, my real pet peeve was the fact that you couldn’t export reports in any meaningful way.

For the older version of the more modern Personal Banking system, I wrote my own scraper in PHP to parse and export data out of the HTML that made up the online report page. It was hacky and roundabout, and when BOI decided to add Export to CSV functionality for personal users I rejoiced.

But the Business Online system was still really behind the times, and reconciling bank statements was a manual chore and time sink.

This year Bank Of Ireland introduced their new Business Online system for beta use. It has a fancier interface that panders a bit to latest trends in UI design, it's clunky as a result and could do with some proper UX / UI treatment based on how people actually bank online. But still, it’s a far better tool than it’s predecessor and fair play to BOI for finally bringing it out.

Guess the first thing I did when I logged in? I went looking for the Export to CSV functionality. Let’s face it: any serious effort at accounting requires bank reconciliation, which in the modern world means manipulating data on spreadsheets or importing data into accounting software. In other countries you can interface with banking systems via APIs, making reconciliation a doddle. But in a Ireland, where we’re thankful to be able to now access bank accounts in a modern browser on a non-Windows machine, we’ll have to make do with Export functionality.

So… where was it? I could see a View Report to get a web view of my statements and a Download PDF link (I mean seriously do I want to print out my statements??) but no Export functionality.

I phoned up the help desk. It turns out that I’m on Package Level 2 and I need to be on Package Level 3 to access the export functionality. Really?? I knew what was coming. “What’s the cost of Package Level 3?” I asked and held my breath. €55 per month.

I had to laugh. Package Level 2 costs €10 per month, which is already a bit cheeky. But I’d have to pay an extra €540 per year to Export to CSV - a feature that comes for free with Personal Banking!

(The only other feature on Level 3 seems to be same day payment).

Well I’m sick of complaining about the bank - and / or waiting for them to produce this functionality - so I decided to figure out my own way of running an export, and for what it’s worth - here it is.

Please note: I make absolutely no claims about the security etc etc of doing things this way.

Scraping Bank Of Ireland Business Online to Export Transactions into a Spreadsheet

Now that you can access BOI BOL on modern browsers like Chrome - you have access to an array of extensions and tools to make some tasks easier.

I came across this scraper extension. The tutorial on its use is here.

  • Install the extension - note: use this at your own risk.
  • Log on to the new BOI BOL on Chrome (it’s in beta - you might need to get invited).
  • Go to the account you’re interested in, then click on Report > View Report - this opens a modal window with 3 months of transaction history
  • Right-click on the table and choose “Scrape Similar” - for more info see the tutorial mentioned above - this will open the scraper tool.
  • To save you the trouble of figuring out XPath - see the screen shot for how you need to set up the scraper to actually collect the transaction data.

Here are the settings to copy and paste:

Path to table: //div[4]/div/div/div[2]/div/table/tbody/tr
Posting Date: normalize-space(./td[3])
Narrative: normalize-space(./td[5])
Debit: normalize-space(./td[6])
Credit: normalize-space(./td[7])
Balance (if you want it): normalize-space(./td[8])

  • Set up the scraper as per the screenshot and click Scrape
  • To save the setup as a Preset for easy use again - click Preset
  • To export to a spreadsheet click Export to Google Docs

And breathe a sigh of satisfaction: you can now do something useful with your bank statements – so get back to those accounts!