I figured this was a good place to learn a little about trusting libraries such as HTMLUnit with sensitive information.

I want to build a service that categorises my spending, kind of like my own personal mint.com. Can I trust an open source library such as HTMLUnit with passwords to sign into my bank account and scrape data. Additionally I would like to know the most secure way to implement the dependency. Currently the dependency is downloaded into my Play framework project via maven. To do this all I had to do was add the following line to the build.sbt file:

libraryDependencies += "net.sourceforge.htmlunit" % "htmlunit % "2.20""  the build.sbt"

Is this a secure way to use such a dependency? Or is there a better way to add the dependency to my project?

Any advice would would be greatly appreciated, thanks in advance for your help.

  • As long as it runs on your own computer it should be fine - Java isn't really vulnerable to buffer overflows or similar exploits, so worst case scenario your app would just not work as expected but wouldn't execute malicious code. Just be careful when dealing with TLS to make sure you're talking to your bank and not someone else. Mar 17, 2016 at 18:25
  • Any http requests to my bank are rerouted to https automatically, http requests dont work, so it appears the only way I can use my internet banking is via TLS. Is there anything else I can do on my end to ensure that TLS is used?
    – DominoSug
    Mar 18, 2016 at 10:23
  • Also sorry, why do you say "as long as it runs on your own computer?", What threats are you concerned about if I run this on another computer (i.e. running this from a secure web server)
    – DominoSug
    Mar 18, 2016 at 10:27
  • The bank's legitimate server is redirecting HTTP to HTTPS. If it was an attacker he definitely wouldn't do that as it's in their best interest if you continue to use HTTP. If you run this on a server you should make sure your server isn't compromised. Jul 16, 2016 at 0:36
  • For prior art, take a look at the OFX protocol, and GnuCash's documentation of same. It is possible, though unlikely, that your specific bank still offers this mechanism to retrieve your transaction records, saving you from screen scraping. If/when screen scraping, follow Noir's advice and be very careful about "testing" against your bank's servers. Many banks now monitor for this behavior. Aug 15, 2016 at 5:41

1 Answer 1


Best practice in software testing is to avoid testing with production systems (Your banking site is a production system and additionally I bet automated crawling is a non intended use case) and further to not test with real data.

I'd try to mock most output of the banking site until I'm pretty sure my code is working as intended. The tests could be interpreted as a hacking attempt which could result in a temporary ban.

However I don't see an issue with using HTMLUnit since it's pretty transparent what the library is doing.

For the second part: Make sure to get the dependency via TLS.

  • thanks for your response, I am using plays dependancy management to download HTMLunit. The Play Framework uses SBT which in turn uses the existing Maven Central repository. the Libraries I use are downloaded when I build the project. How can I ensure that this is using TLS?
    – DominoSug
    Mar 18, 2016 at 10:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .