0

I want to scrape data from Fanball.com to keep track of the fantasy football players that I have drafted, and create a visualization of my exposure to different players. This is a tool that I have seen other people request and I would like to make it available to other users. In order to scrape the data of your leagues you need your auth token. How would I go about creating this tool for other people so that I can use their fanball auth token to scrape the data, without actually ever having the auth token or a password on my server. Instead of a web app, would it be better to make a desktop app that connects to the web? that way they can enter their information and it is stored on their machine?

2
  • You say, "Would it better to make a desktop app that connects to the web?" Better than what? Commented Apr 3, 2018 at 2:54
  • An in browser web app Commented Apr 3, 2018 at 2:55

1 Answer 1

0

Unless the server has configured to allow this use, you will probably be immediately blocked, in the browser, by Cross-Origin Resource Sharing (CORS). I've never completely understood the attack vectors CORS protects against, but one of them is the abuse of the same situation you're suggesting - access to a third-party site without either the third-party or the user's approval.

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

To do what you are asking, within the CORS framework, the server would need to be an active participant. They would need to return the proper headers on a pre-flight OPTIONS request to support CORS for your site's domain.

For some sites, they may have configured CORS to be very permissive and then you could just sort of sneak in. But they'd have to leave the door open for you.

Another way they could willingly help you achieve this sort of interaction is to provide an API, with OAuth support. You could then use OAuth "implicit flow" to authenticate entirely in the browser, and you'd never have had access to their token. Implicit flow is somewhat less secure though... it's usually better to have a server involved in API calls with OAuth. In which case, you'd never have had their password, and your access to the API might be limited by the permission they gave you, but their browser wouldn't be holding the token they fetched for you to use.

If you want to do scraping from a site that is not a willing participant, you shouldn't be able to do it from a web browser without accessing their password or auth token. A desktop app might be the way to go.

If you want to be honest about it though, you could pitch your idea to Fanball.com and see if they'd open an API to you. You might be able to persuade them that your application would add value to their service.

1
  • A side note: If they do not want you scraping their site, they may add counter-measures to make it difficult for your scraper to access their site. Some sites just don't like you accessing their content without permission, even if the user did log in for you.
    – nbering
    Commented Apr 3, 2018 at 3:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .