0

For example google.com depends on ssl.gstatic.com, lh3.googleusercontent.com, fonts.gstatic.com.

I need this list for Proxy settings

closed as off-topic by Lie Ryan, Steffen Ullrich, S.L. Barth, schroeder Dec 11 '17 at 9:33

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question does not appear to be about Information security within the scope defined in the help center." – Lie Ryan, Steffen Ullrich, S.L. Barth, schroeder
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Could you please explain how did you determine that google.com* depend on googleusercontent.com` and gstatic.com? – F. Hauri Dec 11 '17 at 9:09
  • @f-hauri If these 2 domains will be blocked, current version of google.com won't work properly – Rustam Dec 11 '17 at 15:41
1

About dependencies

Domain dependencies are only registrars. There is nothing else.

But you could ask for:

whois gstatistic.com | grep -A1 Holder

Hmm. nothing? Ok looking for names servers:

whois gstatic.com | grep Name.Server
   Name Server: NS1.GOOGLE.COM
   Name Server: NS2.GOOGLE.COM
   Name Server: NS3.GOOGLE.COM
   Name Server: NS4.GOOGLE.COM

and

whois googleusercontent.com | sed '/Name Server/q;d'
   Name Server: NS1.GOOGLE.COM

Also, building a proxy (meaning web proxy), is a HTTP concern. What you call dependencies are only web links to other repositories where to find some objects.

Web domain dependencies

Web pages depend on a lot of things:

  • scripts: Libraries could be stored on development site
  • fonts: Fonts are generally located on fonts servers.
  • styles: Styles could be linked to scripts, stored on development servers.
  • videos: Mostly located on stream servers.
  • map: Geographic maps, stellar maps, oceanographic, human... And other specialised big data systems..
  • photos, images, others...: Some copyrighted files could only be shown from original site (artist, photograf...),

But web domains strongly depends on

  • advertiser where links to advertiser host is required for log count and further analisys.
  • captcha I hate captcha but...
  • online banks required for web shops.

Where to get such a list

On your web history, cache and mostly on the log of your proxy server...

Unfortunely such a list could not exist, because every web site could be build and modified at any time.

So if you plan to build a strongly restrictive proxy server, you have to keep some hotline up and stay ready to change your rules, adding some new kind of third party site, at any time.

And finally, about Top 100 popular sites

Answer could only be subjectives...

You could try:

... Then have a look at your proxy logs ;-)

Tip or trick:

Have a look at my answer about How to display the Subject Alternative Name of a certificate?,

Displaying Subject Alternative Name of SSL certificates give some exhaustive list of directly related domains.

Not the answer you're looking for? Browse other questions tagged or ask your own question.