1

I'm trying to track down a cause on an Internal Server Failure for an Apache server. The IP address changed recently, so I thought the crash might be related. A quick search of the old IP:

$ sudo grep -IR --exclude-dir="/var/run" '45.78.157.165' /var | egrep -v '(audit|access)'
/var/log/secure:Jun 21 14:08:34 cryptopp sshd[19729]: error: Received disconnect from 199.91.135.157: 3: com.jcraft.jsch.JSchException: reject HostKey: 45.78.157.165 [preauth]
/var/log/secure-20160626:Jun 21 14:08:34 cryptopp sshd[19729]: error: Received disconnect from 199.91.135.157: 3: com.jcraft.jsch.JSchException: reject HostKey: 45.78.157.165 [preauth]
/var/log/secure-20160626:Jun 21 14:08:34 cryptopp sshd[19729]: error: Received disconnect from 199.91.135.157: 3: com.jcraft.jsch.JSchException: reject HostKey: 45.78.157.165 [preauth]

com.jcraft.jsch.JSchException: reject HostKey is unusual, so I wanted to gather more information. A quick search revealed com.jcraft is some sort of Java Secure Channel. Then:

$ whois 199.91.135.157
...

NetRange:       199.91.132.0 - 199.91.135.255
CIDR:           199.91.132.0/22
NetName:        BCS-HQ-USA
NetHandle:      NET-199-91-132-0-1
Parent:         NET199 (NET-199-0-0-0-0)
NetType:        Direct Assignment
OriginAS:
Organization:   Bluecoat Systems, Inc. (BLUEC-8)
RegDate:        2011-02-04
Updated:        2012-03-02
Ref:            https://whois.arin.net/rest/net/NET-199-91-132-0-1

OrgName:        Bluecoat Systems, Inc.
OrgId:          BLUEC-8
Address:        420 N Mary Ave.
City:           Sunnyvale
StateProv:      CA
PostalCode:     94085
Country:        US
RegDate:        2007-01-23
Updated:        2013-01-28
Ref:            https://whois.arin.net/rest/org/BLUEC-8

OrgAbuseHandle: LAMAS1-ARIN
OrgAbuseName:   Lama, Soni
OrgAbusePhone:  +1-408-220-2200
OrgAbuseEmail:  soni.lama@bluecoat.com
OrgAbuseRef:    https://whois.arin.net/rest/poc/LAMAS1-ARIN

OrgTechHandle: LAMAS1-ARIN
OrgTechName:   Lama, Soni
OrgTechPhone:  +1-408-220-2200
OrgTechEmail:  soni.lama@bluecoat.com
OrgTechRef:    https://whois.arin.net/rest/poc/LAMAS1-ARIN

I know what Bluecoat is, and I know what they do. I think the behavior is highly unusual, but I want to refrain from making leaps.

To provide a little more context, this is abasically a Linux LAMP server, with no additional software. The VM provided by VirtWire, and the system is virtualized CentOS 7.2. It runs an SSH server, and Apache, MediaWiki and MySQL. Only Admins have SSH access for administration. I patch it regularly, and it usually lags by 3 to 7 days, but not much more.

My question is, why is the server trying to callback to Bluecoat (or why is it receiving connections from them)? Are there any legitimate use cases I may have missed?


Zeroing in on the exception and IP block:

$ sudo egrep -IR --exclude-dir="run" --exclude-dir="spool" '(com.jcraft|199.91.135)' /var
/var/log/secure:Jun 21 14:08:34 cryptopp sshd[19729]: error: Received disconnect from 199.91.135.157: 3: com.jcraft.jsch.JSchException: reject HostKey: 45.78.157.165 [preauth]
/var/log/secure:Jul  1 20:58:28 cryptopp sshd[1235]: error: Received disconnect from 1.55.196.144: 3: com.jcraft.jsch.JSchException: Auth fail [preauth]
/var/log/secure-20160620:Jun 13 13:18:01 cryptopp sshd[23246]: error: Received disconnect from 103.207.36.185: 3: com.jcraft.jsch.JSchException: Auth fail [preauth]
/var/log/secure-20160620:Jun 13 21:03:09 cryptopp sshd[28964]: error: Received disconnect from 116.99.253.189: 3: com.jcraft.jsch.JSchException: Auth fail [preauth]
/var/log/secure-20160620:Jun 13 21:04:20 cryptopp sshd[28971]: error: Received disconnect from 116.99.253.189: 3: com.jcraft.jsch.JSchException: Auth fail [preauth]
/var/log/secure-20160612:Jun  8 21:09:59 cryptopp sshd[3712]: error: Received disconnect from 46.35.253.161: 3: com.jcraft.jsch.JSchException: Auth fail [preauth]
/var/log/secure-20160626:Jun 21 14:08:34 cryptopp sshd[19729]: error: Received disconnect from 199.91.135.157: 3: com.jcraft.jsch.JSchException: reject HostKey: 45.78.157.165 [preauth]
/var/log/secure-20160626:Jun 21 14:08:34 cryptopp sshd[19729]: error: Received disconnect from 199.91.135.157: 3: com.jcraft.jsch.JSchException: reject HostKey: 45.78.157.165 [preauth]

I believe this is a related question: What's the purpose? Strange login attemps “sshd[*] Received disconnect from **...: 11: Bye Bye [preauth]”.

  • Why do you ask a question about Apache crashing and provide no information from the Apache logs, yet you provide tons of information about something completely irrelevant to Apache? If you have two questions, post both separately on the relevant Stack Exchange site. The Apache crash would probably be better addressed on unix.stackexchange.com – Julie Pelletier Jul 2 '16 at 19:58
  • I found the first line of this question quite misleading. – Julie Pelletier Jul 2 '16 at 20:33
  • @JuliePelletier - yes, it was a sentence or two of context; nothing more. I am pursuing the crash on Pro Webmasters at Internal Server Error; enable debugging provides no additional information. – jww Jul 2 '16 at 21:12
  • 2
    jsch is a Java implemention of SSH protocol(s) from jcraft.com who are a company that writes Java software. SSH disconnect reason 3 is keyexchange which is consistent with the text. It appears some machine at Bluecoat is setup so it connects to your address, perhaps due to bad DNS somewhere, but once it connects and sees your hostkey it realizes it doesn't want to connect to you and disconnects. If it only happened once, or rarely, I'd ignore it; if it's frequent enough to be an issue I'd try to complain to them. – dave_thompson_085 Jul 3 '16 at 0:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.