Re: {bb} SSH checks not written to $BBVAR/logs

[Philip Clark <bblist@cannae.uklinux.net>]

On Wed, 2006-02-15 at 14:17, Eric Christian Berg wrote:
> Just an update. Entries are written sporadically to logs, with libmgt2 
> doing so more than libmgt1, but very infrequently. On average, it will 
> stay green for half an hour and then go purple again.
> 
> Does anyone have any ideas?
> 
> Eric Christian Berg wrote:
> > We have a Big Brother installation monitoring 250 machines with a 
> > primary and a failover server (both RHEL 3 boxen with openssh). Today, 
> > the server started kicking out purple alerts for two of the machines on 
> > the ssh check. There is nothing unique about them. They are Solaris 9, 
> > like half of our systems. They run SSH.com, like most of our systems. 
> > They were monitoring fine for a week. The only changes I made today were 
> > to add bbwarnrules.cfg entries for both servers.
> > 
> > I set up tcpdump to confirm that the server was checking every five 
> > minutes. It is. I checked that ssh is working. It is. I removed the 
> > files with bbrm for ssh on those systems and have been watching for an 
> > hour. I see tcpdump talk to both machines on port 22, yet no file is 
> > written to $BBVAR/logs to replace the ones bbrm took out.
> > 
> > So my question is this, why would Big Brother be doing a network check 
> > but not writing the results to the logs directory?
> > 
> > For reference, here are the relevant bb-hosts lines:
> > 128.205.7.27 libmgt2.acsu.buffalo.edu # ssh
> > 128.205.7.26 libmgt1.acsu.buffalo.edu # ssh
> > 
> > Here are the lines from bbwarnrules.cfg:
> > libmgt2.acsu.buffalo.edu*;; conn ssh cpu disk msgs 
> > procs;;*;0500-1700;library-support@gory.acsu.buffalo.edu:15 
> > ext-epagesvc-University_libraries:15 
> > ext-epagesvcsec-University_libraries:~15-30
> > libmgt1.acsu.buffalo.edu*;; conn ssh cpu disk msgs 
> > procs;;*;0500-1700;library-support@gory.acsu.buffalo.edu:15 
> > ext-epagesvc-University_libraries:15 
> > ext-epagesvcsec-University_libraries:~15-30

The log files on those two servers might contain something useful.
It might also be helpful to run the "bbnet" binary manually, in
order to get more detail about what is going on. Something like:

  ./bbnet "libmgt1.acsu.buffalo.edu:22" "Big-Brother-Monitor-1.9i"

should do the trick.

It's also worthwhile pointing out that BB does not try to establish
a real session when it performs the test. It does this in order to
keep the test fast and avoid wasting resources. It's possible that
the ssh server might interpret these repeated failed connections as
a threat and block further communication.

If this is what's happening, you may have to:

  1. Modify BB to send an "acceptable" protocol ID string.
  2. Configure the server so that it does not reject bogus
     connections from the BBNET servers.
  3. Replace the default BB test with an external script that
     uses a real client. This should give you a good starting
     point:

       http://www.deadcat.net/viewfile.php?fileid=44

Cheers, Phil.



-- 
Abstainer, n.: A weak person who yields to the temptation of
denying himself a pleasure.
(Ambrose Bierce, "The Devil's Dictionary")

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=
To unsubscribe from this list, or to subscribe to the bb-digest list
send e-mail to mailto:majordomo@bb4.com with unsubscribe bb -and/or-
subscribe bb-digest in the BODY of the message.