Hello once again! First, I wanna apologize for the lack of updates (again). I have been so busy with new and exciting work and the possibilities and new skills have been really pouring in. Regardless, I wanna thank everyone for your continued support and finding this blog’s articles useful. My sincerest gratitudes. Anyways, I am sharing another really good solution for all of you guys out there.

Have you deployed Nagios and installed the NRPE plugin on your local and remote servers? Have you by any chance encountered, and was not quite able to solve, this error?

CHECK_NRPE: Socket timeout after 10 seconds

Actually this has also bugged our team a lot and we were really sure (or so we thought) that we have configured Nagios and the NRPE plugin correctly.

We thought it might be a problem with Nagios or the NRPE plugin and we haven’t been able to dedicate a lot of time to find out the problem due to other pressing tasks. But recently, I was able to find the time to diagnose the issue and now I am sharing with you the fix. And yes, it was really really straight forward and so damn simple! DOH!

On your remote machine, make sure that you have installed the NRPE plugin correctly. You can download the NRPE plugin documentation at http://nagios.sourceforge.net/docs/nrpe/NRPE.pdf.

Once, you’ve configured your remote machine’s NRPE already, do local checking first.

# /usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.8


If you get a response with a version of your NRPE, then you’re all set.

And do not forget to open port 5666 on your firewall (iptables or other wise). Refer to the NRPE documentation for more details.

On your monitoring machine, where Nagios was installed, install the NRPE plugin as well. Then do the remote NRPE check.

# /usr/local/nagios/libexec/check_nrpe -H
CHECK_NRPE: Socket timeout after 10 seconds.


Now, don’t be surprised if you get the CHECK_NRPE: Socket timeout error. We now need to make sure that our monitoring machine allows incoming and outgoing connections via port 5666.

To do just that, we open up our IPTables by doing:

# /sbin/iptables -A INPUT -s -p tcp -m tcp –dport 5666 -j ACCEPT
# /sbin/iptables -A OUTPUT -p tcp -m tcp –dport 5666 -j ACCEPT

The first command will allow our monitoring box to accept incoming connections from our remote_host via port 5666 and no other hosts and the second command will allow our remote machine to initiate connection via port 5666 to any remote machine. Of course we can make it tighter by specifying the destination box, but I don’t see the need for the meantime.

After adding the rules, make sure to save your new IPTables rules by doing

# /sbin/iptables-save > /etc/sysconfig/iptables

That’s it!

Try out some of these combinations to your remote host and see what the outputs are.

# /usr/local/nagios/libexec/check_nrpe -H -c check_load -t 120
#/usr/local/nagios/libexec/check_nrpe -H -c check_users
#/usr/local/nagios/libexec/check_nrpe -H -c check_zombie_procs

Enjoy!

Technorati linux, nagios, nrpe, monitoring
Site Search Tags: linux, nagios, nrpe, monitoring

Funny as it may seem, I fell for it (yet again perhaps?). I’ve installed CentOS5 on top of VirtualBox 2.2.2 on my Ubuntu Hardy 8.04 for the sole purpose of testing out ASP.NET hosting using Mono, XSP, and Apache. And I was in for a surprise that I could not access my virtual machine’s Apache web server instance.

I was able to install VirtualBox 2.2.2 vanilla (not the official Ubuntu package) without any problems. I followed it up by installing CentOS5 as a virtual machine. I then proceeded and configured the network interface for the virtual machine on the VirtualBox Management panel, and set it up to use “Bridged” networking.

I proceeded by booting the virtual machine and watched every boot message zip by without a hitch. I then logged into the virtual machine as root and configured the IP address for the network interface. After which, I pinged the IP address and got the reply I want. I then tried to log in via SSH, ang was able to get in. “Hmm… everything seems to be in order. Might as well continue.” I said to myself. Boy was I in for a surprise.

I proceeded and configured the web server, enabling name-based virtual hosting, started Apache and tried to browse the test page. And then… nothing! ACCKK!!! I checked the IPTables rules if there’s something a-miss. So far everything seems to be in order (really?) . Tried a couple more times, and still get a failure from Firefox. Hmmm.. what could it be. Pondering for hours what could have been missed. A colleague suggested to flush the entire set of IPTables rules, which I did and tried accessing the web server. What do you know! I was able to browse the basic landing page. “Hmmm.. there must be something wrong with the IPTables rules” I said whispering.

I proceeded and checked again /etc/sysconfig/iptables and sure enough, I found the culprit. I mistakenly added the IPTables rule that opens port 80 AFTER the reject rule! No wonder port 80 doesn’t open up. I edited the /etc/sysconfig/iptables again, and place the port 80 rule on top of the reject rule, which will then allow it to take effect first, before the reject rule is activated. And then everything worked as it should.

Lesson? Sometimes the obvious things are really hard to find and double checking definitely will prevent the unnecessary debugging for when your system goes to production. I’m just glad this is a simulation system and not production.

I think it was stupid of me not to notice the order of the rules in the iptables file. What do you think?

Technorati linux, virtualization, virtualbox, centos5, apache, webserver, iptables
Site Search Tags: linux, virtualization, virtualbox, centos5, apache, webserver, iptables

Yesterday, a Sunday, was a busy day for me. Although as much as I wanted to spend time with my family for that day, I have to go and pay a visit to a client who is currently planning to migrate fully to Linux.

The problem or problems were simple really. Below are some details regarding their infrastructure.

The Client
Actually I am doing this on a freelance basis. Since I am not available on weekdays because of my day job as a Tech Support and Pre-Sales for a local IT vendor, I asked the client that the planned initial steps for their migration be set on Sundays. The proposal I previously submitted projected the “migration” will take about five days (Sundays).

The client is a real estate developer. Their computers were fixed-function stations with the primary purpose of generating documentation, reporting, email, internet browsing, and billing. Pretty standard office stuff. The migration will not be as difficult.

The Objectives
The client intends to cut down costs in terms of software licensing, increase control over users’ internet use without invading user privacy, increase the security of the main business network by preventing unauthorized access into the main network that might come from the wireless portion of the entire network, and maintain compatibility of pre-established and used document formats.

The Infrastructure
Their existing infrastructure consists of the ff:
- one Systems Admin
- 10 - 20 workstations for documentation, reports, email, browsing, and billing running Windows XP
- no definite specific server/s
- mixed set of printers (Epson, HP, Canon) ranging from Dot Matrix, Deskjet/Bubblejet, and Lasers.
- a wired (copper) network using 10BaseT/UTP
- a wireless network using LinkSys wireless access point/switch/router
- broadband (ADSL) internet connection
- commercial anti-virus
- file and printer sharing
- Applications: PeachTree Accounting, Norton AntiVirus, internal custom systems developed using Clipper, and MS Office.

The Diagnosis
My initial inspection of the infrastructure found their existing inefficient and vulnerable. Here are the initial findings:

1. The network(s) were inefficiently using IP addressing. Each department were using a different IP class for only very few computers. One department uses class B IP addressing (10.0.0.x) and another department uses class C IP addressing (192.168.x.x). Each department only has an average of 4 or 5 computers maximum.

2. The wireless network is directly interconnected to the wired network. The main network gets IP addresses via the DHCP service provided for by the LinkSys Switch Router. Mobile clients that will access internet connection via the wireless access point can access other machines within the main business network.

3. No restrictions and control is implemented for users within the main business network. Users tend to surf and download malicious content through the web.

4. Virus infections were common and left unresolved.

The Recommendation
With prior talks with their System Admin, I recommended SuSE Linux despite the existence of other “Windows-like” Linux distributions due to the fact that a local vendor offers SuSE Linux and can extend support to them locally.

Further recommendations were:
1. Use ONLY class C IP addresses and use subnetting (192.168.0.x, 192.168.1.x, …). Each department will use a subnet of the same IP class.
2. The wireless network will be assigned a separate subnet than the main business network.
3. A “server” will be setup to provide proxy service, routing between the main business network and the wireless router.
4. The “server” will use iptables-based firewall with one network interface designated as a demilitarized zone for the wireless network, and an internal zone for the main business network. This will in turn block those from the DMZ from accessing the main business network.
5. Squid will be used as the internal proxy server which allows for easy installation, maintenance, flexibility in implementing restriction via access control lists (ACLs).
6. Samba will be used for file and printer sharing allowing internal workstations that will still continue using Windows to use network shares and resources.

Already Implemented
I had just started to implement changes to their infrastructure and one day will not be enough for two people (me and the System Admin) to perform the entire implementation. The fact that the Admin is also more inclined to Windows than Linux does not help speed up the implementation.

So far, we were able to accomplish putting up the internal proxy server using Squid. ACLs for Squid will still be added later. The client’s System Admin will do the updating to new IP addressing for the rest of the workstations. File and printer sharing will be done later.

Here’s the diagram of the proposed modification to the network that is in the process of implementation:

- Click picture to enlarge.

This project is very much a first for me to implement, hopefully this “pilot” project will be one of many “migration” projects that I will be doing in the future. Certainly, Linux, for some local businesses is starting to look pretty viable an alternative cost-wise and security-wise. Look forward to more updates regarding this project soon.

If you have any suggestions, feel free to comment. I’d love to hear your suggestions to make the client’s foray into Linuxland more beneficial and worthwhile.