NSClient++ Help (#1) - nsclient++ Memory Leak? (#133) - Message List

nsclient++ Memory Leak?

Hi,

We have nscp deployed to about 30 server 2003 R2 machines. A few of them have developed a memory leak, causing memory use for nscp to shoot into the 200MB range, and VM size in to the 750MB range. I have no logical explination, as the nsclient configs are identical, and a few machines that have twice as many checks as those exhibiting the leak have memory use still in the below 10MB Realm. Is there a known memory leak in certain plugins that can be disabled? We're using NSClient++ 0.3.1.14 2008-03-12, if it helps.

-J

  • Message #348

    do you use any counter checks? if so what do you check?

    (others have reported problems with HP counters)

    Might also wanna take a look at the handle count.

    MickeM

    • Message #349

      I'm not 100% clear on which functions are counters, but what we're using is 2 service checks (tomcat and one other), Disk Space (C: and D:), CPU Load, and Memory Usage. I assume the latter 2 are based on perfmon "counters"?

      Handle count is 234... 5 threads. Nothing that seems worrisome.

      • Message #350

        well, what I had hoped for was that you checked some specific HP performance counters (as there seems to be a problem with that) but in this case it is standard stuff, and should not hog memory... strange...It would be usefull if you could run the service with the /test command option (or enable debug logging) and send me the log.

        debug logging (look in nsc.ini [log] section) for the /test thingy:

        nsclient++ /stop
        nsclient++ /test
        ... wait for interesting things to happen ...
        exit
        nsclient++ /start
        
        • Message #354

          I ran it for probably about an hour with /test. Glancing through the logs, I don't see anything that looks weird. Also, there was no significant memory increase during the time it was running. I can leave it run over the weekend, but I imagine the log would get fairly large.

        • Message #366

          Any thoughts? This seems like a showstopper.

          • Message #367

            Well, since I am not able to reproduce the error, I cannot really resolve it, if you could provide me with more details or help me figure out whats wrong (ie. try to figure out which checks are causing the problem) I might be able to help.

            But as is I don't know of any memory leaks, though there have been a few reports, the only one we have been able to track down thus far was a bug in the HP performance counters.

            MickeM

            • Message #368

              I'm trying to track it down. I can say it's related to a check though, because every time i run any kind of query, the process gains ~60KB of memory.

            • Message #369

              And with that, i've got it narrowed down to the service checks.

              #define service{ # use generic-service # host_name HOSTNAME # service_description SERVICE NAME # check_command check_nt!SERVICESTATE!-d SHOWALL -l "SERVICENAME" # }

              I have the service name in quotes in the check command because the name contains a space. I hope this is helpful in some way -- please let me know what other information i can provide that would be useful in troubleshooting.

            • Message #370

              Okay, I can say for sure that this is it. It only happens when the service checks are enabled. I disabled all of them on two boxes, and saw no growth throughout the day, where as the other processes had put on ~10MB.

              • Message #409

                Woot. Thanks for all the help -- hope you're feeling better.

                • Message #416

                  Yeah... no biggie just a cold, but it kept me in bed away from my computer :)

                  Anyways, new nightly out now that fixes this issue, and sorry for it, it was a pretty lame bug I ought to write better code then that... :/

                  Check for: NSClient++-Win32-20080514-2247.zip Please test this and I will make a new 0.3.2 version (same thing as the nightly) for tomorrow.

                  MickeM

                  MickeM

              • Message #377

                thank you this is exactly the stuff I was looking for, I shall try to fix this ASAP hopefully early next week (for the 0.3.x branch), sorry for not responding sooner but been ill since I got back from my trip to Germany :(

                MickeM

              • Message #378

                I added a ticket #170 for this so you can track the error there.

                MickeM

Subscriptions