← Back to team overview

dhis2-devs team mailing list archive

Re: Server processor use 100%

 

Thanks Bob.

On Thu, Jul 13, 2017 at 10:13 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
wrote:

> Yes, Hannan that is similar to what I have seen a number of times this
> year.  The attacker makes use of atd and/or crontab to execute malicious
> code.  The good thing is that your tomcat was not running as root which
> would be potentially more damaging.
>
> Obviously with access to the tomcat user then access to the database
> itself has been exposed.  There is no indication that the database was the
> target of previous exploits so probably (hopefully) that is your case too.
> It is a really good illustration though of why, when you have multiple
> instances attaching to a database server, you should always use a separate
> database role/user for each.  So when one database is exposed (through
> access to dhis.conf), at least they are not all exposed.
>
> Enjoy your holiday.  I am hoping to get off as well soon :-)
>
> Regards
> Bob
>
> On 13 July 2017 at 16:01, Hannan Khan <hannank@xxxxxxxxx> wrote:
>
>> Dear Bob
>>
>> Sorry for replaying late. I quite busy to complete few incomplete tasks
>> before I am going on holiday tomorrow for a week.
>>
>> I have checked for few day with various options and my conclusion is that
>> the security hole might be created by our old war file (version 16) with
>> Stuart vulnerability which Lars warn all of us earlier. We upgraded all our
>> servers and applications except this server. No suspicious files in the tmp
>> folders.
>>
>> It took control of Tomcat8 user and run SSHD and occupies 100% of 2
>> processors. When we kill the process and remove all war files and stop
>> tomcat8 service it stared ATD command and it also occupy 100% of 2
>> processors. The data seems intact (through query and size). As our all DB
>> servers have similar IP structure we immediately remove tomcat8 service,
>> package and user. The VM server will also be decommissioned and will setup
>> a new server with new cardinals. I will start upgrade work after I return.
>>
>> Thank you for your valuable advice and kind concern.
>>
>> Best regards
>>
>> Hannan
>>
>> On Mon, Jul 10, 2017 at 8:21 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
>> wrote:
>>
>>> Sorry that should have been 'ls -la /tmp'
>>>
>>> On 10 July 2017 at 10:50, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
>>>
>>>> Hi Hannan
>>>>
>>>> There is no circumstance that tomcat user should be running the sshd
>>>> command.  It could be this machine has been compromised.  Unless you have
>>>> some strange setup that you are logging in as tomcat user.
>>>>
>>>> Please contact me directly if you want me to check.
>>>>
>>>> Meanwhile you might want to have a look in /tmp directory and tomcat8
>>>> home directory to see if there are any strange files there:
>>>>
>>>> ls -ls /tmp
>>>>
>>>> You might find that there is a rogue sshd program that has been
>>>> installed there.  Note that if you are running a very old war file your
>>>> risk of compromise is very high.
>>>>
>>>> Bob
>>>>
>>>> On 10 July 2017 at 05:09, Hannan Khan <hannank@xxxxxxxxx> wrote:
>>>>
>>>>> Dear Experts
>>>>>
>>>>> I have an wired situation. one of our DHIS2 server running older war
>>>>> files (version 16), the OS was outdated and we have to upgrade the OS.
>>>>> After installing new OS Ubuntu 16.04 LTS all necessary component Java 8 and
>>>>> Tomcat 7 was installed by after running war file (version 16) after few
>>>>> minutes the tomcat7 is not operational as the processor use is 100%. there
>>>>> is only 1 user logged in and the application server using 2 processor and
>>>>> DB server is separate.
>>>>>
>>>>> After trying several times I remove tomcat7 and install tomcat 8 with
>>>>> same war file, but situation is same. I called it wired as the db size is
>>>>> quite small, user is only few and the listing showing SSHD command by
>>>>> tomcat8 user is using 100% processor.
>>>>>
>>>>> Any idea about the under line reason? need urgent help. Thank you all
>>>>> in advance.
>>>>>
>>>>> Regards
>>>>>
>>>>> Muhammad Abdul Hannan Khan
>>>>> Team Leader
>>>>> Support to the National HMIS
>>>>> MIS, Director General of Health Service
>>>>> Ministry of Health and Family Welfare
>>>>>
>>>>> T +880-2- 58816459 <+880%202-58816459>, 58816412 ext 118
>>>>> F +88 02 58813 875
>>>>> M+88 01819 239 241
>>>>> M+88 01534 312 066
>>>>> E hannank@xxxxxxxxx
>>>>> S hannan.khan.dhaka
>>>>> B hannan-tech.blogspot.com
>>>>> L https://bd.linkedin.com/in/hannankhan
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Muhammad Abdul Hannan Khan
>> Team Leader
>> Support to the National HMIS
>> MIS, Director General of Health Service
>> Ministry of Health and Family Welfare
>>
>> T +880-2- 58816459 <+880%202-58816459>, 58816412 ext 118
>> F +88 02 58813 875
>> M+88 01819 239 241
>> M+88 01534 312 066
>> E hannank@xxxxxxxxx
>> S hannan.khan.dhaka
>> B hannan-tech.blogspot.com
>> L https://bd.linkedin.com/in/hannankhan
>>
>>
>>
>>
>


-- 
Muhammad Abdul Hannan Khan
Team Leader
Support to the National HMIS
MIS, Director General of Health Service
Ministry of Health and Family Welfare

T +880-2- 58816459, 58816412 ext 118
F +88 02 58813 875
M+88 01819 239 241
M+88 01534 312 066
E hannank@xxxxxxxxx
S hannan.khan.dhaka
B hannan-tech.blogspot.com
L https://bd.linkedin.com/in/hannankhan

References