Editing Category:Emergency Diagnostics Check


(Please note that for access credentials you need last-pass access, otherwise contact the person, who has access to it)

Steps : 
      1 : Determine the server host of the application.  
        
        1.1 : You can find it from https://whois.domaintools.com/  or by using command ping <domain name> (Here you can check if the domain is active or not also)
        1.2 : If it is not resolving then find the DNS (you can find it here https://whois.domaintools.com/) and update the records 
        1.3 : Once find the host, login into it using credentials, which will be available from last-pass or contact the concerned person,who has access to last-pass login

      2 : Check whether if we are getting response from the corresponding server.
      
        2.1 : You can check it using command ping <Server IP>

        2.2 : Try to login into the server   (Please note that to access AWS server, SDC and Kiifb server, check the login steps included in the concerned section under 2.2.2)
            
            2.2.1 : If not getting the response 

                  2.2.1.a : AWS Server
                          2.2.1.a.1 : Check the status of the server in console (Console credentials are there in last-pass / contact concerned person)
                          2.2.1.a.2 : If it in stop state, then start it manually by change the instance state to start by right click on it (under EC2 service)
                          2.2.1.a.3 : If it is running but still can't access it, then contact AWS support for identifying the issue
                  2.2.1.b : SDC Server  
                          2.2.1.b.1 : Will not get ping to the server, so if there any access issue contact SDC Team
                  2.2.1.c : Shared Server   
                          2.2.1.c.1 : Contact HostDime team using ticket system (Login credentials are there in last-pass) , chat or through telephone
                  2.2.1.d : HostDime Servers 
                          2.2.1.d.1 : Try to start the vps from console (Login credentials are there in last-pass). The option is available under products >> VPS >> <vps name>
                          2.2.2.d.2 : Still can't access the server, then contact HostDime through ticket system, chat or telephone (Login credentials are there in last-pass)
                  2.2.1.e : Department Servers 
                          2.2.1.e.1 : If it is inaccessible, then contact the concerned team and inform them

             
            2.2.2 : Can access the server

                  2.2.2.a : AWS Server
                          2.2.2.a.1 : Login to server CLI using key
                                    a) Firstly, download key from the Google drive or contact concerned person, who has the key
                                    b) Place the key under a folder and try to access the server from there
                                    c) command to access the server has been available from AWS console "Connect" button under EC2 service
                          2.2.2.a.2 : Check the status of the services and ports
                                    a) The services and ports needed to be checked are apache & postgresql , 80 & 5432
                                    b) status check command is /etc/init.d/<service name> status (may vary according to distros)
                                    c) If it is in stop state, start it by using the command /etc/init.d/<service name> start
                                    d) For checking the port use the command netstat -ntlp | grep <port>
                                    e) If it is not listening check the service configuration files (it vary corresponding to the distros, so find it from internet if you dont know), /etc/services etc..
                          2.2.2.a.3 : Check the server load 
                                    a) check the server load by the command top or ps -auxf
                                    b) check the memory and CPU usage section for finding the process with high resource usage
                                    c) If unwanted kill the process by command kill -9 <process ID>
                                    d) Restart the services / reboot the server for lower down the load
                                    e) Otherwise check postgres slow query log from log file under "/var/lib/pgsql94/data/pg_log" (may vary with distros) and share the long time executing query with                     developer
                                    f) command to find log is tail -f  <log file mentioned above>
                          2.2.3.a.4 : Check the storage
                                    a) check the storage by command df-h
                                    b) using command  du -sh /* , we get the usage of each directories and files under /
                                    c) After checking it, remove unwanted files and directories using rm -rf command
                                    d) If the temp directory "/var/lib/pgsql93/data/base/pgsql_tmp" (may vary according to distros) is using more space, then restart the postgresql by /etc/init.d/postgresql restart. It will release the space used by that directory
                                    e) If the temp directory still using more space, then check with the developer that any query running, which cause the issue
                          2.2.3.a.5 : If there is intermittent issue, then check the logs for finding the root cause
                                    a) check the php error log under the domain's document root and inform developer if the issue is due to the code error
                                    b) check the apache log in "/var/log/apache2/error_log" (may vary according to distros) and solve it accordingly
                                    c) check system log in "/var/log/syslog" and recommend for server upgrade if issue is due to resource deficiency
                                    d) To find the issue from the log use the command tail -f <log file> and recreate the issue
                          2.2.3.a.6 : If still issue persist 
                                    a) check the document root and ensure that everything is correct
                          2.2.3.a.7 : Check the DB connectivity with the application 
                                    a) check the db connection string in "<domian directroy>/application/config/dbconfig.ini"   (may vary according to frameworks)

                  2.2.2.b : SDC Server      - Connect to the server using the whitelisted static IP
                          2.2.2.b.1 : Access the server CLI using the whitelisted static IP
                                    a) Firstly change the office network to Airtel (static IP) and connect to the server CLI using the credentials in Last-pass
                                    b) If there any problem with Airtel network, then change the  network to Asianet and connect to either thumpi or origami AWS server and from there you can connect to SDC server using credentials
                          2.2.2.b.2 : Check the status of the services and corresponding ports
                                    a) The services and ports needed to be checked are apache & postgresql , 80 & 5432
                                    b) status check command is /etc/init.d/<service name> status (may vary according to distros)
                                    c) If it is in stop state, start it by using the command /etc/init.d/<service name> start
                                    d) For checking the port use the command netstat -ntlp | grep <port>
                                    e) If it is not listening check the service configuration files (it vary corresponding to the distros, so find it from internet if you don't know), /etc/services etc.. 
                                    f) If it is SDC shared server, then you need to contact SDC team in-order to know exact issue
                          2.2.2.b.3 : Check the server load
                                    a) check the server load by the command top or ps -auxf
                                    b) check the memory and CPU usage section for finding the process with high resource usage
                                    c) If unwanted kill the process by command kill -9 <process ID>
                                    d) Otherwise check postgres slow query log from log file under "/var/lib/pgsql94/data/pg_log" (may vary with distros) and share the long time executing query with developer
                                    e) command to find log is tail -f  <log file mentioned above>
                          2.2.2.b.4 : Check if any storage issue persist and remove unwanted backup files
                                    a) check the storage by command df-h
                                    b) using command  du -sh /* , we get the usage of each directories and files under /
                                    c) After checking it, remove unwanted files and directories using rm -rf command
                                    d) If the temp directory "/var/lib/pgsql93/data/base/pgsql_tmp" (may vary according to distros) is using more space, then restart the postgresql by /etc/init.d/postgresql restart. It will release the space used by that directory
                                    e) If the temp directory still using more space, then check with the developer that any query running, which cause the issue
                          2.2.2.b.5 : Check the configurations and document root and ensure that  everything is fine
                                    a) You can find the document root from the apache configuration of the corresponding distro
                          2.2.2.b.6 : Check the DB connectivity with the application 
                                    a) check the db connection string in "<domian directroy>/application/config/dbconfig.ini"   (may vary according to frameworks)
                          2.2.2.b.7 : Check the log files for finding the error if any and resolve it accordingly
                                    a) check the php error log under the domain's document root and inform developer if the issue is due to the code error
                                    b) check the apache log in "/var/log/apache2/error_log" (may vary according to distros) and solve it accordingly
                                    c) check system log in "/var/log/syslog" and recommend for server upgrade if issue is due to resource deficiency
                                    d) To find the issue from the log use the command tail -f <log file> and recreate the issue
                          2.2.2.b.8 : If the issue still persist, then contact the SDC team.

                  2.2.2.c : Shared Server 
                          2.2.2.c.1 : Login to the cPanel console using the credentials
                          2.2.2.c.2 : Check whether the document root path is correct 
                                    a) You can confirm it by verifying that the code is under the domain's path which is given in the "Domains" section of cPanel
                                    b) If not, move it to the location / Restore from the backup
                          2.2.2.c.3 : Check if there any unwanted redirection has been given in the redirects
                                    a) You can check it under "Redirects" section of cPanel
                          2.2.2.c.4 : Check if there any  unwanted redirection is there in .htaccess
                                    a) You can find the .htaccess file under document root directory
                          2.2.2.c.5 : If the issue need to resolved from the root level, then contact HostDime Team

                  2.2.2.d : HostDime Servers  - Login to server CLI and cPanel
                          2.2.2.d.1 : Login to server CLI and cPanel using the credentials
                          2.2.2.d.2 : Check the status of the services and corresponding ports 
                                    a) The services and ports needed to be checked are apache & postgresql , 80 & 5432
                                    b) status check command is /etc/init.d/<service name> status
                                    c) If it is in stop state, start it by using the command /etc/init.d/<service name> start
                                    d) For checking the port use the command netstat -ntlp | grep <port>
                                    e) If it is not listening check the service configuration files (it vary corresponding to the distros, so find it from internet if you don't know), /etc/services etc.. 
                          2.2.2.d.3 : Check the server load 
                                    a) check the server load by the command top or ps -auxf
                                    b) check the memory and CPU usage section for finding the process with high resource usage
                                    c) If unwanted kill the process by command kill -9 <process ID>
                                    d) Restart the  services / reboot the server for lower done the server load 
                                    e) Otherwise check postgres slow query log from log file under "/var/lib/pgsql94/data/pg_log" (may vary with distro) and share the long time executing query with developer
                                    f) command to find log is tail -f  <log file mentioned above>
                          2.2.2.d.4 : Remove unwanted backup files if the issue is due to insufficient storage
                                    a) check the storage by command df-h
                                    b) using command  du -sh /* , we get the usage of each directories and files under /
                                    c) After checking it, remove unwanted files and directories using rm -rf command
                                    d) If the temp directory "/var/lib/pgsql93/data/base/pgsql_tmp" (may vary according to distro) is using more space, then restart the postgresql by /etc/init.d/postgresql restart. It will release the space used by that directory
                                    e) If the temp directory still using more space, then check with the developer that any query running, which cause the issue
                          2.2.2.d.5 : Check the DB connectivity with the application 
                                    a) check the db connection string in "<domain directory>/application/config/dbconfig.ini"   (may vary according to frameworks)
                          2.2.2.d.6 : Check the configurations and document root
                                    a) You can find the document root from the apache configuration of the corresponding distro
                          2.2.2.d.7 : Check that all the necessary codes are there under the domain document root
                          2.2.2.d.8 : Check if there any unwanted redirections are specified for the domain in redirects section
                                    a) You can check it under "Redirects" section of cPanel
                          2.2.2.d.9 : If the issue still can't be identified, then contact the HostDime Team by opening a support ticket

                  2.2.2.e : Department Servers - Currently Kiifb and KWA server are under this category

                          1 : Kiifb Sever
                            1.a : Connect to the corresponding server using vpn and credentials
                                a) For connecting to VPN, we need the vpn client configuration which can be available from "https://122.15.70.117:444/userportal/webpages/myaccount/login.jsp"  
                                b) Download the client configuration and install it (For windows just extract and install its corresponding client and connect to server using putty by running the vpn)
                                   For Linux OS 1. Download and extract the file
                                                2. Install OpenVpn using the command sudo apt install openvpn
                                                3. Open Terminal and Navigate to where the ovpn file was downloaded and execute the following command
                                                    sudo openvpn --config <configuration-File.ovpn> 
                                                4.  Enter username and password
                                                5. Then open another terminal and connect to the server using the credentials
                            1.b : Check the status of the services and corresponding ports 
                                a) The services and ports needed to be checked are apache & postgresql , 80 & 5432
                                b) status check command is /etc/init.d/<service name> status ( may vary according to distros)
                                c) If it is in stop state, start it by using the command /etc/init.d/<service name> start
                                d) For checking the port use the command netstat -ntlp | grep <port>
                                e) If it is not listening check the service configuration files (it vary corresponding to the distros, so find it from internet if you dont know), /etc/services etc.. 
                            1.c : Check the server load 
                                a) check the server load by the command top or ps -auxf
                                b) check the memory and CPU usage section for finding the process with high resource usage
                                c) If unwanted kill the process by command kill -9 <process ID>
                                d) Restart the services
                                e) Otherwise check postgres slow query log from log file under "/var/lib/pgsql94/data/pg_log" (may vary with distro) and share the long time executing query with developer
                                f) command to find log is tail -f  <log file mentioned above>
                            1.d : Remove unwanted backup files if the issue is due to insufficient storage
                                a) check the storage by command df-h
                                b) using command  du -sh /* , we get the usage of each directories and files under /
                                c) After checking it, remove unwanted files and directories using rm -rf command
                            1.e : Insufficient storage issue may also occur due to pgsql temp data accumilation 
                                a) If the temp directory "/var/lib/pgsql93/data/base/pgsql_tmp" (may vary according to distro) is using more space, then restart the postgresql by /etc/init.d/postgresql restart. It will release the space used by that directory
                                b) If the temp directory still using more space, then check with the developer that any query running, which cause the issue
                            1.f : Check the DB connectivity with the application 
                                a) check the db connection string in "<domian directroy>/application/config/dbconfig.ini"   (may vary according to frameworks)
                            1.g : Check the configurations and document root
                                a) You can find the document root from the apache configuration of the coresponding distro
                            1.h : Check the logs for finding errors and fix it accordingly
                                a) check the php error log under the domain's document root and inform developer if the issue is due to the code error
                                b) check the apache log in "/var/log/apache2/error_log" (may vary according to distros) and solve it accordingly
                                c) check system log in "/var/log/syslog" and recommend for server upgrade if issue is due to resource deficiency
                                d) To find the issue from the log use the command tail -f <log file> and recreate the issue
                            1.i : If there any firewall related issue contact the concerned team of kiifb

                          2 : KWA Server
                            2.a : We have cPanel user access, So login to the console and CLI using credentials
                            2.b : Check the serives and ports status 
                                a) The services and ports needed to be checked are apache & postgresql , 80 & 5432
                                b) status check command is service <service name> status
                                c) If it is in stop state, start it by using the command service <service name> start
                                d) For checking the port use the command netstat -ntlp | grep <port>
                                e) If it is not listening check the service configuration files (it vary corresponding to the distros, so find it from internet if you dont know), /etc/services etc..  
                            2.c : Remove old backup for avioding the space issue
                                a) check the storage by command df-h
                                b) using command  du -sh /* , we get the usage of each directories and files under /
                                c) After checking it, remove unwanted files and directories using rm -rf command
                            2.d : Check the DB connectivity with the application 
                                a) check the db connection string in "<domian directroy>/application/config/dbconfig.ini"   (may vary according to frameworks)
                            2.e : Check everything is fine under the document root
                                a) You can find the document root from the apache configuration of the coresponding distro
                            2.f : Check if there any unwanted redirections are configured
                                a) You can check it under "Redirects" section of cPanel
                            2.g : Check the logs for find the error and fix it
                                a) check the php error log under the domain's document root and inform developer if the issue is due to the code error
                                b) check the apache log in "/var/log/apache2/error_log" (may vary according to distros) and solve it accordingly
                                c) check system log in "/var/log/syslog" and recommend for server upgrade if issue is due to resource deficiency
                                d) To find the issue from the log use the command tail -f <log file> and recreate the issue
                            2.h : If need root access, infrom the issue to concerned person and collect root credentials from them

      3 : when issue is resolved, infrom the concerned person, who initimated the issue.