Saturday, July 6, 2019

System Forensics -- Locating Relevant Log Files

Every once in a while you'll get an assignment to figure out something about a system that no one knows anything about.  Someone will point you toward an executable, like a system daemon and wish you on your way with a hearty 'good luck'.  Now what?

A good start would be to take a peek at any relevant log files, but how do you know where to look?  An intellectual game of hide and seek begins.

Perhaps the process status will provide you with a clue; a log directory or log file may be specified in the command line arguments.  If not, what's next?

You can continue by seeing if they use the typical system log conventions, examining the /var/log/ log files for traces of relevant events.  If that path proved unfruitful....then what?

The 'list open files' utility can be a solid next step.  The 'lsof' utility, when provided a process id, will show all files the process is accessing.  A hard-coded log file which may be difficult to locate in the source code (if available) is easily found with this particular utility.  Sorting through the plethora of open file list can take some doing, it's pretty exhaustive and a wealth of information, showing shared libraries the process is using, open sockets, open files,..... If the executable is writing to a log file, it'll be in the lsof file listing.  This comes in extremely handy in finding otherwise elusive log files.

Let's do one quick example; say you know nothing about the apache service but want to know what log files it's using.




root@river:~# ps -aef | grep apache
root      1467     1  0 May25 ?        00:00:00 /usr/sbin/apache2 -k start
www-data  9948  1467  0 20:59 ?        00:00:00 /usr/sbin/apache2 -k start
www-data  9949  1467  0 20:59 ?        00:00:00 /usr/sbin/apache2 -k start


We'll focus on the process 1467.




root@river:~# lsof -p 1467
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF      NODE NAME
apache2 1467 root  cwd    DIR    8,1     4096         2 /
apache2 1467 root  rtd    DIR    8,1     4096         2 /
apache2 1467 root  txt    REG    8,1   662560 114558879 /usr/sbin/apache2
apache2 1467 root  mem    REG    8,1    47600 120590386 /lib/x86_64-linux-gnu/libnss_files-2.23.so
apache2 1467 root  mem    REG    8,1    47648 120590377 /lib/x86_64-linux-gnu/libnss_nis-2.23.so
apache2 1467 root  mem    REG    8,1    93128 120590384 /lib/x86_64-linux-gnu/libnsl-2.23.so
apache2 1467 root  mem    REG    8,1    35688 120590392 /lib/x86_64-linux-gnu/libnss_compat-2.23.so
apache2 1467 root  mem    REG    8,1    22536 114821038 /usr/lib/apache2/modules/mod_status.so
apache2 1467 root  mem    REG    8,1    14344 114833078 /usr/lib/apache2/modules/mod_setenvif.so
apache2 1467 root  mem    REG    8,1    34832 114833033 /usr/lib/apache2/modules/mod_negotiation.so
apache2 1467 root  mem    REG    8,1    63504 114833026 /usr/lib/apache2/modules/mod_mpm_event.so
apache2 1467 root  mem    REG    8,1    18440 114833076 /usr/lib/apache2/modules/mod_mime.so
apache2 1467 root  mem    REG    8,1    18440 114821057 /usr/lib/apache2/modules/mod_filter.so
apache2 1467 root  mem    REG    8,1    10248 114833082 /usr/lib/apache2/modules/mod_env.so
apache2 1467 root  mem    REG    8,1    14344 114821225 /usr/lib/apache2/modules/mod_dir.so
apache2 1467 root  mem    REG    8,1   104864 120586429 /lib/x86_64-linux-gnu/libz.so.1.2.8
apache2 1467 root  mem    REG    8,1    34824 114833080 /usr/lib/apache2/modules/mod_deflate.so
apache2 1467 root  mem    REG    8,1    38928 114833054 /usr/lib/apache2/modules/mod_autoindex.so
apache2 1467 root  mem    REG    8,1    10256 114832998 /usr/lib/apache2/modules/mod_authz_user.so
apache2 1467 root  mem    REG    8,1    10256 114821227 /usr/lib/apache2/modules/mod_authz_host.so
apache2 1467 root  mem    REG    8,1    22544 114833022 /usr/lib/apache2/modules/mod_authz_core.so
apache2 1467 root  mem    REG    8,1    10256 114833069 /usr/lib/apache2/modules/mod_authn_file.so
apache2 1467 root  mem    REG    8,1    10256 114833074 /usr/lib/apache2/modules/mod_authn_core.so
apache2 1467 root  mem    REG    8,1    14352 114833010 /usr/lib/apache2/modules/mod_auth_basic.so
apache2 1467 root  mem    REG    8,1    14344 114833059 /usr/lib/apache2/modules/mod_alias.so
apache2 1467 root  mem    REG    8,1    10256 114833015 /usr/lib/apache2/modules/mod_access_compat.so
apache2 1467 root  mem    REG    8,1    14608 120590365 /lib/x86_64-linux-gnu/libdl-2.23.so
apache2 1467 root  mem    REG    8,1    18976 120586259 /lib/x86_64-linux-gnu/libuuid.so.1.3.0
apache2 1467 root  mem    REG    8,1   166032 120586445 /lib/x86_64-linux-gnu/libexpat.so.1.6.0
apache2 1467 root  mem    REG    8,1    39224 120590381 /lib/x86_64-linux-gnu/libcrypt-2.23.so
apache2 1467 root  mem    REG    8,1  1868984 120590390 /lib/x86_64-linux-gnu/libc-2.23.so
apache2 1467 root  mem    REG    8,1   138696 120590374 /lib/x86_64-linux-gnu/libpthread-2.23.so
apache2 1467 root  mem    REG    8,1   204936 114559448 /usr/lib/x86_64-linux-gnu/libapr-1.so.0.5.2
apache2 1467 root  mem    REG    8,1   159488 114559436 /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0.5.4
apache2 1467 root  mem    REG    8,1   456632 120586274 /lib/x86_64-linux-gnu/libpcre.so.3.13.2
apache2 1467 root  mem    REG    8,1   162632 120590373 /lib/x86_64-linux-gnu/ld-2.23.so
apache2 1467 root  DEL    REG    0,5              22729 /dev/zero
apache2 1467 root    0r   CHR    1,3      0t0         6 /dev/null
apache2 1467 root    1w   CHR    1,3      0t0         6 /dev/null
apache2 1467 root    2w   REG    8,1      279  93324825 /var/log/apache2/error.log
apache2 1467 root    3u  sock    0,8      0t0     20189 protocol: TCP
apache2 1467 root    4u  IPv6  20190      0t0       TCP *:http (LISTEN)
apache2 1467 root    5r  FIFO   0,10      0t0    159430 pipe
apache2 1467 root    6w  FIFO   0,10      0t0    159430 pipe
apache2 1467 root    7w   REG    8,1        0  93336006 /var/log/apache2/other_vhosts_access.log
apache2 1467 root    8w   REG    8,1        0  93323356 /var/log/apache2/access.log



 BAM!  The last two 7 lines show 3 log files worth investigating.  It also shows shared libraries, open sockets,..... The utility essentially examines the /proc virtual file system, which you could do instead if you wished, but lsof is significantly more convenient.

Happy seeking, hope this helps.  It's particularly been useful to me these past few days on a business-logic seeking mission of my own.

No comments:

Post a Comment