linux - Filter log file entries based on date range -

my server having unusually high cpu usage, , can see apache using way memory. have feeling, i'm being dos'd single ip - maybe can me find him?

i've used following line, find 10 "active" ips:

cat access.log | awk '{print $1}' |sort  |uniq -c |sort -n |tail 

the top 5 ips have 200 times many requests server, "average" user. however, can't find out if these 5 frequent visitors, or attacking servers.

is there way, specify above search time interval, eg. last 2 hours or between 10-12 today?

cheers!

updated 23 oct 2011 - commands needed:

get entries within last x hours [here 2 hours]

awk -vdate=`date -d'now-2 hours' +[%d/%b/%y:%h:%m:%s` ' { if ($4 > date) print date fs $4}' access.log 

get active ips within last x hours [here 2 hours]

awk -vdate=`date -d'now-2 hours' +[%d/%b/%y:%h:%m:%s` ' { if ($4 > date) print $1}' access.log | sort  |uniq -c |sort -n | tail 

get entries within relative timespan

awk -vdate=`date -d'now-4 hours' +[%d/%b/%y:%h:%m:%s` -vdate2=`date -d'now-2 hours' +[%d/%b/%y:%h:%m:%s` ' { if ($4 > date && $4 < date2) print date fs date2 fs $4}' access.log 

get entries within absolute timespan

awk -vdate=`date -d '13:20' +[%d/%b/%y:%h:%m:%s` -vdate2=`date -d'13:30' +[%d/%b/%y:%h:%m:%s` ' { if ($4 > date && $4 < date2) print $0}' access.log  

get active ips within absolute timespan

awk -vdate=`date -d '13:20' +[%d/%b/%y:%h:%m:%s` -vdate2=`date -d'13:30' +[%d/%b/%y:%h:%m:%s` ' { if ($4 > date && $4 < date2) print $1}' access.log | sort  |uniq -c |sort -n | tail 

yes, there multiple ways this. here how go this. starters, no need pipe output of cat, open log file awk.

awk -vdate=`date -d'now-2 hours' +[%d/%b/%y:%h:%m:%s` '$4 > date {print date, $0}' access_log 

assuming log looks mine (they're configurable) date stored in field 4. , bracketed. doing above finding within last 2 hours. note -d'now-2 hours' or translated literally minus 2 hours me looks this: [10/oct/2011:08:55:23

so doing storing formatted value of 2 hours ago , comparing against field four. conditional expression should straight forward.i printing date, followed output field separator (ofs -- or space in case) followed whole line $0. use previous expression , print $1 (the ip addresses)

awk -vdate=`date -d'now-2 hours' +[%d/%b/%y:%h:%m:%s` '$4 > date {print $1}' | sort  |uniq -c |sort -n | tail 

if wanted use range specify 2 date variables , construct expression appropriately.

so if wanted find between 2-4hrs ago expression might looks this

awk -vdate=`date -d'now-4 hours' +[%d/%b/%y:%h:%m:%s` -vdate2=`date -d'now-2 hours' +[%d/%b/%y:%h:%m:%s` '$4 > date && $4 < date2 {print date, date2, $4} access_log' 

here question answered regarding dates in bash might find helpful. print date monday of current week (in bash)