See what's in the DB_CACHE_BUFFER

See what's in you db_cache buffer at that time, log on as SYSDBA and run the following query.
You will see what occupies the db_cache buffer at that time.

OLUMN object_name FORMAT a40
COLUMN number_of_blocks FORMAT 999,999,999,999

SELECT o.object_name, COUNT(1) number_of_blocks
  FROM DBA_OBJECTS o, V$BH bh
 WHERE o.object_id  = bh.objd
   AND o.owner     != 'SYS'
 GROUP BY o.object_name
 ORDER BY count(1);