hi,

in todays 17.0.x.y-val (x86_64-slc5-gcc43-opt) rel_0 the trigmon_bs job crashed with a bunch of fatals

logfile:
https://atlasrtt.cern.ch/rtt/rel_0/17.0.x.y-val/build/x86_64-slc5-gcc43-opt/offline/recperftests/trigmon_bs/trigmon_bs_log

toolsvc.rpclv1rawdatavalalg                          info  deleting rpclv1rawdatavalalg 
toolsvc.larcellbuilderfromlarrawchanneltool          info  in destructor 

(pid=18723 ppid=18578) received fatal signal 11 (segmentation fault)
signal context:
  signo  = 11, errno = 0, code = 2 (invalid permissions for mapped object)
  pid    = 156196160, uid = 10923
  value  = (-1397303151, 0x2aaaacb6d891)
  addr   = 0x2aab094f5d40
  stack  = (2, 0, (nil))

  01000000 00000000 23490000 00000000 00000000 00000000 d0c8baac aa2a0000
  0f270000 00000000 c0c43a21 9a2b0000 80de230f ab2a0000 00000000 00000000
  90389df3 aa2a0000 39b2430b ab2a0000 08781b0f ab2a0000 90389df3 aa2a0000
  01000000 00000000 d0fb680f ab2a0000 ffffffff 00000000 68b5a91c ff7f0000
  405d4f09 ab2a0000 06020100 00000000 33000000 00000000 15000000 00000000
  0e000000 00000000 00000000 00000000 405d4f09 ab2a0000 e0b2a91c ff7f0000
  00000000 00000000 00000000 01000000 00000000 00000000 20d8b6ac aa2a0000
  40b4a91c ff7f0000 006523ac aa2a0000 04000014 00000000 10eb60e2 3e000000

stack trace:
 0x2b9a28e8ccbc athena::debugaids::stacktrace(int)+0x4c ??:0   + 0x4c [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]
 0x2b9a28e96732 athena::signal::fataldump(int, siginfo*, void*)+0x152 ??:0   + 0x152 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]
 0x2b9a28e96a44 athena::signal::fatal(int, siginfo*, void*)+0x194 ??:0   + 0x194 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]
 0x3ee260eb10 __restore_rt ??:0   + 0x3ee260eb10 [/lib64/libpthread.so.0]
 0x2aab094f5d40 &lt;unknown function&gt;
py:rawtoesd          info rawtoesd has completed running of athena with exit code -11.
py:rawtoesd         error athena received signal 11. exit code reset to athena exit code 8.
info: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/atlas_error_categories.db
info: read 52 error patterns from file /afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/atlas_error_patterns.db
info: read 61 ignore patterns from file /afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/atlas_error_ignore.db
athenalogchecker start [rawtoesd log] wall-time: sun sep 11 11:27:33 2011 cest, vmem: 286.7578125 mb, rss: 95.046875 mb
py:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_0 ...
py:rawtoesd          info athena initialise()...
py:rawtoesd          info athena execute()...
py:rawtoesd          info athena finalise()...
py:rawtoesd          info done scanning 12845 lines of file rawtoesd.log. summary:
py:rawtoesd          info    ignored : 10
py:rawtoesd          info    warnings: 397
py:rawtoesd          info    errors  : 14
py:rawtoesd          info    fatals  : 7
athenalogchecker stop [rawtoesd log] wall-time: sun sep 11 11:27:34 2011 cest, vmem: 286.88671875 mb, rss: 95.08984375 mb
athenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 01 secs, cpu-time: 00 hrs 00 mins 01 secs, vmem: 0.12890625 mb, rss: 0.04296875 mb
py:rawtoesd          info jobtransform completed for rawtoesd with error code 69999 (exit code 99)
py:rawtoesd          info transform timer stop [rawtoesd] wall-time: sun sep 11 11:27:34 2011 cest, vmem: 286.88671875 mb, rss: 95.1328125 mb
py:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 07 mins 50 secs, cpu-time: 00 hrs 07 mins 21 secs, vmem: 0.796875 mb, rss: 0.80078125 mb
===== job report produced by rawtoesd version unknown =====
command=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/bin/rawtoesd_trf.py inputbsfile=/afs/cern.ch/atlas/project/pmb/data/data11_7tev.00182726.physics_jettauetmiss.merge.raw._lb0215._sfo-5._0001.1_1000evts maxevents=250 autoconfiguration=everything --athenaopts=--stdcmalloc preexec=from@perfmoncomps.perfmonflags@import@jobproperties@as@pmjp,,pmjp.perfmonflags.domallocmonitoring=true,,pmjp.perfmonflags.dopostprocessing=true,,pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true outputdqmonitorfile=mymon.root
exitcode=99
errorcode=69999 (trf_unknown)
errorcategory=trf_unknown,	69999,	unknown transform error (20 times)
errorcategory=trf_segfault,	60010,	segmentation fault
fatals=7
errors=6
warnings=8
errorinfo
  acronym=trf_unknown (69999)
  message=out of memory: saving summary ...
  severity=fatal
  who=athenasummarysvc
  producer=rawtoesd
  stage=execute
errorinfo (count=3)
  acronym=trf_unknown (69999)
  message=can not retrieve larrawchannelcontainer: larrawchannels
  severity=error
  who=toolsvc.larcellbuilderfromlarrawchanneltool
  producer=rawtoesd
  stage=execute
errorinfo (count=3)
  acronym=trf_unknown (69999)
  message=error executing tool larcellbuilderfromlarrawchanneltool
  severity=error
  who=calocellmaker
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=unknown exception caught, while filling histograms
  severity=error
  who=toolsvc.hltcalo
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=unknown exception caught, while filling histograms
  severity=error
  who=toolsvc.hltcalofeb
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=standard std::exception is caught
  severity=fatal
  who=hadtopocluster
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=std::bad_alloc
  severity=error
  who=hadtopocluster
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=standard std::exception is caught
  severity=fatal
  who=athalgseq
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=std::exception
  severity=error
  who=athalgseq
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=standard std::exception is caught
  severity=fatal
  who=athmasterseq
  producer=rawtoesd
  stage=execute
errorinfo
  acronym=trf_unknown (69999)
  message=standard std::exception is caught
  severity=fatal
  who=indetglobalmanager
  producer=rawtoesd
  stage=finalise
errorinfo
  acronym=trf_unknown (69999)
  message=standard std::exception is caught
  severity=fatal
  who=larmonmanager
  producer=rawtoesd
  stage=finalise
errorinfo
  acronym=trf_segfault (60010) segmentation fault
  message=(pid=18723 ppid=18578) received fatal signal 11 (segmentation fault)
  severity=fatal
  producer=rawtoesd
  stage=finalise
workdir=/pool/lsf/rtt/168597434
athcode=8
rundiravailabledisk=172464226304
dbrelease=
athacronym=an unknown exception occurred
rundiruseddisk=6513421
atlasrelease=rel_0
writing jobinfo_rawtoesd.xml file...
