(86559,'atlas','error in mb_pu_mu30_d549_wtrigtruth / unknown exception occured','hi,\r\n\r\nthere was an error in mb_pu_mu30_d549_wtrigtruth / unknown exception occured for todays 17.0.x.y (x86_64-slc5-gcc43-opt) rel5\r\n\r\nthe logfile can be found here:\r\nhttps://atlasrtt.cern.ch/rtt/rel_5/17.0.x.y/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/mb_pu_mu30_d549_wtrigtruth/mb_pu_mu30_d549_wtrigtruth_log\r\n\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_5/installarea/share/bin/reco_trf.py ami=f380 autoconfiguration=everything inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000001.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000002.pool.root.1 outputaodfile=myaod.pool.root outputesdfile=myesd.pool.root outputhistfile=myhist.root\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=10\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 11000 (exit code 10)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: fri sep 09 06:21:53 2011 cest, vmem: 317.890625 mb, rss: 101.92578125 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 00 hrs 13 mins 37 secs, cpu-time: 00 hrs 11 mins 07 secs, vmem: 1.2734375 mb, rss: 1.30859375 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_5/installarea/share/bin/reco_trf.py ami=f380 autoconfiguration=everything inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000001.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000002.pool.root.1 outputaodfile=myaod.pool.root outputesdfile=myesd.pool.root outputhistfile=myhist.root\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=10\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=26535653\r\nrundiravailabledisk=123991117824\r\nworkdir=/pool/lsf/rtt/168069477\r\nathcode=8\r\nrundiravailabledisk=123991355392\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=26314907\r\natlasrelease=rel_5\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\ncheers michael','100','5211','2011-09-09 16:26:02','5','none','closed','none',';error;mb_pu_mu30_d549_wtrigtruth;unknown;exception;occur;today;x86_64;slc5;gcc43;rel5;logfile;http;atlasrtt.cern.ch;rel_5;build;x86_64;slc5;gcc43;offline;recjobtransformtest;mb_pu_mu30_d549_wtrigtruth;mb_pu_mu30_d549_wtrigtruth_log;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_5;installarea;share;reco_trf.py;autoconfiguration;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;outputaodfile;myaod.pool.root;outputesdfile;myesd.pool.root;outputhistfile;myhist.root;exitcode;errorcode;ath_failure;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;frus;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;atlasproduction;rel_5;installarea;share;reco_trf.py;autoconfiguration;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;outputaodfile;myaod.pool.root;outputesdfile;myesd.pool.root;outputhistfile;myhist.root;exitcode;errorcode;ath_failure;fatal;error;warning;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_5;write;jobinfo_recotrf.xml;file;michael;error;mb_pu_mu30_d549_wtrigtruth;unknown;exception;occur'),(86563,'atlas','esd to esd re-reconstruction fails for convertedmuidcbtracks','hi,\r\n\r\ntrying re-reconstruction of esds with 17.0.3.4 one gets this error:\r\nstoregatesvc                                      warning  setupproxy:: error setting up proxy for key convertedmuidcbtracks and clid 117682415\r\n pre-existing valid dataproxy @0x17fa7570 found in store for key convertedmuidcbtracks with clid 117682415\r\nstoregatesvc                                      warning record: problem setting up the proxy for object @0x34eacfc0\r\n recorded with key convertedmuidcbtracks of type trackcollection (clid 117682415) in dataobject @0x34ee4078\r\nmuidcombined                                        error could not store the combined tracks\r\nathalgseq                                            info execute of [muidcombined] did not succeed\r\nathmasterseq                                         info execute of [athalgseq] did not succeed\r\n\r\nto reproduce, run\r\nreco_trf.py maxevents=10 inputesdfile=mc11_valid.108351.pythia8_minbias_inelastic.recon.esd.e816_s1310_s1300_r2666_tid505772_00/esd.505772._000001.pool.root.1 outputesdfile=./newesd.root preexec=\' rec.doesd=true;rec.dotrigger=false\' autoconfiguration=everything \r\n\r\ncould this be fixed, please ?\r\n\r\nthanks, rolf\r\n\r\nbtw - i am using a patched patjobtransforms:\r\n&gt; svn diff\r\nindex: scripts/reco_trf.py\r\n===================================================================\r\n--- scripts/reco_trf.py (revision 457748)\r\n+++ scripts/reco_trf.py (working copy)\r\n@@ -138,15 +138,15 @@\r\n         if(allok and self.substepisexecuted(\'e2e\')):\r\n             domerge=true\r\n             dic=self.dicesdtoesd.copy()\r\n-            print &quot;esdtoesd dic:&quot;,dic\r\n-            if \'preexec\' in dic:\r\n-                if dic[\'preexec\'].find(&quot;rec.doesd.set_value_and_lock(true)&quot;)&gt;0 or dic[\'preexec\'].find(&quot;rec.doesd=true&quot;)&gt;0:\r\n-                    print &quot;user wants re-reconstruct on esd !!!&quot;\r\n-                    domerge=false\r\n-            if domerge:\r\n-                esd = mergepooljobtransform(dic)\r\n-            else:\r\n-                esd = esdtoesdjobtransform(dic)\r\n+            #print &quot;esdtoesd dic:&quot;,dic\r\n+            #if \'preexec\' in dic:\r\n+            #    if dic[\'preexec\'].find(&quot;rec.doesd.set_value_and_lock(true)&quot;)&gt;0 or dic[\'preexec\'].find(&quot;rec.doesd=true&quot;)&gt;0:\r\n+            #        print &quot;user wants re-reconstruct on esd !!!&quot;\r\n+            #        domerge=false\r\n+            #if domerge:\r\n+            esd = mergepooljobtransform(dic)\r\n+            #else:\r\n+            #    esd = esdtoesdjobtransform(dic)\r\n             esd._lastinchain=false\r\n             esd.setparent(self)\r\n             esd.setjobreportoptions(\'summary\')\r\n','396','1658','2011-09-09 17:23:30','5','fixed','closed','muonidentification/muid',';reconstruction;17.0.3.4;error;storegatesvc;warn;setupproxy;error;proxy;convertedmuidcbtrack;clid;exist;valid;dataproxy;store;convertedmuidcbtrack;clid;storegatesvc;warn;record;problem;proxy;object;record;convertedmuidcbtrack;type;trackcollection;clid;dataobject;muidcombine;error;store;combine;track;athalgseq;info;execute;muidcombine;succeed;athmasterseq;info;execute;athalgseq;succeed;reproduce;reco_trf.py;maxevent;inputesdfile;mc11_valid;pythia8_minbias_inelastic.recon.esd.e816_s1310_s1300_r2666_tid505772_00;pool.root;outputesdfile;newesd.root;preexec;rec.doesd;true;rec.dotrigger;false;autoconfiguration;rolf;patch;patjobtransform;diff;index;script;reco_trf.py;script;reco_trf.py;revision;script;reco_trf.py;work;copy;allok;self.substepisexecuted;domerge;true;self.dicesdtoesd.copy;print;esdtoesd;preexec;preexec;rec.doesd.set_value_and_lock;true;preexec;rec.doesd;true;print;user;want;reconstruct;domerge;false;domerge;mergepooljobtransform;esdtoesdjobtransform;print;esdtoesd;preexec;preexec;rec.doesd.set_value_and_lock;true;preexec;rec.doesd;true;print;user;want;reconstruct;domerge;false;domerge;mergepooljobtransform;esdtoesdjobtransform;false;esd.setparent;self;esd.setjobreportoptions;summary;reconstruction;fail;convertedmuidcbtrack;muonidentification;muid'),(86575,'atlas','cannot create tool rec::muontrackquery','hello,\r\n\r\nthe atn test recexrecotest_testconfiguration#recoshift_esd is crashing in today\'s 17.x.0 rel_6 with the following error:\r\n\r\ntoolsvc                                             error cannot create tool rec::muontrackquery (no factory found)\r\ntoolsvc.muonidhelper                                fatal failed to retrieve tool muontrackquery = publictoolhandle(\'rec::muontrackquery/muontrackquery\')\r\nnameauditor                                          info just exited toolsvc.muonidhelper with auditor trigger initialize\r\nmemstatauditor                                       info memory usage has changed after  toolsvc.muonidhelper 	initialization method 	virtual size = 1182 mb 	resident set size = 723.371 mb deltavsize = 0.417969  mb \r\ntoolsvc                                             error error initializing tool \'toolsvc.muonidhelper\'\r\nmuidmooretpcreator                                  fatal failed to retrieve tool muonidentificationhelper = publictoolhandle(\'rec::muonidentificationhelper/muonidhelper\')\r\nnameauditor                                          info just exited muidmooretpcreator with auditor trigger initialize\r\nmemstatauditor                                       info memory usage has changed after  muidmooretpcreator 	initialization method 	virtual size = 1182 mb 	resident set size = 723.383 mb deltavsize = 0  mb \r\nathalgseq                                           error unable to initialize algorithm muidmooretpcreator\r\nnameauditor                                          info just exited athalgseq with auditor trigger initialize\r\nathmasterseq                                        error unable to initialize algorithm athalgseq\r\nnameauditor                                          info just exited athmasterseq with auditor trigger initialize\r\nathenaeventloopmgr                                  error unable to initialize algorithm: athmasterseq\r\nathenaeventloopmgr                                  error failed to initialize base class minimaleventloopmgr\r\nnameauditor                                          info just exited athenaeventloopmgr with auditor trigger initialize\r\nservicemanager                                      error unable to initialize service: athenaeventloopmgr\r\npy:athena            info leaving with code 33: &quot;failure in initialization&quot;\r\n\r\nthe full logfile can be found here:\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x064bs5g4recopt/nicos_testlog_rel_6/reconstruction_recexample_recexrecotest_26__recexrecotest_testconfiguration__recoshift_esd__a.html\r\n\r\ncould some of the experts please take a look?\r\nthanks.\r\ncheers,\r\ndaniela\r\nps = not sure about what is causing the crash first, i leave the category and the item group empty','100','5211','2011-09-10 10:24:47','5','fixed','closed','muonidentification/muid',';hello;test;recexrecotest_testconfiguration;recoshift_esd;crash;today\'s;rel_6;follow;error;toolsvc;error;create;tool;muontrackquery;factory;toolsvc.muonidhelper;fatal;fail;retrieve;tool;muontrackquery;publictoolhandle;muontrackquery;muontrackquery;nameauditor;info;just;exit;toolsvc.muonidhelper;auditor;trigger;initialize;memstatauditor;info;memory;usage;change;toolsvc.muonidhelper;initialization;method;virtual;size;resident;size;deltavsize;toolsvc;error;error;initialize;tool;toolsvc.muonidhelper;muidmooretpcreator;fatal;fail;retrieve;tool;muonidentificationhelper;publictoolhandle;muonidentificationhelper;muonidhelper;nameauditor;info;just;exit;muidmooretpcreator;auditor;trigger;initialize;memstatauditor;info;memory;usage;change;muidmooretpcreator;initialization;method;virtual;size;resident;size;deltavsize;athalgseq;error;unable;initialize;algorithm;muidmooretpcreator;nameauditor;info;just;exit;athalgseq;auditor;trigger;initialize;athmasterseq;error;unable;initialize;algorithm;athalgseq;nameauditor;info;just;exit;athmasterseq;auditor;trigger;initialize;athenaeventloopmgr;error;unable;initialize;algorithm;athmasterseq;athenaeventloopmgr;error;fail;initialize;base;class;minimaleventloopmgr;nameauditor;info;just;exit;athenaeventloopmgr;auditor;trigger;initialize;servicemanager;error;unable;initialize;service;athenaeventloopmgr;py:athena;info;leave;code;failure;initialization;logfile;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x064bs5g4recopt;nicos_testlog_rel_6;reconstruction_recexample_recexrecotest_26__recexrecotest_testconfiguration__recoshift_esd__a.html;expert;look;danielum;sure;cause;crash;leave;category;item;group;create;tool;muontrackquery;muonidentification;muid'),(86599,'atlas','error in trigmon_bs','hi,\r\n\r\nin todays 17.0.x.y-val (x86_64-slc5-gcc43-opt) rel_0 the trigmon_bs job crashed with a bunch of fatals\r\n\r\nlogfile:\r\nhttps://atlasrtt.cern.ch/rtt/rel_0/17.0.x.y-val/build/x86_64-slc5-gcc43-opt/offline/recperftests/trigmon_bs/trigmon_bs_log\r\n\r\ntoolsvc.rpclv1rawdatavalalg                          info  deleting rpclv1rawdatavalalg \r\ntoolsvc.larcellbuilderfromlarrawchanneltool          info  in destructor \r\n\r\n(pid=18723 ppid=18578) received fatal signal 11 (segmentation fault)\r\nsignal context:\r\n  signo  = 11, errno = 0, code = 2 (invalid permissions for mapped object)\r\n  pid    = 156196160, uid = 10923\r\n  value  = (-1397303151, 0x2aaaacb6d891)\r\n  addr   = 0x2aab094f5d40\r\n  stack  = (2, 0, (nil))\r\n\r\n  01000000 00000000 23490000 00000000 00000000 00000000 d0c8baac aa2a0000\r\n  0f270000 00000000 c0c43a21 9a2b0000 80de230f ab2a0000 00000000 00000000\r\n  90389df3 aa2a0000 39b2430b ab2a0000 08781b0f ab2a0000 90389df3 aa2a0000\r\n  01000000 00000000 d0fb680f ab2a0000 ffffffff 00000000 68b5a91c ff7f0000\r\n  405d4f09 ab2a0000 06020100 00000000 33000000 00000000 15000000 00000000\r\n  0e000000 00000000 00000000 00000000 405d4f09 ab2a0000 e0b2a91c ff7f0000\r\n  00000000 00000000 00000000 01000000 00000000 00000000 20d8b6ac aa2a0000\r\n  40b4a91c ff7f0000 006523ac aa2a0000 04000014 00000000 10eb60e2 3e000000\r\n\r\nstack trace:\r\n 0x2b9a28e8ccbc athena::debugaids::stacktrace(int)+0x4c ??:0   + 0x4c [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0x2b9a28e96732 athena::signal::fataldump(int, siginfo*, void*)+0x152 ??:0   + 0x152 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0x2b9a28e96a44 athena::signal::fatal(int, siginfo*, void*)+0x194 ??:0   + 0x194 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0x3ee260eb10 __restore_rt ??:0   + 0x3ee260eb10 [/lib64/libpthread.so.0]\r\n 0x2aab094f5d40 &lt;unknown function&gt;\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code -11.\r\npy:rawtoesd         error athena received signal 11. exit code reset to athena exit code 8.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/atlas_error_categories.db\r\ninfo: read 52 error patterns from file /afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: sun sep 11 11:27:33 2011 cest, vmem: 286.7578125 mb, rss: 95.046875 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_0 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info athena finalise()...\r\npy:rawtoesd          info done scanning 12845 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 10\r\npy:rawtoesd          info    warnings: 397\r\npy:rawtoesd          info    errors  : 14\r\npy:rawtoesd          info    fatals  : 7\r\nathenalogchecker stop [rawtoesd log] wall-time: sun sep 11 11:27:34 2011 cest, vmem: 286.88671875 mb, rss: 95.08984375 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 01 secs, cpu-time: 00 hrs 00 mins 01 secs, vmem: 0.12890625 mb, rss: 0.04296875 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: sun sep 11 11:27:34 2011 cest, vmem: 286.88671875 mb, rss: 95.1328125 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 07 mins 50 secs, cpu-time: 00 hrs 07 mins 21 secs, vmem: 0.796875 mb, rss: 0.80078125 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_0/installarea/share/bin/rawtoesd_trf.py inputbsfile=/afs/cern.ch/atlas/project/pmb/data/data11_7tev.00182726.physics_jettauetmiss.merge.raw._lb0215._sfo-5._0001.1_1000evts maxevents=250 autoconfiguration=everything --athenaopts=--stdcmalloc preexec=from@perfmoncomps.perfmonflags@import@jobproperties@as@pmjp,,pmjp.perfmonflags.domallocmonitoring=true,,pmjp.perfmonflags.dopostprocessing=true,,pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true outputdqmonitorfile=mymon.root\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (20 times)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault\r\nfatals=7\r\nerrors=6\r\nwarnings=8\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=out of memory: saving summary ...\r\n  severity=fatal\r\n  who=athenasummarysvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo (count=3)\r\n  acronym=trf_unknown (69999)\r\n  message=can not retrieve larrawchannelcontainer: larrawchannels\r\n  severity=error\r\n  who=toolsvc.larcellbuilderfromlarrawchanneltool\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo (count=3)\r\n  acronym=trf_unknown (69999)\r\n  message=error executing tool larcellbuilderfromlarrawchanneltool\r\n  severity=error\r\n  who=calocellmaker\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=unknown exception caught, while filling histograms\r\n  severity=error\r\n  who=toolsvc.hltcalo\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=unknown exception caught, while filling histograms\r\n  severity=error\r\n  who=toolsvc.hltcalofeb\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=hadtopocluster\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=std::bad_alloc\r\n  severity=error\r\n  who=hadtopocluster\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=athalgseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=std::exception\r\n  severity=error\r\n  who=athalgseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=athmasterseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=indetglobalmanager\r\n  producer=rawtoesd\r\n  stage=finalise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=larmonmanager\r\n  producer=rawtoesd\r\n  stage=finalise\r\nerrorinfo\r\n  acronym=trf_segfault (60010) segmentation fault\r\n  message=(pid=18723 ppid=18578) received fatal signal 11 (segmentation fault)\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=finalise\r\nworkdir=/pool/lsf/rtt/168597434\r\nathcode=8\r\nrundiravailabledisk=172464226304\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=6513421\r\natlasrelease=rel_0\r\nwriting jobinfo_rawtoesd.xml file...\r\n','100','5211','2011-09-11 16:48:32','5','none','closed','none',';today;x86_64;slc5;gcc43;rel_0;trigmon_bs;crash;bunch;fatal;logfile;http;atlasrtt.cern.ch;rel_0;build;x86_64;slc5;gcc43;offline;recperftest;trigmon_bs;trigmon_bs_log;toolsvc.rpclv1rawdatavalalg;info;delete;rpclv1rawdatavalalg;toolsvc.larcellbuilderfromlarrawchanneltool;info;destructor;ppid;receive;fatal;signal;segmentation;fault;signal;context;signo;errno;code;invalid;permission;object;value;addr;stack;stack;trace;athena;debugaid;stacktrace;atla;software;release;17.0.3;atlascore;17.0.3;installarea;x86_64;slc5;gcc43;libcxxutils.so;athena;signal;fataldump;siginfo;void;atla;software;release;17.0.3;atlascore;17.0.3;installarea;x86_64;slc5;gcc43;libcxxutils.so;athena;signal;fatal;siginfo;void;atla;software;release;17.0.3;atlascore;17.0.3;installarea;x86_64;slc5;gcc43;libcxxutils.so;lib64;libpthread.so;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;py:rawtoesd;error;athena;receive;signal;exit;code;reset;athena;exit;code;info;read;error;category;file;atla;software;build;nightly;atlasproduction;rel_0;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;build;nightly;atlasproduction;rel_0;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;build;nightly;atlasproduction;rel_0;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_0;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;athena;finalise;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_0;installarea;share;rawtoesd_trf.py;inputbsfile;atla;project;datum;data11_7tev;physics_jettauetmiss.merge.raw;maxevent;autoconfiguration;athenaopt;stdcmalloc;preexec;perfmoncomps.perfmonflags;import;jobproperty;pmjp;pmjp.perfmonflags.domallocmonitoring;true;pmjp.perfmonflags.dopostprocessing;true;pmjp.perfmonflags.dosemidetailedmonitoringfullprint;true;outputdqmonitorfile;mymon.root;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;errorcategory;trf_segfault;segmentation;fault;fatal;error;warning;errorinfo;acronym;trf_unknown;message;memory;save;summary;severity;fatal;athenasummarysvc;producer;rawtoesd;stage;execute;errorinfo;count;acronym;trf_unknown;message;retrieve;larrawchannelcontainer;larrawchannel;severity;error;toolsvc.larcellbuilderfromlarrawchanneltool;producer;rawtoesd;stage;execute;errorinfo;count;acronym;trf_unknown;message;error;execute;tool;larcellbuilderfromlarrawchanneltool;severity;error;calocellmaker;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;unknown;exception;catch;fill;histogram;severity;error;toolsvc.hltcalo;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;unknown;exception;catch;fill;histogram;severity;error;toolsvc.hltcalofeb;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;hadtopocluster;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;bad_alloc;severity;error;hadtopocluster;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;athalgseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;exception;severity;error;athalgseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;athmasterseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;indetglobalmanager;producer;rawtoesd;stage;finalise;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;larmonmanager;producer;rawtoesd;stage;finalise;errorinfo;acronym;trf_segfault;segmentation;fault;message;ppid;receive;fatal;signal;segmentation;fault;severity;fatal;producer;rawtoesd;stage;finalise;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_0;write;jobinfo_rawtoesd.xml;file;error;trigmon_bs'),(86656,'atlas','muidcalotrackstateonsurface extrapolates in wrong direction','these lines in muidcalotrackstateonsurface:\r\n\r\nelse if (dynamic_cast&lt;const trk::straightlinesurface*&gt;(parameters.associatedsurface()))\r\n    {\r\n      if (parameters.associatedsurface()-&gt;center().mag()      &gt; 1.2*meter) trackoutwards = false;\r\n\r\ngive the wrong result - trt straws in the endcap can be up to three meters away from the origin, and we clearly want to track outward when starting in the trt.\r\n\r\nthe net effect of the bug is a significant underestimation of the scattering sigmas in the calo, by up to an order of magnitude at low momentum. the scattering sigmas are being evaluated with a wrong momentum (p+de instead of p, and p instead of p-de).\r\n\r\nthis bug affects all combined tracks fitted with the globalchi2fitter. \r\n','396','721','2011-09-13 01:36:25','7','fixed','closed','none',';line;muidcalotrackstateonsurface;dynamic_cast;parameters.associatedsurface;parameters.associatedsurface;center;meter;trackoutward;false;wrong;result;straw;endcap;meter;away;origin;clearly;want;track;outward;start;effect;significant;underestimation;scatter;sigma;calo;order;magnitude;momentum;scatter;sigma;evaluate;wrong;momentum;instead;instead;affect;combine;track;globalchi2fitter;muidcalotrackstateonsurface;extrapolate;wrong;direction'),(86681,'atlas','desd merge failed by trackcollection_combined_tracks (run188949)','hi, \r\n\r\ntrackcollection_combined_tracks runs only if toroid field is off. \r\nthis caused the problems in desd file merging. \r\n- lb0120-lb0137._0001.job of data11_7tev.00188949.physics_minbias.recon.desd_mbias.f403_m976.dpdmerge\r\n- lb0065-lb0167._0001.job of  data11_7tev.00188949.physics_egamma.recon.desd_sglel.f403_m976.dpdmerge\r\n- lb0065-lb0168._0001.job of data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge\r\n\r\nthis problem has been seen in \r\nbug #81109: fast repro merge task 307469: streamesd error incorrect object/count\r\n\r\ni\'ll forward this to atlas reconstruction. \r\n\r\ncheers, \r\nsusumu \r\n\r\n+verbatim+\r\ninput file(s):\r\n--------------------------------------------------------------------------------\r\nstage_svcclass=t0merge\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0065._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0066._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0067._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0068._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0069._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0070._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0071._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0072._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0073._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0074._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0075._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0076._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0077._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0078._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0079._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0164._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0165._sfo-10._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0165._sfo-11._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0165._sfo-12._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0165._sfo-4._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0165._sfo-9._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0166._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0167._sfo-all._0001.1\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403._lb0168._sfo-all._0001.1\r\n\r\n\r\ntier-0 transformation command run:\r\n--------------------------------------------------------------------------------\r\nsource /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/setup/setuptrf.sh /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/patches/atlasproduction-17.0.3.4 atlasproduction-17.0.3.4 /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/setup/specialsetup_tier0.sh ; python -u `which merging_trf.py` --argdict=/afs/cern.ch/atlas/tzero/prod1/log2/13439/13439038_data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.3//data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.argdict.gpickle  &amp;&gt; data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.tmp.log\r\n\r\n\r\nami configuration tag (more details):\r\n--------------------------------------------------------------------------------\r\nsee http://ami.in2p3.fr:8080/opencms/opencms/ami/www/referencetables/ --&gt; configurationtag --&gt; interpret --&gt; m976\r\n\r\n\r\nlogfile location:\r\n--------------------------------------------------------------------------------\r\nstage_svcclass=t0merge\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00188949/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task.tmp.log/data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.tmp.log.3\r\n\r\n\r\ntail of the log file (300 lines):\r\n--------------------------------------------------------------------------------\r\npmonsd [evt]  177      0      0@0         0      0@0         0      0@0    athregseq\r\npmonsd [evt]  178      0      0@0         0      0@0         0      0@0    streamdesd_phojet_fh\r\npmonsd [evt]  177      0                  0                  0             [total_for_2_comps]\r\npmonsd [evt]  178    901                224              31425             [total_for_6_comps]\r\npmonsd ======================================= step fin =======================================\r\npmonsd          n    cpu    vmem  malloc component\r\npmonsd [fin]    1    140       0  -87205 detectorstore\r\npmonsd [fin]    1     50       0  -32319 storegatesvc\r\npmonsd [fin]    1     30       0     126 chronostatsvc\r\npmonsd [fin]    1     30       0  -26322 [collapsed_119_comps]\r\npmonsd [fin]    1    250       0 -145723 [total_for_122_comps]\r\npmonsd ======================================= step dso =======================================\r\npmonsd          n    cpu    vmem  malloc component\r\npmonsd [dso]    1    200   33120       6 liblcg_oracleaccess.so\r\npmonsd [dso]    1   1710   20748     290 libtrigeventathenapoolpoolcnv.so\r\npmonsd [dso]    1    160   10124     225 liblumiblockcomps.so\r\npmonsd [dso]    1    350    9892     165 /afs/cern.ch/atlas/software/releases/17.0.3/atlasevent/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libtrigdecisioneventtpcnv1dict.so\r\npmonsd [dso]    1    550    7036     170 libmuoneventathenapoolpoolcnv.so\r\npmonsd [dso]    1    380    5696     937 /afs/cern.ch/atlas/software/releases/17.0.3/atlasevent/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libtrigmuoneventdict.so\r\npmonsd [dso]    1    130    3748     446 /afs/cern.ch/atlas/software/releases/17.0.3/atlasconditions/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libtrt_conditionsdatadict.so\r\npmonsd [dso]    1     90    3384     527 /afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libatlasstladdreflexdict.so\r\npmonsd [dso]    1    140    2788     696 /afs/cern.ch/atlas/software/releases/17.0.3/atlasevent/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libcaloeventdict.so\r\npmonsd [dso]    1    140    2664     489 /afs/cern.ch/atlas/software/releases/17.0.3/atlasevent/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libtileeventdict.so\r\npmonsd [dso]    1   7340  162852    7277 [collapsed_150_comps]\r\npmonsd [dso]    1  11190  262052   11228 [total_for_160_comps]\r\npmonsd ================================== step preloadproxy ===================================\r\npmonsd                   n    cpu    vmem  malloc component\r\npmonsd [preloadproxy]    1    610    2048    2819 condattrlistcollection[/tile/ofl01/calib/ces]\r\npmonsd [preloadproxy]    1    270    2048    1736 condattrlistcollection[/tile/onl01/calib/ems]\r\npmonsd [preloadproxy]    1     70    1024     901 condattrlistcollection[/muonalign/mdt/barrel]\r\npmonsd [preloadproxy]    1     10    1024     508 condattrlistcollection[/trigger/hlt/menu]\r\npmonsd [preloadproxy]    1     50    1024      11 alignabletransformcontainer[/indet/align]\r\npmonsd [preloadproxy]    2     10       0     231 condattrlistcollection[/trigger/hlt/prescales]\r\npmonsd [preloadproxy]    3      0       0      38 condattrlistcollection[/trigger/lvl1/prescales]\r\npmonsd [preloadproxy]    1     90       0      12 detcondkeytrans[/lar/align]\r\npmonsd [preloadproxy]    3      0       0       1 condattrlistcollection[/ext/dcs/magnets/sensordata]\r\npmonsd [preloadproxy]    2      0       0       1 athenaattributelist[/trigger/hlt/prescalekey]\r\npmonsd [preloadproxy]    3      0       0       0 athenaattributelist[/trigger/lvl1/lvl1configkey]\r\npmonsd [preloadproxy]    1     40       0     783 [collapsed_70_comps]\r\npmonsd [preloadproxy]    1   1140    7168    6770 [total_for_76_comps]\r\npmonsd [preloadproxy]    2     10       0     232 [total_for_2_comps]\r\npmonsd [preloadproxy]    3      0       0      40 [total_for_3_comps]\r\npmonsd ===================================== special info =====================================\r\npmonsd          n      cpu     wall     vmem   malloc component\r\npmonsd [---]  179      966      998        -        - evtloop_time\r\npmonsd [---]    1    25584    46066        -        - overhead_time\r\npmonsd [---]    1        0        0   175476    71780 snapshot_pre_ini\r\npmonsd [---]    1    17250    35201   536208   217474 snapshot_post_ini\r\npmonsd [---]    1    26270    46597   782576   338569 snapshot_post_1stevt\r\npmonsd [---]    1   198130   224251   823372   322359 snapshot_post_lastevt\r\npmonsd [---]    1   198410   224718   823372   176206 snapshot_post_fin\r\npmonsd [---]   40        -        -      633      488 leakperevt_evt11to50\r\npmonsd [---]  129        -        -       51       52 leakperevt_evt51plus\r\npmonsd [---] vmem_peak=823376 vmem_mean=815307 rss_mean=643607\r\npmonsd [---] jobcfg_walltime=50440 jobstart=2011-09-13t07:26:04+0000\r\npmonsd [---] cpu_bmips=4534 cpu_res=10 release=17.0.3.4/i686-slc5-gcc43-opt\r\npmonsd [---] cpu_model=intel(r)_xeon(r)_cpu_l5640_@_2.27ghz/12288_kb\r\npmonsd [---] malloc=libtcmalloc_minimal.so/ok pycintex_vmemfix=0\r\npmonsd [---] pmonsd_cost_onceperjob=40 pmonsd_cost_perevt=0\r\npmonsd =========================== semi-detailed perfmon info / end ===========================\r\npy:perfmonsvc        info finalizing [perfmonsvc]...\r\npy:perfmonsvc        info events processed: 179\r\npy:perfmonsvc        info job had [4] algorithms\r\npy:perfmonsvc        info statistics for \'ini\': (nbr entries = 1)\r\npy:perfmonsvc        info &lt;cpu&gt;:    (  16286.000 +/-      0.000 ) ms\r\npy:perfmonsvc        info &lt;real&gt;:   (  34142.000 +/-      0.000 ) ms\r\npy:perfmonsvc        info &lt;vmem&gt;:   (    523.641 +/-      0.000 ) mb\r\npy:perfmonsvc        info &lt;malloc&gt;: (      0.000 +/-      0.000 ) mb\r\npy:perfmonsvc        info &lt;nalloc&gt;: (      0.000 +/-      0.000 ) calls\r\npy:perfmonsvc        info &lt;rt&gt;: (  16226.386 +/-      0.000 ) ns\r\npy:perfmonsvc        info statistics for \'evt\': (nbr entries = 178)\r\npy:perfmonsvc        info &lt;cpu&gt;:    (    901.337 +/-     18.630 ) ms\r\npy:perfmonsvc        info &lt;real&gt;:   (    928.584 +/-     22.408 ) ms\r\npy:perfmonsvc        info &lt;vmem&gt;:   (    796.417 +/-      0.807 ) mb\r\npy:perfmonsvc        info &lt;malloc&gt;: (      0.000 +/-      0.000 ) mb\r\npy:perfmonsvc        info &lt;nalloc&gt;: (      0.000 +/-      0.000 ) calls\r\npy:perfmonsvc        info &lt;rt&gt;: (    899.558 +/-     18.595 ) ns\r\npy:perfmonsvc        info statistics for \'fin\': (nbr entries = 1)\r\npy:perfmonsvc        info &lt;cpu&gt;:    (     52.000 +/-      0.000 ) ms\r\npy:perfmonsvc        info &lt;real&gt;:   (    -84.000 +/-      0.000 ) ms\r\npy:perfmonsvc        info &lt;vmem&gt;:   (    804.074 +/-      0.000 ) mb\r\npy:perfmonsvc        info &lt;malloc&gt;: (      0.000 +/-      0.000 ) mb\r\npy:perfmonsvc        info &lt;nalloc&gt;: (      0.000 +/-      0.000 ) calls\r\npy:perfmonsvc        info &lt;rt&gt;: (     51.484 +/-      0.000 ) ns\r\npy:perfmonsvc        info memory infos from [/proc/self/status]:\r\npy:perfmonsvc        info vmdata:	518264 kb\r\npy:perfmonsvc        info vmexe:	1196 kb\r\npy:perfmonsvc        info vmhwm:	654252 kb\r\npy:perfmonsvc        info vmlck:	0 kb\r\npy:perfmonsvc        info vmlib:	289368 kb\r\npy:perfmonsvc        info vmpte:	1628 kb\r\npy:perfmonsvc        info vmpeak:	823376 kb\r\npy:perfmonsvc        info vmrss:	654224 kb\r\npy:perfmonsvc        info vmsize:	823376 kb\r\npy:perfmonsvc        info vmstk:	392 kb\r\npy:perfmonsvc        info vmem-leak estimation: (nevts=179)\r\npy:perfmonsvc        info   first-evt vmem:   764.234 mb\r\npy:perfmonsvc        info   10th -evt vmem:   774.309 mb\r\npy:perfmonsvc        info   last -evt vmem:   803.141 mb\r\npy:perfmonsvc        info   evt  2-20 fitted vmem-slope ( 19 points):   750.7 kb/evt\r\npy:perfmonsvc        info   evt 21-50 fitted vmem-slope ( 30 points):   581.1 kb/evt\r\npy:perfmonsvc        info   evt 51+   fitted vmem-slope (129 points):    47.5 kb/evt\r\npy:perfmonsvc        info si2k: 2900.0 =&gt; &lt;cpu time&gt;:   2613.878 ksi2k\r\npy:perfmonsvc        info writing out collected data... [ntuple_esdto.pmon.gz]\r\npy:perfmonsvc        info  --&gt; [ntuple_esdto.pmon.dat] =&gt;    5.000 kb\r\npy:perfmonsvc        info  --&gt; [ntuple_esdto.pmon.stream] =&gt;  164.446 kb\r\npy:perfmonsvc        info  --&gt; [ntuple_esdto.pmon.pmonsd.txt] =&gt;   47.189 kb\r\npy:perfmonsvc        info writing out collected data... [ntuple_esdto.pmon.gz] =&gt;   32.935 kb\r\napplicationmgr                                       info application manager finalized successfully\r\napplicationmgr                                       info application manager terminated successfully\r\nsummarizing all message counts\r\n=====================================================\r\n message source              |   level |    count\r\n-----------------------------+---------+-------------\r\n algerrorauditor             |   error |        3\r\n athenaeventloopmgr          | warning |        1\r\n athenaeventloopmgr          |   error |        1\r\n athenapoolcnvsvc            |   error |        2\r\n athenapoolconverter         |   error |       22\r\n histogrampersistencysvc     | warning |        1\r\n iovdbsvc                    | warning |       10\r\n mugm:muonstation            | warning |       18\r\n poolsvc                     | warning |       23\r\n poolsvc                     |   error |        4\r\n poolsvc                     |   fatal |        1\r\n servicemanager              |   error |        1\r\n streamdesd_phojet           |   error |        2\r\n streamdesd_phojet           |   fatal |        1\r\n streamdesd_phojet.streamdesd_phojettool|   error |        1\r\n streamdesd_phojet_fh        |   error |        1\r\n streamdesd_phojet_fh        |   fatal |        2\r\n streamdesd_phojet_fh.streamdesd_phojet_fhtool|   error |       23\r\n thistsvc                    | warning |        1\r\n toolsvc.goodrunslistselectortool| warning |        1\r\n toolsvc.lartoweremhec       | warning |        1\r\n=====================================================\r\npy:athena            info leaving with code 65: &quot;failure in an algorithm execute&quot;\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.4/installarea/share/atlas_error_categories.db\r\n/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.4/installarea/share/bin/athena.py -b runargs.mergepool.py patjobtransforms/skeleton.mergepool.py mergepool_messagesvc_joboptions.py - exit code 65\r\ncoralapplication info delete the cool coralapplication...\r\ncoralapplication info delete the cool database service\r\nraldatabasesvc info delete the raldatabasesvc...\r\nraldatabasesvc info purge the connection pool\r\nraldatabasesvc info reset the ics pointer\r\nraldatabasesvc info delete the raldatabasesvc... done\r\ncoralapplication info delete the coral connection service\r\ncoralapplication info delete the cool coralapplication... done\r\ncoral/services/connectionservice info deleting the connectionpool\r\npy:mergepool         info mergepool has completed running of athena with exit code 65.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.4/installarea/share/atlas_error_categories.db\r\ninfo: read 52 error patterns from file /afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.4/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.4/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [mergepool log] wall-time: tue sep 13 09:30:38 2011 cest, vmem: 101.39453125 mb, rss: 68.23046875 mb\r\npy:mergepool         info scanning athena logfile mergepool.log assuming atlas release all,17.0.3.4 ...\r\npy:mergepool         info athena initialise()...\r\npy:mergepool         info athena execute()...\r\npy:mergepool         info athena finalise()...\r\npy:mergepool         info done scanning 6825 lines of file mergepool.log. summary:\r\npy:mergepool         info    ignored : 13\r\npy:mergepool         info    warnings: 56\r\npy:mergepool         info    errors  : 57\r\npy:mergepool         info    fatals  : 4\r\nathenalogchecker stop [mergepool log] wall-time: tue sep 13 09:30:39 2011 cest, vmem: 101.39453125 mb, rss: 68.23046875 mb\r\nathenalogchecker report [mergepool log] wall-time: 00 hrs 00 mins 00 secs, cpu-time: 00 hrs 00 mins 00 secs, vmem: 0.0 mb, rss: 0.0 mb\r\npy:mergepool         info jobtransform completed for mergepool with error code 69999 (exit code 99)\r\npy:mergepool         info transform timer stop [mergepool] wall-time: tue sep 13 09:30:39 2011 cest, vmem: 101.39453125 mb, rss: 68.25 mb\r\npy:mergepool         info transform timer report [mergepool] wall-time: 00 hrs 05 mins 41 secs, cpu-time: 00 hrs 04 mins 49 secs, vmem: 0.421875 mb, rss: 0.5703125 mb\r\n===== job report produced by mergepool version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.4/installarea/share/bin/merging_trf.py --argdict=/afs/cern.ch/atlas/tzero/prod1/log2/13439/13439038_data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.3//data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.argdict.gpickle\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (61 times)\r\nfatals=4\r\nerrors=10\r\nwarnings=5\r\nwriting jobinfo_mergepool.xml file...\r\nmergeok is  false\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\npy:merging_trf       info jobtransform completed for merging_trf with error code 69999 (exit code 99)\r\npy:mergepool         info transform timer stop [merging_trf] wall-time: tue sep 13 09:30:39 2011 cest, vmem: 101.5625 mb, rss: 68.46484375 mb\r\npy:mergepool         info transform timer report [merging_trf] wall-time: 00 hrs 05 mins 41 secs, cpu-time: 00 hrs 04 mins 49 secs, vmem: 0.58984375 mb, rss: 0.80859375 mb\r\n===== job report produced by mergingtrf =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.4/installarea/share/bin/merging_trf.py --argdict=/afs/cern.ch/atlas/tzero/prod1/log2/13439/13439038_data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.3//data11_7tev.00188949.physics_egamma.recon.desd_phojet.f403_m976.dpdmerge.task._lb0065-lb0168._0001.job.argdict.gpickle\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (61 times)\r\nfatals=4\r\nerrors=10\r\nwarnings=5\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=incorrect object/count: rec::trackparticlecontainer_combined_trackparticles, 178 should be: 179\r\n  severity=error\r\n  who=streamdesd_phojet\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=incorrect object/count: trackcollection_combined_tracks, 178 should be: 179\r\n  severity=error\r\n  who=streamdesd_phojet\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo (count=2)\r\n  acronym=trf_unknown (69999)\r\n  message=pool commitandhold failed 0xdde5e40\r\n  severity=error\r\n  who=poolsvc\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo (count=2)\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed to commitandhold outputstream.\r\n  severity=error\r\n  who=athenapoolcnvsvc\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=unable to commit output data11_7tev.00188949.physics_egamma.merge.desd_phojet.f403_m976._lb0065-lb0168._0001\r\n  severity=error\r\n  who=streamdesd_phojet.streamdesd_phojettool\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed.\r\n  severity=fatal\r\n  who=streamdesd_phojet\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=terminating event processing loop due to errors\r\n  severity=error\r\n  who=athenaeventloopmgr\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=athenapoolconverter createrep failed to get token, key = streamdesd_phojet\r\n  severity=error\r\n  who=athenapoolconverter\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=could not create rep for dataobject (clid/key):167728019 streamdesd_phojet\r\n  severity=error\r\n  who=streamdesd_phojet_fh.streamdesd_phojet_fhtool\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=athenapoolconverter createrep failed to get token, key = incompletelumiblocks\r\n  severity=error\r\n  who=athenapoolconverter\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=could not create rep for dataobject (clid/key):210948284 incompletelumiblocks\r\n  severity=error\r\n  who=streamdesd_phojet_fh.streamdesd_phojet_fhtool\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=streamobjects failed.\r\n  severity=fatal\r\n  who=streamdesd_phojet_fh\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed.\r\n  severity=fatal\r\n  who=streamdesd_phojet_fh\r\n  producer=mergepool\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot disconnect stream: 1\r\n  severity=fatal\r\n  who=poolsvc\r\n  producer=mergepool\r\n  stage=execute\r\nathacronym=failure in an algorithm execute\r\nathcode=65\r\nrundiruseddisk=1260370654\r\nrundiravailabledisk=1162548207616\r\nworkdir=/pool/lsf/atlast0/169159854\r\nathcode=65\r\nrundiravailabledisk=1162548805632\r\ndbrelease=current\r\nathacronym=failure in an algorithm execute\r\n\r\n-verbatim-','721','4045','2011-09-13 12:58:19','9','fixed','closed','muonidentification/muid',';trackcollection_combined_tracks;toroid;field;cause;problem;desd;file;merge;lb0120;lb0137;data11_7tev;physics_minbias.recon.desd_mbias.f403_m976;dpdmerge;lb0065;lb0167;data11_7tev;physics_egamma.recon.desd_sglel.f403_m976;dpdmerge;lb0065;lb0168;data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge;problem;fast;repro;merge;task;streamesd;error;incorrect;object;count;i\'ll;forward;atla;reconstruction;susumu;verbatim;input;file;stage_svcclass;t0merge;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403;data11_7tev;physics_egamma.recon.desd_phojet.f403;tier;transformation;command;source;atla;tzero;prod1;inst;project;data11;setup;setuptrf.sh;atla;tzero;prod1;inst;project;data11;patch;atlasproduction;17.0.3.4;atlasproduction;17.0.3.4;atla;tzero;prod1;inst;project;data11;setup;specialsetup_tier0;python;merging_trf.py;argdict;atla;tzero;prod1;log2;13439038_data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;job.argdict.gpickle;data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;job.tmp.log;configuration;detail;http;ami.in2p3;opencm;opencm;referencetable;configurationtag;interpret;logfile;location;stage_svcclass;t0merge;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task.tmp.log;data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;job.tmp.log;tail;file;line;pmonsd;athregseq;pmonsd;streamdesd_phojet_fh;pmonsd;total_for_2_comps;pmonsd;total_for_6_comps;pmonsd;step;pmonsd;vmem;malloc;component;pmonsd;detectorstore;pmonsd;storegatesvc;pmonsd;chronostatsvc;pmonsd;collapsed_119_comps;pmonsd;total_for_122_comps;pmonsd;step;pmonsd;vmem;malloc;component;pmonsd;liblcg_oracleaccess.so;pmonsd;libtrigeventathenapoolpoolcnv.so;pmonsd;liblumiblockcomps.so;pmonsd;atla;software;release;17.0.3;atlasevent;17.0.3;installarea;slc5;gcc43;libtrigdecisioneventtpcnv1dict.so;pmonsd;libmuoneventathenapoolpoolcnv.so;pmonsd;atla;software;release;17.0.3;atlasevent;17.0.3;installarea;slc5;gcc43;libtrigmuoneventdict.so;pmonsd;atla;software;release;17.0.3;atlascondition;17.0.3;installarea;slc5;gcc43;libtrt_conditionsdatadict.so;pmonsd;atla;software;release;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libatlasstladdreflexdict.so;pmonsd;atla;software;release;17.0.3;atlasevent;17.0.3;installarea;slc5;gcc43;libcaloeventdict.so;pmonsd;atla;software;release;17.0.3;atlasevent;17.0.3;installarea;slc5;gcc43;libtileeventdict.so;pmonsd;collapsed_150_comps;pmonsd;total_for_160_comps;pmonsd;step;preloadproxy;pmonsd;vmem;malloc;component;pmonsd;preloadproxy;condattrlistcollection;tile;ofl01;calib;pmonsd;preloadproxy;condattrlistcollection;tile;onl01;calib;pmonsd;preloadproxy;condattrlistcollection;muonalign;barrel;pmonsd;preloadproxy;condattrlistcollection;trigger;menu;pmonsd;preloadproxy;alignabletransformcontainer;indet;align;pmonsd;preloadproxy;condattrlistcollection;trigger;prescale;pmonsd;preloadproxy;condattrlistcollection;trigger;lvl1;prescale;pmonsd;preloadproxy;detcondkeytran;align;pmonsd;preloadproxy;condattrlistcollection;magnet;sensorda;pmonsd;preloadproxy;athenaattributelist;trigger;prescalekey;pmonsd;preloadproxy;athenaattributelist;trigger;lvl1;lvl1configkey;pmonsd;preloadproxy;collapsed_70_comps;pmonsd;preloadproxy;total_for_76_comps;pmonsd;preloadproxy;total_for_2_comps;pmonsd;preloadproxy;total_for_3_comps;pmonsd;special;info;pmonsd;wall;vmem;malloc;component;pmonsd;evtloop_time;pmonsd;overhead_time;pmonsd;snapshot_pre_ini;pmonsd;snapshot_post_ini;pmonsd;snapshot_post_1stevt;pmonsd;snapshot_post_lastevt;pmonsd;snapshot_post_fin;pmonsd;leakperevt_evt11to50;pmonsd;leakperevt_evt51plus;pmonsd;vmem_peak;vmem_mean;rss_mean;pmonsd;jobcfg_walltime;jobstart;13t07;pmonsd;cpu_bmips;cpu_res;release;17.0.3.4;slc5;gcc43;pmonsd;cpu_model;intel;12288_kb;pmonsd;malloc;libtcmalloc_minimal.so;pycintex_vmemfix;pmonsd;pmonsd_cost_onceperjob;pmonsd_cost_perevt;pmonsd;semus;detailed;perfmon;info;py:perfmonsvc;info;finalize;perfmonsvc;py:perfmonsvc;info;event;process;py:perfmonsvc;info;algorithm;py:perfmonsvc;info;statistics;inus;entry;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;call;py:perfmonsvc;info;py:perfmonsvc;info;statistics;entry;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;call;py:perfmonsvc;info;py:perfmonsvc;info;statistics;entry;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;py:perfmonsvc;info;call;py:perfmonsvc;info;py:perfmonsvc;info;memory;info;proc;self;status;py:perfmonsvc;info;vmda;py:perfmonsvc;info;vmexe;py:perfmonsvc;info;vmhwm;py:perfmonsvc;info;vmlck;py:perfmonsvc;info;vmlib;py:perfmonsvc;info;vmpte;py:perfmonsvc;info;vmpeak;py:perfmonsvc;info;vmrss;py:perfmonsvc;info;vmsize;py:perfmonsvc;info;vmstk;py:perfmonsvc;info;vmem;leak;estimation;nevt;py:perfmonsvc;info;vmem;py:perfmonsvc;info;vmem;py:perfmonsvc;info;vmem;py:perfmonsvc;info;vmem;slope;point;py:perfmonsvc;info;vmem;slope;point;py:perfmonsvc;info;vmem;slope;point;py:perfmonsvc;info;si2k;ksi2k;py:perfmonsvc;info;write;collect;datum;ntuple_esdto.pmon.gz;py:perfmonsvc;info;ntuple_esdto.pmon.dat;py:perfmonsvc;info;ntuple_esdto.pmon.stream;py:perfmonsvc;info;ntuple_esdto.pmon.pmonsd.txt;py:perfmonsvc;info;write;collect;datum;ntuple_esdto.pmon.gz;applicationmgr;info;application;manager;finalize;successfully;applicationmgr;info;application;manager;terminate;successfully;summarize;message;count;message;source;level;count;algerrorauditor;error;athenaeventloopmgr;warn;athenaeventloopmgr;error;athenapoolcnvsvc;error;athenapoolconverter;error;histogrampersistencysvc;warn;iovdbsvc;warn;mugm:muonstation;warn;poolsvc;warn;poolsvc;error;poolsvc;fatal;servicemanager;error;streamdesd_phojet;error;streamdesd_phojet;fatal;streamdesd_phojet.streamdesd_phojettool;error;streamdesd_phojet_fh;error;streamdesd_phojet_fh;fatal;streamdesd_phojet_fh.streamdesd_phojet_fhtool;error;thistsvc;warn;toolsvc.goodrunslistselectortool;warn;toolsvc.lartoweremhec;warn;py:athena;info;leave;code;failure;algorithm;execute;info;read;error;category;file;atla;software;release;17.0.3;atlasproduction;17.0.3.4;installarea;share;atlas_error_categories.db;atla;software;release;17.0.3;atlasproduction;17.0.3.4;installarea;share;athena.py;runargs.mergepool.py;patjobtransform;skeleton.mergepool.py;mergepool_messagesvc_joboptions.py;exit;code;coralapplication;info;delete;cool;coralapplication;coralapplication;info;delete;cool;database;service;raldatabasesvc;info;delete;raldatabasesvc;raldatabasesvc;info;purge;connection;pool;raldatabasesvc;info;reset;pointer;raldatabasesvc;info;delete;raldatabasesvc;coralapplication;info;delete;coral;connection;service;coralapplication;info;delete;cool;coralapplication;coral;service;connectionservice;info;delete;connectionpool;py:mergepool;info;mergepool;complete;athena;exit;code;info;read;error;category;file;atla;software;release;17.0.3;atlasproduction;17.0.3.4;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;release;17.0.3;atlasproduction;17.0.3.4;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;release;17.0.3;atlasproduction;17.0.3.4;installarea;share;atlas_error_ignore.db;athenalogchecker;start;mergepool;wall;time;cest;vmem;py:mergepool;info;scanning;athena;logfile;mergepool.log;assume;atla;release;17.0.3.4;py:mergepool;info;athena;initialise;py:mergepool;info;athena;execute;py:mergepool;info;athena;finalise;py:mergepool;info;scanning;line;file;mergepool.log;summary;py:mergepool;info;ignore;py:mergepool;info;warning;py:mergepool;info;error;py:mergepool;info;fatal;athenalogchecker;stop;mergepool;wall;time;cest;vmem;athenalogchecker;report;mergepool;wall;time;time;vmem;py:mergepool;info;jobtransform;complete;mergepool;error;code;exit;code;py:mergepool;info;transform;timer;stop;mergepool;wall;time;cest;vmem;py:mergepool;info;transform;timer;report;mergepool;wall;time;time;vmem;report;produce;mergepool;version;unknown;command;atla;software;release;17.0.3;atlasproduction;17.0.3.4;installarea;share;merging_trf.py;argdict;atla;tzero;prod1;log2;13439038_data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;job.argdict.gpickle;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;write;jobinfo_mergepool.xml;file;mergeok;false;skip;step;skip;step;py:merging_trf;info;jobtransform;complete;merging_trf;error;code;exit;code;py:mergepool;info;transform;timer;stop;merging_trf;wall;time;cest;vmem;py:mergepool;info;transform;timer;report;merging_trf;wall;time;time;vmem;report;produce;mergingtrf;command;atla;software;release;17.0.3;atlasproduction;17.0.3.4;installarea;share;merging_trf.py;argdict;atla;tzero;prod1;log2;13439038_data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;data11_7tev;physics_egamma.recon.desd_phojet.f403_m976;dpdmerge.task;lb0168;job.argdict.gpickle;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;incorrect;object;count;trackparticlecontainer_combined_trackparticles;severity;error;streamdesd_phojet;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;incorrect;object;count;trackcollection_combined_tracks;severity;error;streamdesd_phojet;producer;mergepool;stage;execute;errorinfo;count;acronym;trf_unknown;message;pool;commitandhold;fail;severity;error;poolsvc;producer;mergepool;stage;execute;errorinfo;count;acronym;trf_unknown;message;commitoutput;fail;commitandhold;outputstream;severity;error;athenapoolcnvsvc;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;unable;commit;output;data11_7tev;physics_egamma.merge.desd_phojet.f403_m976;lb0168;severity;error;streamdesd_phojet.streamdesd_phojettool;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;commitoutput;fail;severity;fatal;streamdesd_phojet;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;terminate;event;processing;loop;error;severity;error;athenaeventloopmgr;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;athenapoolconverter;createrep;fail;token;streamdesd_phojet;severity;error;athenapoolconverter;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;create;dataobject;clid;streamdesd_phojet;severity;error;streamdesd_phojet_fh.streamdesd_phojet_fhtool;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;athenapoolconverter;createrep;fail;token;incompletelumiblock;severity;error;athenapoolconverter;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;create;dataobject;clid;incompletelumiblock;severity;error;streamdesd_phojet_fh.streamdesd_phojet_fhtool;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;streamobject;fail;severity;fatal;streamdesd_phojet_fh;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;commitoutput;fail;severity;fatal;streamdesd_phojet_fh;producer;mergepool;stage;execute;errorinfo;acronym;trf_unknown;message;disconnect;stream;severity;fatal;poolsvc;producer;mergepool;stage;execute;athacronym;failure;algorithm;execute;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;atlast0;athcode;rundiravailabledisk;dbrelease;current;athacronym;failure;algorithm;execute;verbatim;desd;merge;fail;trackcollection_combined_tracks;run188949;muonidentification;muid'),(86683,'atlas','fpe in btagging in verbose mode','hi p.a.,\r\n\r\nwhen running a job in verbose, one sees this &quot;crash&quot;:\r\nantikt6towerjets.applyem:offset_origin_etajes_s...verbose checking if pt and fraction are in the range of the correction\r\nantikt6towerjets                                    fatal  standard std::exception is caught\r\nantikt6towerjets                                    error vector::_m_range_check\r\nantikt6towerjets                                  verbose servicelocatorhelper::service: found service chronostatsvc\r\n\r\nhttps://atlasrtt.cern.ch/get/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/atlasproduction/recexrecotest/mc11_outputlevel_verbosereco#\r\n\r\nhow to reproduce:\r\nsetup devval,rel_1\r\nathena -c \'jp.athenacommonflags.evtmax=50;rec.dowriteesd=false;rec.outputlevel=verbose;rec.dofloatingpointexception=true;jp.athenacommonflags.filesinput=[&quot;/afs/cern.ch/atlas/project/rig/referencefiles/mc/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e835_s1256_s1258_d566_t\r\nid464353_00/rdo.464353._000001.pool.root.1&quot;]\'\r\n\r\ncheers, rolf\r\n','1693','1658','2011-09-13 14:10:45','5','fixed','closed','jet',';verbose;crash;antikt6towerjets.applyem:offset_origin_etajes_s;verbose;check;fraction;range;correction;antikt6towerjet;fatal;standard;exception;catch;antikt6towerjet;error;vector;antikt6towerjet;verbose;servicelocatorhelper;service;service;chronostatsvc;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;atlasproduction;recexrecotest;mc11_outputlevel_verbosereco;reproduce;setup;devval;rel_1;athena;jp.athenacommonflags.evtmax;rec.dowriteesd;false;rec.outputlevel;verbose;rec.dofloatingpointexception;true;jp.athenacommonflags.filesinput;atla;project;referencefile;valid1.105200;t1_mcatnlo_jimmy.digit.rdo.e835_s1256_s1258_d566_t;id464353_00;pool.root;rolf;btagg;verbose;mode'),(86684,'atlas','rtt: 	data_7tev_diffinput 		no such file or directory','hi in rtt: 	data_7tev_diffinput 		no such file or directory\r\n\r\ncheers\r\ndaniel\r\n\r\n\r\n\r\n/afs/cern.ch/atlas/project/rtt/prod/results/rtt/rel_2/16.6.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_diffinput/runscript.sh: line 79: /afs/cern.ch/atlas/maxidisk/d74/rtt_data/data09_900gev.00142193.physics_minbias.merge.raw.1event._lb0098._0001.1,/afs/cern.ch/atlas/maxidisk/d74/rtt_data/data09_900gev.00142194.physics_minbias.merge.raw.1event._lb0016._0001.1: no such file or directory\r\n\r\nsee log \r\nhttps://atlasrtt.cern.ch/rtt/rel_2/16.6.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_diffinput/data_7tev_diffinput_log','100','5211','2011-09-13 14:14:53','5','duplicate','closed','reconstruction',';data_7tev_diffinput;file;directory;daniel;atla;project;prod;result;rel_2;build;slc5;gcc43;offline;recjobtransformtest;data_7tev_diffinput;runscript.sh;line;atla;maxidisk;rtt_data;data09_900gev;physics_minbias.merge.raw;1event;atla;maxidisk;rtt_data;data09_900gev;physics_minbias.merge.raw;1event;file;directory;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;recjobtransformtest;data_7tev_diffinput;data_7tev_diffinput_log;data_7tev_diffinput;file;directory;reconstruction'),(86685,'atlas','rtt: data_7tev_nocalo 		athena received signal 11. exit code reset to athena exit code 8.','hi in 	data_7tev_nocalo\r\n		athena received signal 11. exit code reset to athena exit code 8.\r\n\r\n\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code -11.\r\npy:rawtoesd         error athena received signal 11. exit code reset to athena exit code 8.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/share/atlas_error_categories.db\r\ninfo: read 52 error patterns from file /afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: tue sep 13 08:44:08 2011 cest, vmem: 109.29296875 mb, rss: 63.82421875 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_2 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info done scanning 13477 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 0\r\npy:rawtoesd          info    warnings: 5630\r\npy:rawtoesd          info    errors  : 0\r\npy:rawtoesd          info    fatals  : 2\r\nathenalogchecker stop [rawtoesd log] wall-time: tue sep 13 08:44:15 2011 cest, vmem: 109.42578125 mb, rss: 63.84375 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 07 secs, cpu-time: 00 hrs 00 mins 07 secs, vmem: 0.1328125 mb, rss: 0.01953125 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 60010 (exit code 6)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: tue sep 13 08:44:15 2011 cest, vmem: 109.42578125 mb, rss: 63.8671875 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 19 mins 46 secs, cpu-time: 00 hrs 19 mins 22 secs, vmem: 0.515625 mb, rss: 0.46875 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/share/bin/reco_trf.py inputbsfile=/afs/cern.ch/atlas/project/rig/referencefiles/datastreams_high_mu/high_mu-data11_7tev.00179725.physics_jettauetmiss.merge.raw._lb0021.data maxevents=300 autoconfiguration=everything preexec=rec.docalo=false outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root\r\nexitcode=6\r\nerrorcode=60010 (trf_segfault)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault (2 times)\r\nfatals=2\r\nerrors=0\r\nwarnings=10\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 60010 (exit code 6)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: tue sep 13 08:44:16 2011 cest, vmem: 109.5859375 mb, rss: 64.08203125 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 00 hrs 19 mins 46 secs, cpu-time: 00 hrs 19 mins 22 secs, vmem: 0.67578125 mb, rss: 0.765625 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/share/bin/reco_trf.py inputbsfile=/afs/cern.ch/atlas/project/rig/referencefiles/datastreams_high_mu/high_mu-data11_7tev.00179725.physics_jettauetmiss.merge.raw._lb0021.data maxevents=300 autoconfiguration=everything preexec=rec.docalo=false outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root\r\nexitcode=6\r\nerrorcode=60010 (trf_segfault)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault (2 times)\r\nfatals=2\r\n\r\nsee log \r\nhttps://atlasrtt.cern.ch/rtt/rel_2/16.6.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_nocalo/data_7tev_nocalo_log\r\n\r\ncheers\r\ndaniel\r\n','1658','5211','2011-09-13 14:18:34','5','none','closed','reconstruction',';data_7tev_nocalo;athena;receive;signal;exit;code;reset;athena;exit;code;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;py:rawtoesd;error;athena;receive;signal;exit;code;reset;athena;exit;code;info;read;error;category;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_2;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;reco_trf.py;inputbsfile;atla;project;referencefile;datastreams_high_mu;high_mu;data11_7tev;physics_jettauetmiss.merge.raw;datum;maxevent;autoconfiguration;preexec;rec.docalo;false;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;exitcode;errorcode;trf_segfault;errorcategory;trf_segfault;segmentation;fault;time;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;reco_trf.py;inputbsfile;atla;project;referencefile;datastreams_high_mu;high_mu;data11_7tev;physics_jettauetmiss.merge.raw;datum;maxevent;autoconfiguration;preexec;rec.docalo;false;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;exitcode;errorcode;trf_segfault;errorcategory;trf_segfault;segmentation;fault;time;fatal;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;recjobtransformtest;data_7tev_nocalo;data_7tev_nocalo_log;daniel;data_7tev_nocalo;athena;receive;signal;exit;code;reset;athena;exit;code;reconstruction'),(86686,'atlas','rtt 	mc10_rdotobs 		valueerror:  none is not the expected type for: jobproperties.global.detdescrversion','hi in 	mc10_rdotobs\r\n		valueerror:  none is not the expected type for: jobproperties.global.detdescrversion\r\n\r\nvalueerror:  none is not the expected type for: jobproperties.global.detdescrversion\r\npy:athena            info leaving with code 8: &quot;an unknown exception occurred&quot;\r\npy:sg-dumper        error ================================================================================\r\npy:sg-dumper        error problem running jobo\r\n\r\n\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/share/bin/reco_trf.py inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/mc10_7tev.105200.t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00/rdo.197091._000001.pool.root.1 outputbsfile=t1_mcatnlo_jimmy.data autoconfiguration=everything maxevents=300\r\nexitcode=41\r\nerrorcode=64140 (trf_outfile_corrup)\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_outfile_corrup (64140) transform output file corrupted\r\n  message=output file t1_mcatnlo_jimmy.data failed validation [8]. file corrupt. argument outputbsfile\r\n  severity=fatal\r\n  producer=rdotobs\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/trf.py&quot;, line 1858, in dopostrunactions\r\n              cmd.postrunaction()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1197, in postrunaction\r\n              self.validatefile()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1184, in validatefile\r\n              self._filetype.validatefile( self, **self.__validationdict )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/trfutil.py&quot;, line 1114, in validatefile\r\n              raise transformvalidationerror( fname, \'failed validation [%s]. file corrupt. argument %s\' % ( sc, argname ) )\r\nathacronym=successful run\r\nathcode=0\r\nrundiruseddisk=646912232\r\nrundiravailabledisk=188181762048\r\nworkdir=/pool/lsf/rtt/169098083\r\nathcode=0\r\nrundiravailabledisk=188181999616\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=646686919\r\natlasrelease=rel_2\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\nsee log \r\nhttps://atlasrtt.cern.ch/rtt/rel_2/16.6.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/mc10_rdotobs/mc10_rdotobs_log\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-13 14:21:49','5','duplicate','closed','reconstruction',';mc10_rdotobs;valueerror;expect;type;jobproperties.global.detdescrversion;valueerror;expect;type;jobproperties.global.detdescrversion;py:athena;info;leave;code;unknown;exception;occur;py:sg;dumper;error;py:sg;dumper;error;problem;jobo;report;produce;recotrf;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;reco_trf.py;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;mc10_7tev;t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00;pool.root;outputbsfile;t1_mcatnlo_jimmy.data;autoconfiguration;maxevent;exitcode;errorcode;trf_outfile_corrup;fatal;error;warning;errorinfo;acronym;trf_outfile_corrup;transform;output;file;corrupt;message;output;file;t1_mcatnlo_jimmy.data;fail;validation;file;corrupt;argument;outputbsfile;severity;fatal;producer;rdotob;backtrace;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;trf.py;line;dopostrunaction;cmd.postrunaction;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;basic_trfarg.py;line;postrunaction;self.validatefile;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;basic_trfarg.py;line;validatefile;self;self;self;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;trfutil.py;line;validatefile;raise;transformvalidationerror;fname;fail;validation;file;corrupt;argument;argname;athacronym;successful;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_2;write;jobinfo_recotrf.xml;file;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;recjobtransformtest;mc10_rdotobs;mc10_rdotobs_log;daniel;mc10_rdotobs;valueerror;expect;type;jobproperties.global.detdescrversion;reconstruction'),(86687,'atlas','rtt  ttbar_pu_mu0_d557_wtrigtruth 			fatal in &lt;troot::initsystem&gt;: home directory not set','hi in 		ttbar_pu_mu0_d557_wtrigtruth\r\n			fatal in &lt;troot::initsystem&gt;: home directory not set\r\n\r\n/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_2/installarea/share/bin/athena.py runargs.esdtoaod.py recjobtransforms/skeleton.esdtoaod.py esdtoaod_messagesvc_joboptions.py --drop-and-reload\r\npy:athena            info including file &quot;athenacommon/preparation.py&quot;\r\npy:athena            info using release [atlasproduction-17.0.3.5] [17.0.x.y-prod/rel_2] -- built on [2011 09/12 21:30]\r\npy:athena            info including file &quot;athenacommon/bootstrap.py&quot;\r\npy:athena            info including file &quot;athenacommon/atlas.unixstandardjob.py&quot;\r\nfatal in &lt;troot::initsystem&gt;: home directory not set\r\n\r\n *** break *** segmentation violation\r\npy:esdtoaod          info esdtoaod has completed running of athena with exit code -11.\r\npy:esdtoaod         error athena received signal 11. exit code reset to athena exit code 8.\r\nathenalogchecker start [esdtoaod log] wall-time: tue sep 13 07:25:12 2011 cest, vmem: 110.90234375 mb, rss: 71.27734375 mb\r\npy:esdtoaod          info scanning athena logfile esdtoaod.log assuming atlas release all,rel_2 ...\r\npy:esdtoaod          info athena initialise()...\r\npy:esdtoaod          info done scanning 222 lines of file esdtoaod.log. summary:\r\npy:esdtoaod          info    ignored : 0\r\npy:esdtoaod          info    warnings: 0\r\npy:esdtoaod          info    errors  : 0\r\npy:esdtoaod          info    fatals  : 1\r\nathenalogchecker stop [esdtoaod log] wall-time: tue sep 13 07:25:12 2011 cest, vmem: 110.90234375 mb, rss: 71.2890625 mb\r\nathenalogchecker report [esdtoaod log] wall-time: 00 hrs 00 mins 00 secs, cpu-time: 00 hrs 00 mins 00 secs, vmem: 0.0 mb, rss: 0.01171875 mb\r\npy:esdtoaod          info jobtransform completed for esdtoaod with error code 60000 (exit code 6)\r\npy:esdtoaod          info transform timer stop [esdtoaod] wall-time: tue sep 13 07:25:12 2011 cest, vmem: 110.90234375 mb, rss: 71.296875 mb\r\npy:esdtoaod          info transform timer report [esdtoaod] wall-time: 00 hrs 00 mins 01 secs, cpu-time: 00 hrs 00 mins 01 secs, vmem: 0.16796875 mb, rss: 0.0625 mb\r\n===== job report produced by esdtoaod version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_2/installarea/share/bin/reco_trf.py ami=f380 autoconfiguration=everything inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00/rdo.364680._000001.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00/rdo.364680._000002.pool.root.1 outputaodfile=myaod.pool.root outputesdfile=myesd.pool.root outputhistfile=myhist.root\r\nexitcode=6\r\nerrorcode=60000 (trf_segvio)\r\nerrorcategory=trf_segvio,	60000,	segmentation violation\r\nfatals=1\r\nerrors=0\r\nwarnings=0\r\nwriting jobinfo_esdtoaod.xml file...\r\naodok is  false\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 60000 (exit code 6)\r\npy:esdtoaod          info transform timer stop [reco_trf] wall-time: tue sep 13 07:25:12 2011 cest, vmem: 111.03125 mb, rss: 71.46875 mb\r\npy:esdtoaod          info transform timer report [reco_trf] wall-time: 00 hrs 33 mins 37 secs, cpu-time: 00 hrs 32 mins 33 secs, vmem: 1.01953125 mb, rss: 1.01953125 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_2/installarea/share/bin/reco_trf.py ami=f380 autoconfiguration=everything inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00/rdo.364680._000001.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00/rdo.364680._000002.pool.root.1 outputaodfile=myaod.pool.root outputesdfile=myesd.pool.root outputhistfile=myhist.root\r\nexitcode=6\r\nerrorcode=60000 (trf_segvio)\r\nerrorcategory=trf_segvio,	60000,	segmentation violation\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nfile myesd.pool.root:\r\n  guid=cec59a05-c5dd-e011-93e5-00304896b94a\r\n  metadata:\r\n    amitag=f380\r\n    beamtype=collisions\r\n    filetype=esd\r\n    autoconfiguration=[\'everything\']\r\n    dataset=\r\n    maxevents=-1\r\n    preexec=[\'indetflags.usebeamconstraint.set_value_and_lock(true);from calotools.calonoiseflags import jobproperties;jobproperties.calonoiseflags.fixedluminosity.set_value_and_lock(1.45);aodflags.trackparticleslimmer=true;\']\r\n    geometryversion=atlas-geo-16-00-00\r\n    events=100\r\n    size=147402602\r\nerrorinfo\r\n  acronym=trf_segvio (60000) segmentation violation\r\n  message=*** break *** segmentation violation\r\n  severity=fatal\r\n  producer=esdtoaod\r\n  stage=initialise\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=176194538\r\nrundiravailabledisk=147053826048\r\nworkdir=/pool/lsf/rtt/169096176\r\nathcode=0\r\nrundiravailabledisk=147054804992\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=175273631\r\natlasrelease=rel_2\r\nworkdir=/pool/lsf/rtt/169096176\r\nathcode=8\r\nrundiravailabledisk=147054108672\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=175931611\r\natlasrelease=rel_2\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\nsee log \r\nhttps://atlasrtt.cern.ch/rtt/rel_2/17.0.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/ttbar_pu_mu0_d557_wtrigtruth/ttbar_pu_mu0_d557_wtrigtruth_log\r\n\r\ncheers\r\ndaniel\r\n','100','5211','2011-09-13 14:29:37','5','duplicate','closed','reconstruction',';ttbar_pu_mu0_d557_wtrigtruth;fatal;home;directory;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;athena.py;runargs.esdtoaod.py;recjobtransform;skeleton.esdtoaod.py;esdtoaod_messagesvc_joboptions.py;drop;reload;py:athena;info;include;file;athenacommon;preparation.py;py:athena;info;release;atlasproduction;17.0.3.5;prod;rel_2;build;py:athena;info;include;file;athenacommon;bootstrap.py;py:athena;info;include;file;athenacommon;atlas.unixstandardjob.py;fatal;home;directory;break;segmentation;violation;py:esdtoaod;info;esdtoaod;complete;athena;exit;code;py:esdtoaod;error;athena;receive;signal;exit;code;reset;athena;exit;code;athenalogchecker;start;esdtoaod;wall;time;cest;vmem;py:esdtoaod;info;scanning;athena;logfile;esdtoaod.log;assume;atla;release;rel_2;py:esdtoaod;info;athena;initialise;py:esdtoaod;info;scanning;line;file;esdtoaod.log;summary;py:esdtoaod;info;ignore;py:esdtoaod;info;warning;py:esdtoaod;info;error;py:esdtoaod;info;fatal;athenalogchecker;stop;esdtoaod;wall;time;cest;vmem;athenalogchecker;report;esdtoaod;wall;time;time;vmem;py:esdtoaod;info;jobtransform;complete;esdtoaod;error;code;exit;code;py:esdtoaod;info;transform;timer;stop;esdtoaod;wall;time;cest;vmem;py:esdtoaod;info;transform;timer;report;esdtoaod;wall;time;time;vmem;report;produce;esdtoaod;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;reco_trf.py;autoconfiguration;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105200;t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105200;t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00;pool.root;outputaodfile;myaod.pool.root;outputesdfile;myesd.pool.root;outputhistfile;myhist.root;exitcode;errorcode;trf_segvio;errorcategory;trf_segvio;segmentation;violation;fatal;error;warning;write;jobinfo_esdtoaod.xml;file;aodok;false;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:esdtoaod;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:esdtoaod;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;reco_trf.py;autoconfiguration;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105200;t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105200;t1_mcatnlo_jimmy.digit.rdo.e603_s932_d557_tid364680_00;pool.root;outputaodfile;myaod.pool.root;outputesdfile;myesd.pool.root;outputhistfile;myhist.root;exitcode;errorcode;trf_segvio;errorcategory;trf_segvio;segmentation;violation;fatal;error;warning;file;myesd.pool.root;guid;metada;amitag;beamtype;collision;filetype;autoconfiguration;dataset;maxevent;preexec;indetflags.usebeamconstraint.set_value_and_lock;true;calotools.calonoiseflags;import;jobproperty;jobproperties.calonoiseflags.fixedluminosity.set_value_and_lock;aodflags.trackparticleslimmer;true;geometryversion;atla;event;size;errorinfo;acronym;trf_segvio;segmentation;violation;message;break;segmentation;violation;severity;fatal;producer;esdtoaod;stage;initialise;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_2;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_2;write;jobinfo_recotrf.xml;file;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;recjobtransformtest;ttbar_pu_mu0_d557_wtrigtruth;ttbar_pu_mu0_d557_wtrigtruth_log;daniel;ttbar_pu_mu0_d557_wtrigtruth;fatal;home;directory;reconstruction'),(86689,'atlas','rtt 		amitag_q121_notrig 			fatal in &lt;troot::initsystem&gt;: home directory not set','hi in 		amitag_q121_notrig\r\n			fatal in &lt;troot::initsystem&gt;: home directory not set\r\n\r\nfatal in &lt;troot::initsystem&gt;: home directory not set\r\n\r\n *** break *** segmentation violation\r\npy:esdtoaod          info esdtoaod has completed running of athena with exit code -11.\r\npy:esdtoaod         error athena received signal 11. exit code reset to athena exit code 8.\r\nathenalogchecker start [esdtoaod log] wall-time: tue sep 13 04:24:41 2011 cest, vmem: 110.87890625 mb, rss: 71.3046875 mb\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_2/installarea/share/bin/reco_trf.py ami=q121 append_preexec=rec.dotrigger=false;rec.donameauditor=true\r\nexitcode=6\r\n\r\n\r\nsee log \r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_2/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/amitag_q121_notrig/amitag_q121_notrig_log\r\n\r\n\r\ncheers\r\ndaniel\r\n','100','5211','2011-09-13 14:33:51','5','duplicate','closed','reconstruction',';amitag_q121_notrig;fatal;home;directory;fatal;home;directory;break;segmentation;violation;py:esdtoaod;info;esdtoaod;complete;athena;exit;code;py:esdtoaod;error;athena;receive;signal;exit;code;reset;athena;exit;code;athenalogchecker;start;esdtoaod;wall;time;cest;vmem;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;reco_trf.py;append_preexec;rec.dotrigger;false;rec.donameauditor;true;exitcode;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;recjobtransformtest;amitag_q121_notrig;amitag_q121_notrig_log;daniel;amitag_q121_notrig;fatal;home;directory;reconstruction'),(86732,'atlas','rtt: amitag_q125 			highptemclusters                                    fatal ../src/clusterfilter.cxx:42','hi in \r\n\r\namitag_q125\r\n			highptemclusters                                    fatal ../src/clusterfilter.cxx:42 (virtual statuscode clusterfilter::execute()): code 0: evtstore()-&gt;retrieve( cluscoll, m_inputname )\r\n\r\n\r\n\r\n===== job report produced by aodtodpd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_3/installarea/share/bin/reco_trf.py ami=q125\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (2 times)\r\nfatals=1\r\nerrors=1\r\nwarnings=7\r\nwriting jobinfo_aodtodpd.xml file...\r\ndpdaodok is  false\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 69999 (exit code 99)\r\npy:aodtodpd          info transform timer stop [reco_trf] wall-time: wed sep 14 08:53:51 2011 cest, vmem: 109.80859375 mb, rss: 71.2578125 mb\r\npy:aodtodpd          info transform timer report [reco_trf] wall-time: 00 hrs 06 mins 00 secs, cpu-time: 00 hrs 05 mins 49 secs, vmem: 0.6875 mb, rss: 0.80078125 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_3/installarea/share/bin/reco_trf.py ami=q125\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (2 times)\r\nfatals=1\r\nerrors=1\r\nwarnings=7\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=../src/clusterfilter.cxx:42 (virtual statuscode clusterfilter::execute()): code 0: evtstore()-&gt;retrieve( cluscoll, m_inputname )\r\n  severity=fatal\r\n  who=highptemclusters\r\n  producer=aodtodpd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=terminating event processing loop due to errors\r\n  severity=error\r\n  who=athenaeventloopmgr\r\n  producer=aodtodpd\r\n  stage=execute\r\nathacronym=failure in an algorithm execute\r\nathcode=65\r\nrundiruseddisk=5323540\r\nrundiravailabledisk=159422447616\r\nworkdir=/pool/lsf/rtt/169342060\r\nathcode=65\r\nrundiravailabledisk=159422676992\r\ndbrelease=\r\nathacronym=failure in an algorithm execute\r\nrundiruseddisk=5111271\r\natlasrelease=rel_3\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\nsee log file\r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_3/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/amitag_q125/amitag_q125_log\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-14 14:55:42','5','fixed','closed','reconstruction',';amitag_q125;highptemcluster;fatal;clusterfilter.cxx;virtual;statuscode;clusterfilter;execute;code;evtstore;retrieve;cluscoll;m_inputname;report;produce;aodtodpd;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_3;installarea;share;reco_trf.py;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;write;jobinfo_aodtodpd.xml;file;dpdaodok;false;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:aodtodpd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:aodtodpd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;atlasproduction;rel_3;installarea;share;reco_trf.py;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;clusterfilter.cxx;virtual;statuscode;clusterfilter;execute;code;evtstore;retrieve;cluscoll;m_inputname;severity;fatal;highptemcluster;producer;aodtodpd;stage;execute;errorinfo;acronym;trf_unknown;message;terminate;event;processing;loop;error;severity;error;athenaeventloopmgr;producer;aodtodpd;stage;execute;athacronym;failure;algorithm;execute;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;failure;algorithm;execute;rundiruseddisk;atlasrelease;rel_3;write;jobinfo_recotrf.xml;file;file;http;atlasrtt.cern.ch;rel_3;build;slc5;gcc43;offline;recjobtransformtest;amitag_q125;amitag_q125_log;daniel;amitag_q125;highptemcluster;fatal;clusterfilter.cxx;reconstruction'),(86766,'atlas','rtt: data_7tev_memleak 			 error athena received signal -9 = sigkill. athena was killed, job will terminate','hi in data_7tev_memleak\r\n			 error athena received signal -9 = sigkill. athena was killed, job will terminate\r\n\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_4/installarea/share/bin/reco_trf.py inputbsfile=/afs/cern.ch/atlas/project/rig/referencefiles/datastreams_high_mu/high_mu-data11_7tev.00179725.physics_jettauetmiss.merge.raw._lb0021.data maxevents=300 autoconfiguration=everything outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root --athenaopts=--leak-check-execute\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=10\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 11000 (exit code 10)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: thu sep 15 12:30:48 2011 cest, vmem: 100.3359375 mb, rss: 67.1953125 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 02 hrs 38 mins 22 secs, cpu-time: 02 hrs 36 mins 43 secs, vmem: 0.73046875 mb, rss: 0.79296875 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_4/installarea/share/bin/reco_trf.py inputbsfile=/afs/cern.ch/atlas/project/rig/referencefiles/datastreams_high_mu/high_mu-data11_7tev.00179725.physics_jettauetmiss.merge.raw._lb0021.data maxevents=300 autoconfiguration=everything outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root --athenaopts=--leak-check-execute\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=10\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=505123681\r\nrundiravailabledisk=723388162048\r\nworkdir=/pool/lsf/rtt/169648802\r\nathcode=8\r\nrundiravailabledisk=723388305408\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=504991680\r\natlasrelease=rel_4\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\nsee log\r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_4/17.0.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_memleak/data_7tev_memleak_log\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-15 14:23:15','5','fixed','closed','reconstruction',';data_7tev_memleak;error;athena;receive;signal;sigkill;athena;kill;terminate;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_4;installarea;share;reco_trf.py;inputbsfile;atla;project;referencefile;datastreams_high_mu;high_mu;data11_7tev;physics_jettauetmiss.merge.raw;datum;maxevent;autoconfiguration;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;athenaopt;leak;check;execute;exitcode;errorcode;ath_failure;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;atlasproduction;rel_4;installarea;share;reco_trf.py;inputbsfile;atla;project;referencefile;datastreams_high_mu;high_mu;data11_7tev;physics_jettauetmiss.merge.raw;datum;maxevent;autoconfiguration;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;athenaopt;leak;check;execute;exitcode;errorcode;ath_failure;fatal;error;warning;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_4;write;jobinfo_recotrf.xml;file;http;atlasrtt.cern.ch;rel_4;build;slc5;gcc43;offline;recjobtransformtest;data_7tev_memleak;data_7tev_memleak_log;daniel;data_7tev_memleak;error;athena;receive;signal;sigkill;athena;kill;terminate;reconstruction'),(86767,'atlas','rtt 		mc10_ignore 			fatal folder /trigger/hlt/groups does not exist 			segmentation violation','hi in 		mc10_ignore\r\n			fatal folder /trigger/hlt/groups does not exist\r\n			segmentation violation\r\n\r\nsee log\r\nhttps://atlasrtt.cern.ch/rtt/rel_4/17.0.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/mc10_ignore/mc10_ignore_log\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-15 14:26:38','5','none','closed','reconstruction',';mc10_ignore;fatal;folder;trigger;group;exist;segmentation;violation;http;atlasrtt.cern.ch;rel_4;build;slc5;gcc43;offline;recjobtransformtest;mc10_ignore;mc10_ignore_log;daniel;mc10_ignore;fatal;folder;trigger;group;exist;segmentation;violation;reconstruction'),(86793,'atlas','taud3pdmaker 		../src/eventcalcvarsfillertool.cxx:67','hi in taud3pdmaker\r\n		../src/eventcalcvarsfillertool.cxx:67: error: \'num_pileup_and_primary_vertices\' is not a member of \'details\'\r\n\r\nsee log\r\n\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/patchwebarea/nicos_web_area166xyvalprod32bs5g4prodopt/nicos_log_rel_5/physicsanalysis_d3pdmaker_taud3pdmaker.html\r\n\r\ncheers\r\ndaniel\r\n','100','5211','2011-09-16 08:48:58','5','fixed','closed','reconstruction',';taud3pdmaker;eventcalcvarsfillertool.cxx;error;num_pileup_and_primary_vertices;member;detail;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;patchwebarea;nicos_web_area166xyvalprod32bs5g4prodopt;nicos_log_rel_5;physicsanalysis_d3pdmaker_taud3pdmaker.html;daniel;taud3pdmaker;eventcalcvarsfillertool.cxx;reconstruction'),(86813,'atlas','rtt 		mb_pu_mu30_d55 fatal in &lt;troot::initsystem&gt;: home directory not set ','hi in 		mb_pu_mu30_d549\r\n			py:dqhistogrammerge   error athena received signal 11. exit code reset to athena exit code 8.\r\n\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_5/installarea/share/bin/reco_trf.py ami=f380 autoconfiguration=everything inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000001.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000002.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000003.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000004.pool.root.1,root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/pileupsamples/valid1.105001.pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00/rdo.365696._000005.pool.root.1 append_preexec=rec.dotrigger=false;rec.dotruth.set_value_and_lock(false) outputaodfile=myaod.pool.root outputesdfile=myesd.pool.root outputhistfile=myhist.root\r\nexitcode=12\r\nerrorcode=61200 (trf_svrinit)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (3 times)\r\nerrorcategory=trf_svrinit,	61200,	servicemanager unable to initialize service\r\nerrorcategory=trf_segvio,	60000,	segmentation violation\r\nfatals=1\r\nerrors=4\r\nwarnings=20\r\nfile myesd.pool.root:\r\n  guid=1c38c417-30e0-e011-b376-001d09f279e0\r\n  metadata:\r\n    amitag=f380\r\n    beamtype=collisions\r\n    filetype=esd\r\n    autoconfiguration=[\'everything\']\r\n    dataset=\r\n    maxevents=-1\r\n    preexec=[\'indetflags.usebeamconstraint.set_value_and_lock(true);from calotools.calonoiseflags import jobproperties;jobproperties.calonoiseflags.fixedluminosity.set_value_and_lock(1.45);aodflags.trackparticleslimmer=true;\', \'rec.dotrigger=false;rec.dotruth.set_value_and_lock(false)\']\r\n    geometryversion=atlas-geo-16-00-00\r\n    events=250\r\n    size=669587514\r\nfile myaod.pool.root:\r\n  guid=ae6c9c4a-40e0-e011-b534-001d09f279e0\r\n  metadata:\r\n    amitag=f380\r\n    beamtype=collisions\r\n    filetype=aod\r\n    autoconfiguration=[\'everything\']\r\n    dataset=\r\n    maxevents=-1\r\n    preexec=[\'indetflags.usebeamconstraint.set_value_and_lock(true);from calotools.calonoiseflags import jobproperties;jobproperties.calonoiseflags.fixedluminosity.set_value_and_lock(1.45);aodflags.trackparticleslimmer=true;\', \'rec.dotrigger=false;rec.dotruth.set_value_and_lock(false)\']\r\n    geometryversion=atlas-geo-16-00-00\r\n    events=250\r\n    size=85326503\r\nerrorinfo\r\n  acronym=trf_svrinit (61200) servicemanager unable to initialize service\r\n  message=error unable to initialize service &quot;hltconfigsvc&quot;\r\n  severity=error\r\n  who=servicemanager\r\n  producer=esdtoaod\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=servicelocatorhelper::service: can not locate service trigconf::hltconfigsvc/hltconfigsvc\r\n  severity=error\r\n  who=trigconfigsvc.hltconfigsvc\r\n  producer=esdtoaod\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=trigconfigsvc can not return a hltchainlist object from any of the configured sources ds xmll1\r\n  severity=error\r\n  who=trigconfigsvc\r\n  producer=esdtoaod\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=trigconfigsvc can not return a ctpconfig object from any of the configured sources ds xmll1\r\n  severity=error\r\n  who=trigconfigsvc\r\n  producer=esdtoaod\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_segvio (60000) segmentation violation\r\n  message=*** break *** segmentation violation\r\n  severity=fatal\r\n  producer=dqhistogrammerge\r\n  stage=initialise\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=815271785\r\nrundiravailabledisk=131613577216\r\nworkdir=/pool/lsf/rtt/169928452\r\nathcode=0\r\nrundiravailabledisk=131826421760\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=684435039\r\natlasrelease=rel_5\r\nworkdir=/pool/lsf/rtt/169928452\r\nathcode=0\r\nrundiravailabledisk=131615297536\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=814302092\r\natlasrelease=rel_5\r\nworkdir=/pool/lsf/rtt/169928452\r\nathcode=8\r\nrundiravailabledisk=131613896704\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=814973734\r\natlasrelease=rel_5\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\nsee log\r\nhttps://atlasrtt.cern.ch/rtt/rel_5/17.0.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/mb_pu_mu30_d549/mb_pu_mu30_d549_log\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-16 14:44:47','5','duplicate','closed','reconstruction',';mb_pu_mu30_d549;py:dqhistogrammerge;error;athena;receive;signal;exit;code;reset;athena;exit;code;command;atla;software;build;nightly;atlasproduction;rel_5;installarea;share;reco_trf.py;autoconfiguration;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;root;castoratla;castor;atla;atlascerngroupdisk;pileupsample;valid1.105001;pythia_minbias.digit.rdo.e603_s932_d549_tid365696_00;pool.root;append_preexec;rec.dotrigger;false;rec.dotruth.set_value_and_lock;false;outputaodfile;myaod.pool.root;outputesdfile;myesd.pool.root;outputhistfile;myhist.root;exitcode;errorcode;trf_svrinit;errorcategory;trf_unknown;unknown;transform;error;time;errorcategory;trf_svrinit;servicemanager;unable;initialize;service;errorcategory;trf_segvio;segmentation;violation;fatal;error;warning;file;myesd.pool.root;guid;metada;amitag;beamtype;collision;filetype;autoconfiguration;dataset;maxevent;preexec;indetflags.usebeamconstraint.set_value_and_lock;true;calotools.calonoiseflags;import;jobproperty;jobproperties.calonoiseflags.fixedluminosity.set_value_and_lock;aodflags.trackparticleslimmer;true;rec.dotrigger;false;rec.dotruth.set_value_and_lock;false;geometryversion;atla;event;size;file;myaod.pool.root;guid;metada;amitag;beamtype;collision;filetype;autoconfiguration;dataset;maxevent;preexec;indetflags.usebeamconstraint.set_value_and_lock;true;calotools.calonoiseflags;import;jobproperty;jobproperties.calonoiseflags.fixedluminosity.set_value_and_lock;aodflags.trackparticleslimmer;true;rec.dotrigger;false;rec.dotruth.set_value_and_lock;false;geometryversion;atla;event;size;errorinfo;acronym;trf_svrinit;servicemanager;unable;initialize;service;message;error;unable;initialize;service;hltconfigsvc;severity;error;servicemanager;producer;esdtoaod;stage;initialise;errorinfo;acronym;trf_unknown;message;servicelocatorhelper;service;locate;service;trigconf;hltconfigsvc;hltconfigsvc;severity;error;trigconfigsvc.hltconfigsvc;producer;esdtoaod;stage;initialise;errorinfo;acronym;trf_unknown;message;trigconfigsvc;return;hltchainlist;object;configure;source;xmll1;severity;error;trigconfigsvc;producer;esdtoaod;stage;initialise;errorinfo;acronym;trf_unknown;message;trigconfigsvc;return;ctpconfig;object;configure;source;xmll1;severity;error;trigconfigsvc;producer;esdtoaod;stage;initialise;errorinfo;acronym;trf_segvio;segmentation;violation;message;break;segmentation;violation;severity;fatal;producer;dqhistogrammerge;stage;initialise;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_5;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_5;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_5;write;jobinfo_recotrf.xml;file;http;atlasrtt.cern.ch;rel_5;build;slc5;gcc43;offline;recjobtransformtest;mb_pu_mu30_d549;mb_pu_mu30_d549_log;daniel;mb_pu_mu30_d55;fatal;home;directory;reconstruction'),(86837,'atlas','rtt: 		details_all_rdo 			transform output file not readable','hi in 		details_all_rdo\r\n			transform output file not readable\r\n\r\n\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_6/installarea/share/bin/rawtoesd_trf.py inputrdofile=/afs/cern.ch/atlas/project/rig/referencefiles/rttinputfiles/rdo.039414._00001.pool.root.1 maxevents=250 autoconfiguration=everything --athenaopts=--stdcmalloc preexec=rec.dodetailedperfmon=true,,from@perfmoncomps.perfmonflags@import@jobproperties@as@pmjp,,pmjp.perfmonflags.dopostprocessing=true,,pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true outputdqmonitorfile=mymon.root outputesdfile=myesd.pool.root outputntup_muoncalibfile=myntup_muoncalib.root\r\nexitcode=41\r\nerrorcode=64120 (trf_outfile_notread)\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nfile myntup_muoncalib.root:\r\n  guid=eea7ea33-b445-4320-b93b-8d664b4bd100\r\n  metadata:\r\n    conditionstag=oflcond-sim-00-00-00\r\n    beamtype=collisions\r\n    filetype=ntup\r\n    autoconfiguration=[\'everything\']\r\n    dataset=\r\n    maxevents=250\r\n    preexec=[\'rec.dodetailedperfmon=true\', \'from perfmoncomps.perfmonflags import jobproperties as pmjp\', \'pmjp.perfmonflags.dopostprocessing=true\', \'pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true\']\r\n    geometryversion=atlas-geo-02-01-00\r\n    events=250\r\n    size=5868665\r\nerrorinfo\r\n  acronym=trf_outfile_notread (64120) transform output file not readable\r\n  message=output file myesd.pool.root failed validation. events could not be counted argument outputesdfile\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_6/installarea/python/pyjobtransformscore/trf.py&quot;, line 1858, in dopostrunactions\r\n              cmd.postrunaction()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_6/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1197, in postrunaction\r\n              self.validatefile()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_6/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1184, in validatefile\r\n              self._filetype.validatefile( self, **self.__validationdict )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_6/installarea/python/pyjobtransformscore/trfutil.py&quot;, line 1371, in validatefile\r\n              raise transformvalidationerror( fname, \'failed validation. events could not be counted argument %s\' % argname, \'trf_outfile_notread\' )\r\nworkdir=/pool/lsf/rtt/170191742\r\nathcode=0\r\nrundiravailabledisk=185816625152\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=455462154\r\natlasrelease=rel_6\r\nwriting jobinfo_rawtoesd.xml file...\r\n\r\n\r\nsee log\r\nhttps://atlasrtt.cern.ch/rtt/rel_6/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/recperftests/details_all_rdo/details_all_rdo_log\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-17 14:44:55','5','none','closed','reconstruction',';details_all_rdo;transform;output;file;readable;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_6;installarea;share;rawtoesd_trf.py;inputrdofile;atla;project;referencefile;rttinputfile;pool.root;maxevent;autoconfiguration;athenaopt;stdcmalloc;preexec;rec.dodetailedperfmon;true;perfmoncomps.perfmonflags;import;jobproperty;pmjp;pmjp.perfmonflags.dopostprocessing;true;pmjp.perfmonflags.dosemidetailedmonitoringfullprint;true;outputdqmonitorfile;mymon.root;outputesdfile;myesd.pool.root;outputntup_muoncalibfile;myntup_muoncalib.root;exitcode;errorcode;trf_outfile_notread;fatal;error;warning;file;myntup_muoncalib.root;guid;metada;conditionstag;oflcond;beamtype;collision;filetype;ntup;autoconfiguration;dataset;maxevent;preexec;rec.dodetailedperfmon;true;perfmoncomps.perfmonflags;import;jobproperty;pmjp;pmjp.perfmonflags.dopostprocessing;true;pmjp.perfmonflags.dosemidetailedmonitoringfullprint;true;geometryversion;atla;event;size;errorinfo;acronym;trf_outfile_notread;transform;output;file;readable;message;output;file;myesd.pool.root;fail;validation;event;count;argument;outputesdfile;severity;fatal;producer;rawtoesd;backtrace;file;atla;software;build;nightly;atlasproduction;rel_6;installarea;python;pyjobtransformscore;trf.py;line;dopostrunaction;cmd.postrunaction;file;atla;software;build;nightly;atlasproduction;rel_6;installarea;python;pyjobtransformscore;basic_trfarg.py;line;postrunaction;self.validatefile;file;atla;software;build;nightly;atlasproduction;rel_6;installarea;python;pyjobtransformscore;basic_trfarg.py;line;validatefile;self;self;self;file;atla;software;build;nightly;atlasproduction;rel_6;installarea;python;pyjobtransformscore;trfutil.py;line;validatefile;raise;transformvalidationerror;fname;fail;validation;event;count;argument;argname;trf_outfile_notread;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_6;write;jobinfo_rawtoesd.xml;file;http;atlasrtt.cern.ch;rel_6;build;slc5;gcc43;offline;recperftest;details_all_rdo;details_all_rdo_log;daniel;details_all_rdo;transform;output;file;readable;reconstruction'),(86845,'atlas','tct, rtt tests: segfault in recalgs','all standardrecofrombs jobs:\r\nstandardrecofrombs\r\nstandardrecofrombs_noreco\r\nstandardrecofrombs_moore\r\nstandardrecofrombs_muonboy\r\nstandardrecofrombs_mon\r\nstandardrecofrombs_combined\r\n\r\ncrash during startup. i could not find out what package exactly causes the crash, log snippet:\r\n\r\nperfmonsvc:    info [dso] cpu= 120.000 ms vmem= 621.879 mb dvmem=   1.805 mb alloc= +0 -0    0.000 kb  =&gt; [/dso/libmuontrackperformance.so]\r\nathalgseq                                            info member list: timingalg/recotimerbegin, eventcounter,\r\n[...]\r\ntimingalg/recotimerbeforeoutput, cbnt_athenaaware,\r\n[...]\r\ncore dump from coredumpsvc on lxbrg4001.cern.ch at sat sep 17 08:13:30 2011\r\ncaught signal 11(segmentation fault). details: \r\n[...]\r\nstack trace:\r\n[...]\r\n 0x2b68ac592288 std::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::basic_string(char const*, std::allocator&lt;char&gt; const&amp;) ??:0\r\n[...]\r\n 0x2aaab3b11d15 operator[] inlined at ../src/timingalg.cxx:60 in timingalg::initialize() /afs/cern.ch/sw/lcg/contrib/gcc/4.3.5/x86_64-slc5-gcc34-opt/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.3.5/../../../../include/c++/4.3.5/bits/basic_string.h:724   + 0x325 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_6/installarea/x86_64-slc5-gcc43-opt/lib/librecalgs.so]\r\n 0x2b68b459efc5 algorithm::sysinitialize() ??:0   + 0x1e5 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_6/installarea/x86_64-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n\r\ni couldn\'t find out where timingalg resides, but most likely it caused the crash.\r\n\r\nfull log example:\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_6/devval/build/x86_64-slc5-gcc43-opt/offline/muonrecrtt/standardrecofrombs/standardrecofrombs_log\r\n\r\ncheers, christian\r\nmuon sw shifter','1658','8587','2011-09-18 14:38:46','5','none','closed','none',';standardrecofromb;standardrecofromb;standardrecofrombs_noreco;standardrecofrombs_moore;standardrecofrombs_muonboy;standardrecofrombs_mon;standardrecofrombs_combined;crash;startup;package;exactly;cause;crash;snippet;perfmonsvc;info;vmem;dvmem;alloc;libmuontrackperformance.so;athalgseq;info;member;list;timingalg;recotimerbegin;eventcounter;timingalg;recotimerbeforeoutput;cbnt_athenaaware;core;dump;coredumpsvc;lxbrg4001;catch;signal;segmentation;fault;detail;stack;trace;basic_string;allocator;basic_string;char;const;allocator;const;operator;inline;timingalg.cxx;timingalg;initialize;contrib;4.3.5;x86_64;slc5;gcc34;x86_64;unknown;linux;4.3.5;include;4.3.5;basic_string.h;atla;software;build;nightly;devval;atlasreconstruction;rel_6;installarea;x86_64;slc5;gcc43;librecalgs.so;algorithm;sysinitialize;atla;software;build;nightly;devval;gaudus;rel_6;installarea;x86_64;slc5;gcc43;libgaudikernel.so;couldn\'t;timingalg;reside;likely;cause;crash;example;http;atlasrtt.cern.ch;rtt_j_r;rel_6;devval;build;x86_64;slc5;gcc43;offline;muonrecrtt;standardrecofromb;standardrecofrombs_log;christian;muon;shifter;test;segfault;recalg'),(86887,'atlas',' task 512548: athena ran out of memory ','hi, one reprocessing job failed in es cloud with message:\r\n\r\nathena ran out of memory exe: trf_unknown | &quot;out of memory: saving summary ...&quot; | &quot;failed to create persistent object: std::bad_alloc&quot; | &quot;createrep failed, key = tracks&quot; | &quot;could not create rep for dataobject (clid/key):117682415 tracks&quot; | &quot;streamobjects failed.&quot; | &quot;pool commitan\r\nin:   data11_7tev.00187457.physics_background.merge.raw out: data11_7tev.00187457.physics_background.recon.hist.r271\r\n\r\n\r\nlog file extracts:\r\n\r\nmismatch | trf error code mismatch: exeerrorcode = 69999, transexitcode = 99- job report produced by recotrf -\r\ncommand=/software/atlas/prod/releases/rel_17-04/atlasproduction/17.0.3.3/installarea/share/bin/reco_trf.py inputbsfile=data11_7tev.00187457.physics_background.merge.raw._lb0385._sfo-all._0001.1 runnumber=187457 autoconfiguration=everything,conditionstag=comcond-blkp*-005-06 dbrelease=16090101._0187457.tar.gz dbrelease=dbrelease-16.9.1.1.tar.gz amitag=r2713 outputesdfile=esd.512548._000244.pool.root.22 outputhistfile=hist.512548._000244.root.22 --extraignorefilters=trigbsextraction_error_ignore.db\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (58 times)\r\nfatals=7\r\nerrors=10\r\nwarnings=8\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=out of memory: saving summary ...\r\n  severity=fatal\r\n  who=athenasummarysvc\r\n  producer=rawtoesd\r\n  stage=execute\r\n  backtrace=file &quot;/software/atlas/prod/releases/rel_17-04/atlasproduction/17.0.3.3/installarea/python/pyjobtransformscore/trf.py&quot;, line 2041, in runjob\r\n              maxevents = maxeventsarg.value()\r\n  message=failed to create persistent object: std::bad_alloc\r\n  severity=error\r\n  who=athenapoolconverter\r\n  message=createrep failed, key = tracks\r\n  message=could not create rep for dataobject (clid/key):117682415 tracks\r\n  who=streamesd.streamesdtool\r\n  message=streamobjects failed.\r\n  who=streamesd\r\nerrorinfo (count=2)\r\n  message=pool commitandhold failed 0xb3d69d8\r\n  who=poolsvc\r\n  message=commitoutput failed to commitandhold outputstream.\r\n  who=athenapoolcnvsvc\r\n  message=unable to commit output esd.512548._000244.pool.root.22\r\n  message=commitoutput failed.\r\n  message=terminating event processing loop due to errors\r\n  who=athenaeventloopmgr\r\n  message=athenapoolconverter createrep failed to get token, key = streamesd\r\n  message=could not create rep for dataobject (clid/key):167728019 streamesd\r\n  who=streamesd_fh.streamesd_fhtool\r\n  message=athenapoolconverter createrep failed to get token, key = incompletelumiblocks\r\n  who=strea\r\n\r\n- walltime -\r\n\r\n\r\nthe link the the failed job is:\r\n\r\nhttp://panda.cern.ch/server/pandamon/query?job=*&amp;cloud=es&amp;type=production&amp;jobstatus=failed&amp;hours=1&amp;processingtype=reprocessing&amp;workinggroup=reprocessing\r\n\r\nwould you please have a look?\r\n\r\ncheers,\r\nmarisa (adcos shifter)\r\n','100','4281','2011-09-19 15:18:53','9','wont fix','closed','none',';reprocess;fail;cloud;message;athena;memory;trf_unknown;memory;save;summary;fail;create;persistent;object;bad_alloc;createrep;fail;track;create;dataobject;clid;track;streamobject;fail;pool;commitan;data11_7tev;physics_background.merge.raw;data11_7tev;physics_background.recon.hist.r271;file;extract;mismatch;error;code;mismatch;exeerrorcode;transexitcode;report;produce;recotrf;command;software;atla;prod;release;rel_17;atlasproduction;17.0.3.3;installarea;share;reco_trf.py;inputbsfile;data11_7tev;physics_background.merge.raw;runnumber;autoconfiguration;conditionstag;comcond;blkp;dbrelease;tar.gz;dbrelease;dbrelease;16.9.1.1;tar.gz;amitag;outputesdfile;pool.root;outputhistfile;hist;root;extraignorefilter;trigbsextraction_error_ignore.db;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;memory;save;summary;severity;fatal;athenasummarysvc;producer;rawtoesd;stage;execute;backtrace;file;software;atla;prod;release;rel_17;atlasproduction;17.0.3.3;installarea;python;pyjobtransformscore;trf.py;line;runjob;maxevent;maxeventsarg.value;message;fail;create;persistent;object;bad_alloc;severity;error;athenapoolconverter;message;createrep;fail;track;message;create;dataobject;clid;track;streamesd.streamesdtool;message;streamobject;fail;streamesd;errorinfo;count;message;pool;commitandhold;fail;poolsvc;message;commitoutput;fail;commitandhold;outputstream;athenapoolcnvsvc;message;unable;commit;output;pool.root;message;commitoutput;fail;message;terminate;event;processing;loop;error;athenaeventloopmgr;message;athenapoolconverter;createrep;fail;token;streamesd;message;create;dataobject;clid;streamesd;streamesd_fh.streamesd_fhtool;message;athenapoolconverter;createrep;fail;token;incompletelumiblock;strea;walltime;link;fail;http;panda.cern.ch;server;pandamon;query;cloud;type;production;jobstatus;fail;hour;processingtype;reprocess;workinggroup;reprocess;look;marisa;adco;shifter;task;athena;memory'),(86885,'atlas','rtt: a lot of failures du to syntaxerror: invalid syntax','hi\r\n\r\ntoday a lot of rtt tests in 17.0.x.y failed due to problems like\r\n\r\n	traceback (most recent call last):\r\n  			file &quot;/afs/cern.ch/atlas/software/builds/atlasreconstruction/16.6.0/installarea/share/bin/reco_trf.py&quot;, line 8, in ?\r\n    			from pyjobtransformscore.trf import author,jobreport\r\n  			file &quot;/afs/cern.ch/atlas/software/builds/atlascore/16.6.0/installarea/python/pyjobtransformscore/trf.py&quot;, line 117\r\n    			with open( cmtversion ) as cmtversionfile:\r\n            			^\r\n			syntaxerror: invalid syntax\r\n\r\n	traceback (most recent call last):\r\n  	file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_1/installarea/share/bin/rawtoesd_trf.py&quot;, line 5, in &lt;module&gt;\r\n    	from patjobtransforms.baseofbasictrf import baseofbasictrf\r\n  	file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_1/installarea/python/patjobtransforms/baseofbasictrf.py&quot;, line 3, in &lt;module&gt;\r\n    	from pyjobtransformscore.trf import jobtransform\r\n  	file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_1/installarea/python/pyjobtransformscore/trf.py&quot;, line 20, in &lt;module&gt;\r\n    	from pyjobtransformscore.trfutil import *\r\n  	file &quot;/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_1/installarea/python/pyjobtransformscore/trfutil.py&quot;, line 16, in &lt;module&gt;\r\n    	from pyutils import athfile \r\n	importerror: no module named pyutils\r\n\r\n\r\n/afs/cern.ch/atlas/project/rtt/prod/results/rtt/rel_1/17.0.x.y/build/i686-slc5-gcc43-opt/offline/recperftests/easytrf_ftag_muons_3evts/runscript.sh: line 73: easy_trf.py: command not found\r\n\r\n\r\n\r\n\r\nsee log\r\nhttps://atlasrtt.cern.ch/rtt/rel_1/17.0.x.y/build/i686-slc5-gcc43-opt/offline/recperftests/athenaapp_all_rdo/athenaapp_all_rdo_log\r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_1/17.0.x.y/build/i686-slc5-gcc43-opt/offline/recperftests/easytrf_ftag_muons_3evts/easytrf_ftag_muons_3evts_log\r\n\r\ncheers\r\ndaniel\r\n\r\n','100','5211','2011-09-19 14:21:44','5','unreproducible','closed','reconstruction',';today;test;fail;problem;like;traceback;recent;file;atla;software;build;atlasreconstruction;16.6.0;installarea;share;reco_trf.py;line;pyjobtransformscore.trf;import;author;jobreport;file;atla;software;build;atlascore;16.6.0;installarea;python;pyjobtransformscore;trf.py;line;open;cmtversion;cmtversionfile;syntaxerror;invalid;syntax;traceback;recent;file;atla;software;build;nightly;atlasproduction;rel_1;installarea;share;rawtoesd_trf.py;line;patjobtransforms.baseofbasictrf;import;baseofbasictrf;file;atla;software;build;nightly;atlasproduction;rel_1;installarea;python;patjobtransform;baseofbasictrf.py;line;pyjobtransformscore.trf;import;jobtransform;file;atla;software;build;nightly;atlasproduction;rel_1;installarea;python;pyjobtransformscore;trf.py;line;pyjobtransformscore.trfutil;import;file;atla;software;build;nightly;atlasproduction;rel_1;installarea;python;pyjobtransformscore;trfutil.py;line;pyutil;import;athfile;importerror;module;name;pyutil;atla;project;prod;result;rel_1;build;slc5;gcc43;offline;recperftest;easytrf_ftag_muons_3evts;runscript.sh;line;easy_trf.py;command;http;atlasrtt.cern.ch;rel_1;build;slc5;gcc43;offline;recperftest;athenaapp_all_rdo;athenaapp_all_rdo_log;http;atlasrtt.cern.ch;rel_1;build;slc5;gcc43;offline;recperftest;easytrf_ftag_muons_3evts;easytrf_ftag_muons_3evts_log;daniel;failure;syntaxerror;invalid;syntax;reconstruction'),(86895,'atlas','task 512531 (data11_7tev 17.0.3.3 reco) is failing with &quot;allocation failed&quot; error','all jobs from task failed at taiwan-lcg2 with following error:\r\n\r\nerror details: trans: ath_failure - athena non-zero exit \r\n\r\nexample of failed job:\r\nhttp://panda.cern.ch/server/pandamon/query?job=1311397760\r\nlog file extract ( http://voatlas140.cern.ch/monitor/logs/bd021136-de16-4fca-a241-0217c6995e5a/tarball_pandajob_1311397760_taiwan-lcg2/athena_stdout.txt ):\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nsrc/tcmalloc.cc:1908] allocation failed: 12\r\nsrc/tcmalloc.cc:1919]  aborting!\r\n\r\n(pid=32177 ppid=32057) received fatal signal 6 (aborted)\r\nsignal context:\r\n  signo  = 6, errno = 0, code = -6 (*unknown reason*)\r\n  pid    = 32177, uid = 41000\r\n  value  = (-2147361079, 0x8001dec9)\r\n  stack  = (2, 0, (nil))\r\n\r\n  eip: 0023:ffffe410           eflags: 00200296\r\n  eax: 00000000   ebx: 00007db1   ecx: 00007db1   edx: 00000006\r\n  esi: ffd76b0c   edi: 009b8ff4   ebp: ffd76a6c   esp: ffd76a54\r\n   ds: 002b        es: 002b        fs: 0000        ss: 002b\r\n\r\n  signal esp: ffd76a54  trap: 0/0  oldmask: 3ff8ea07   cr2: 00000000\r\n\r\n  fpu:  control = ffff037f\r\n        status  = ffff4020\r\n        tag     = ffffffff\r\n        ip      = 0010:f3fa00aa\r\n        data    = 002b:fff71678\r\n        state   = 00004020\r\n    %fp0 = [4004:7e8b29df9755a5ae]\r\n    %fp1 = [3ffc:376e1558af10ef6c]\r\n    %fp2 = [c008:f074d9ad15ab98ed]\r\n    %fp3 = [c003:dced529f23bdcfa5]\r\n    %fp4 = [3ffd:085d09d7f55c829c]\r\n    %fp5 = [4002:cf3881c9670bfb06]\r\n    %fp6 = [3ff7:c17f52a20e0ff443]\r\n    %fp7 = [0000:0000000000000000]\r\n\r\nstack trace:\r\n 0xf40227a0 athena::debugaids::stacktrace(int)+0x60 ??:0   + 0x60 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0xf40237bf athena::signal::fataldump(int, siginfo*, void*)+0xff ??:0   + 0xff [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0xf4023ad1 athena::signal::fatal(int, siginfo*, void*)+0x181 ??:0   + 0x181 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0xffffe600 &lt;unknown function&gt;\r\n 0xffffe410 &lt;unknown function&gt;\r\n 0x0088cdf0 gsignal+0x50 ??:0   + 0x50 [/lib/libc.so.6]\r\n 0x0088e701 abort+0x101 ??:0   + 0x101 [/lib/libc.so.6]\r\n 0xf7fd2a3a tcmalloc_central_freelist::populate()+0x27a /afs/cern.ch/user/s/seuster/scratch0/gpt/google-perftools-0.99.2c/src/tcmalloc.cc:1920   + 0x27a [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/external/tcmalloc/google-perftools-0.99.2c/i686-slc5-gcc43-opt/lib/libtcmalloc_minimal.so]\r\n 0xf7fd2a98 tcmalloc_central_freelist::fetchfromspanssafe()+0x38 /afs/cern.ch/user/s/seuster/scratch0/gpt/google-perftools-0.99.2c/src/tcmalloc.cc:1872   + 0x38 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/external/tcmalloc/google-perftools-0.99.2c/i686-slc5-gcc43-opt/lib/libtcmalloc_minimal.so]\r\n 0xf7fd4996 tcmalloc_central_freelist::removerange(void**, void**, int)+0x76 /afs/cern.ch/user/s/seuster/scratch0/gpt/google-perftools-0.99.2c/src/tcmalloc.cc:1850   + 0x76 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/external/tcmalloc/google-perftools-0.99.2c/i686-slc5-gcc43-opt/lib/libtcmalloc_minimal.so]\r\n 0xf7fd4ace tcmalloc_threadcache::fetchfromcentralcache(unsigned int, unsigned int)+0x4e /afs/cern.ch/user/s/seuster/scratch0/gpt/google-perftools-0.99.2c/src/tcmalloc.cc:2036   + 0x4e [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/external/tcmalloc/google-perftools-0.99.2c/i686-slc5-gcc43-opt/lib/libtcmalloc_minimal.so]\r\n 0xf7fdb7a8 operator new(unsigned int)+0x3e8 /afs/cern.ch/user/s/seuster/scratch0/gpt/google-perftools-0.99.2c/src/tcmalloc.cc:2003   + 0x3e8 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/external/tcmalloc/google-perftools-0.99.2c/i686-slc5-gcc43-opt/lib/libtcmalloc_minimal.so]\r\n 0xe8dbae6c trk::materialeffectsontrack::clone() const+0x1c ??:0   + 0x1c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlasconditions/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libtrkmaterialontrack.so]\r\n 0xf3c185cd trk::trackstateonsurface::trackstateonsurface(trk::trackstateonsurface const&amp;)+0x9d ??:0   + 0x9d [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlasevent/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libtrktrack.so]\r\n 0xe011f358 indet::sitrajectory_xk::converttonexttrackstateonsurface()+0x1d8 ??:0   + 0x1d8 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlasreconstruction/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libsicombinatorialtrackfindertool_xk.so]\r\n 0xe00ff614 indet::sicombinatorialtrackfinder_xk::converttonexttrack()+0x24 ??:0   + 0x24 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlasreconstruction/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libsicombinatorialtrackfindertool_xk.so]\r\n 0xe00ffff8 indet::sicombinatorialtrackfinder_xk::gettracks(trk::parameterst&lt;trk::charged&gt; const&amp;, std::list&lt;trk::spacepoint const*, std::allocator&lt;trk::spacepoint const*&gt; &gt; const&amp;, std::list&lt;indetdd::sidetectorelement const*, std::allocator&lt;indetdd::sidetectorelement const*&gt; &gt;&amp;, std::multimap&lt;trk::preprawdata const*, trk::track const*, std::less&lt;trk::preprawdata const*&gt;, std::allocator&lt;std::pair&lt;trk::preprawdata const* const, trk::track const*&gt; &gt; &gt;&amp;)+0x188 ??:0   + 0x188 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlasreconstruction/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libsicombinatorialtrackfindertool_xk.so]\r\n 0xe01ce751 indet::sitrackmaker_xk::gettracks(std::list&lt;trk::spacepoint const*, std::allocator&lt;trk::spacepoint const*&gt; &gt; const&amp;)+0x221 ??:0   + 0x221 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlasreconstruction/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libsitrackmakertool_xk.so]\r\n 0xe8c9bfeb indet::sispseededtrackfinder::execute()+0x26b ??:0   + 0x26b [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlasreconstruction/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libsispseededtrackfinder.so]\r\n 0xf5058abd algorithm::sysexecute()+0x11d ??:0   + 0x11d [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0xeb3656fb athsequencer::execute()+0x25b ??:0   + 0x25b [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libgaudisequencer.so]\r\n 0xf5058abd algorithm::sysexecute()+0x11d ??:0   + 0x11d [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0xeb3656fb athsequencer::execute()+0x25b ??:0   + 0x25b [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libgaudisequencer.so]\r\n 0xf5058abd algorithm::sysexecute()+0x11d ??:0   + 0x11d [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0xf260490f athenaeventloopmgr::executealgorithms()+0x6f ??:0   + 0x6f [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf260ae66 athenaeventloopmgr::executeevent(void*)+0x6f6 ??:0   + 0x6f6 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf2608d52 athenaeventloopmgr::nextevent(int)+0x8d2 ??:0   + 0x8d2 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf260b359 athenaeventloopmgr::executerun(int)+0x29 ??:0   + 0x29 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf2c102f9 applicationmgr::executerun(int)+0x79 ??:0   + 0x79 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudisvc.so]\r\n 0xf34f0f2d _zn12_global__n_1l11method_4304epvs0_rkst6vectoris0_sais0_ees0_+0x3d ??:0   + 0xf34f0f2d [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikerneldict.so]\r\n 0xf5912a57 root::cintex::method_stub_with_context(root::cintex::stubcontext_t*, g__value*, char const*, g__param*, int)+0x157 ??:0   + 0x157 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/lcgcmt/lcgcmt_60c/installarea/i686-slc5-gcc43-opt/lib/libcintex.so]\r\n 0xf298d034 &lt;unknown function&gt;\r\n 0xf6c07572 cint::g__callfunc::execute(void*)+0xd2 ??:0   + 0xd2 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/lcgcmt/lcgcmt_60c/installarea/i686-slc5-gcc43-opt/lib/libcint.so]\r\n 0xf78e375c pyroot::trootobjectbyvalueexecutor::execute(cint::g__callfunc*, void*)+0x2c ??:0   + 0x2c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf78ea134 pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::callsafe(void*)+0x64 ??:0   + 0x64 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf78ea2cc pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::execute(void*)+0x6c ??:0   + 0x6c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf78e87f3 pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::operator()(pyroot::objectproxy*, _object*, _object*, long)+0x153 ??:0   + 0x153 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf78efb2a pyroot::(anonymous namespace)::mp_call(pyroot::methodproxy*, _object*, _object*)+0x4aa ??:0   + 0xf78efb2a [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d969d pyeval_evalframeex+0x2dfd /build/agaspar/work/python-2.6.5/python/ceval.c:3968   + 0x2dfd [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dc329 pyeval_evalframeex+0x5a89 /build/agaspar/work/python-2.6.5/python/ceval.c:3846   + 0x5a89 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dddd3 pyeval_evalcode+0x63 /build/agaspar/work/python-2.6.5/python/ceval.c:547   + 0x63 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe044 pyrun_fileexflags+0xb4 /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339   + 0xb4 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d4900 builtin_execfile+0x1d0 /build/agaspar/work/python-2.6.5/python/bltinmodule.c:797   + 0x80d4900 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x081391bd pycfunction_call+0x18d /build/agaspar/work/python-2.6.5/objects/methodobject.c:81   + 0x18d [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dbd09 pyeval_evalframeex+0x5469 /build/agaspar/work/python-2.6.5/python/ceval.c:3750   + 0x5469 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08137caa function_call+0xaa /build/agaspar/work/python-2.6.5/objects/funcobject.c:524   + 0x8137caa [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08064935 instancemethod_call+0x145 /build/agaspar/work/python-2.6.5/objects/classobject.c:2579   + 0x8064935 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080b29e3 slot_tp_call+0x83 /build/agaspar/work/python-2.6.5/objects/typeobject.c:5373   + 0x80b29e3 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d969d pyeval_evalframeex+0x2dfd /build/agaspar/work/python-2.6.5/python/ceval.c:3968   + 0x2dfd [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dddd3 pyeval_evalcode+0x63 /build/agaspar/work/python-2.6.5/python/ceval.c:547   + 0x63 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe044 pyrun_fileexflags+0xb4 /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339   + 0xb4 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d4900 builtin_execfile+0x1d0 /build/agaspar/work/python-2.6.5/python/bltinmodule.c:797   + 0x80d4900 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x081391bd pycfunction_call+0x18d /build/agaspar/work/python-2.6.5/objects/methodobject.c:81   + 0x18d [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dbd09 pyeval_evalframeex+0x5469 /build/agaspar/work/python-2.6.5/python/ceval.c:3750   + 0x5469 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08137caa function_call+0xaa /build/agaspar/work/python-2.6.5/objects/funcobject.c:524   + 0x8137caa [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08064935 instancemethod_call+0x145 /build/agaspar/work/python-2.6.5/objects/classobject.c:2579   + 0x8064935 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080b29e3 slot_tp_call+0x83 /build/agaspar/work/python-2.6.5/objects/typeobject.c:5373   + 0x80b29e3 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d969d pyeval_evalframeex+0x2dfd /build/agaspar/work/python-2.6.5/python/ceval.c:3968   + 0x2dfd [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dddd3 pyeval_evalcode+0x63 /build/agaspar/work/python-2.6.5/python/ceval.c:547   + 0x63 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe044 pyrun_fileexflags+0xb4 /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339   + 0xb4 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe3a4 pyrun_simplefileexflags+0x204 /build/agaspar/work/python-2.6.5/python/pythonrun.c:935   + 0x204 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe69a pyrun_anyfileexflags+0x7a /build/agaspar/work/python-2.6.5/python/pythonrun.c:739   + 0x7a [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08058d46 py_main+0xc56 /build/agaspar/work/python-2.6.5/modules/main.c:572   + 0xc56 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08057e42 main+0x32 /build/agaspar/work/python-2.6.5/./modules/python.c:24   + 0x32 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x00879e9c __libc_start_main+0xdc ??:0   + 0xdc [/lib/libc.so.6]\r\n 0x08057d51 _start+0x21 ??:0   + 0x8057d51 [/cvmfs/atlas.cern.ch/repo/sw/software/i686-slc5-gcc43-opt/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code -6.\r\npy:rawtoesd         error athena received signal 6. exit code reset to athena exit code 8.','100','6514','2011-09-19 17:25:47','9','wont fix','closed','none',';task;fail;taiwan;lcg2;follow;error;error;detail;trans;ath_failure;athena;zero;exit;example;fail;http;panda.cern.ch;server;pandamon;query;file;extract;http;voatlas140;monitor;tarball_pandajob_1311397760_taiwan;lcg2;athena_stdout.txt;dataproxy;warn;accessda;iconversionsvc;tcmalloc.cc;allocation;fail;tcmalloc.cc;abort;ppid;receive;fatal;signal;aborted;signal;context;signo;errno;code;unknown;reason;value;stack;eflag;esus;edus;signal;trap;oldmask;control;status;datum;state;stack;trace;athena;debugaid;stacktrace;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libcxxutils.so;athena;signal;fataldump;siginfo;void;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libcxxutils.so;athena;signal;fatal;siginfo;void;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libcxxutils.so;gsignal;libc.so;abort;libc.so;tcmalloc_central_freelist;populate;user;seuster;scratch0;google;perftool;0.99.2c;tcmalloc.cc;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;tcmalloc;google;perftool;0.99.2c;slc5;gcc43;libtcmalloc_minimal.so;tcmalloc_central_freelist;fetchfromspanssafe;user;seuster;scratch0;google;perftool;0.99.2c;tcmalloc.cc;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;tcmalloc;google;perftool;0.99.2c;slc5;gcc43;libtcmalloc_minimal.so;tcmalloc_central_freelist;removerange;void;void;user;seuster;scratch0;google;perftool;0.99.2c;tcmalloc.cc;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;tcmalloc;google;perftool;0.99.2c;slc5;gcc43;libtcmalloc_minimal.so;tcmalloc_threadcache;fetchfromcentralcache;unsigned;unsigned;user;seuster;scratch0;google;perftool;0.99.2c;tcmalloc.cc;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;tcmalloc;google;perftool;0.99.2c;slc5;gcc43;libtcmalloc_minimal.so;operator;unsigned;user;seuster;scratch0;google;perftool;0.99.2c;tcmalloc.cc;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;tcmalloc;google;perftool;0.99.2c;slc5;gcc43;libtcmalloc_minimal.so;materialeffectsontrack;clone;const;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascondition;17.0.3;installarea;slc5;gcc43;libtrkmaterialontrack.so;trackstateonsurface;trackstateonsurface;trackstateonsurface;const;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlasevent;17.0.3;installarea;slc5;gcc43;libtrktrack.so;indet;sitrajectory_xk;converttonexttrackstateonsurface;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlasreconstruction;17.0.3;installarea;slc5;gcc43;libsicombinatorialtrackfindertool_xk.so;indet;sicombinatorialtrackfinder_xk;converttonexttrack;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlasreconstruction;17.0.3;installarea;slc5;gcc43;libsicombinatorialtrackfindertool_xk.so;indet;sicombinatorialtrackfinder_xk;gettrack;parameterst;const;list;const;list;multimap;allocator;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlasreconstruction;17.0.3;installarea;slc5;gcc43;libsicombinatorialtrackfindertool_xk.so;indet;sitrackmaker_xk;gettrack;list;const;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlasreconstruction;17.0.3;installarea;slc5;gcc43;libsitrackmakertool_xk.so;indet;sispseededtrackfinder;execute;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlasreconstruction;17.0.3;installarea;slc5;gcc43;libsispseededtrackfinder.so;algorithm;sysexecute;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikernel.so;athsequencer;execute;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libgaudisequencer.so;algorithm;sysexecute;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikernel.so;athsequencer;execute;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libgaudisequencer.so;algorithm;sysexecute;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikernel.so;athenaeventloopmgr;executealgorithm;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;athenaeventloopmgr;executeevent;void;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;athenaeventloopmgr;nextevent;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;athenaeventloopmgr;executerun;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;applicationmgr;executerun;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudisvc.so;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikerneldict.so;root;cintex;method_stub_with_context;root;cintex;stubcontext_t;g__value;char;const;g__param;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;lcgcmt;lcgcmt_60c;installarea;slc5;gcc43;libcintex.so;cint;g__callfunc;execute;void;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;lcgcmt;lcgcmt_60c;installarea;slc5;gcc43;libcint.so;pyroot;trootobjectbyvalueexecutor;execute;cint;g__callfunc;void;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;callsafe;void;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;execute;void;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;operator;pyroot;objectproxy;long;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;anonymous;namespace;mp_call;pyroot;methodproxy;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcode;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_fileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;builtin_execfile;build;agaspar;work;python;2.6.5;python;bltinmodule.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pycfunction_call;build;agaspar;work;python;2.6.5;object;methodobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;function_call;build;agaspar;work;python;2.6.5;object;funcobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;instancemethod_call;build;agaspar;work;python;2.6.5;object;classobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;slot_tp_call;build;agaspar;work;python;2.6.5;object;typeobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcode;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_fileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;builtin_execfile;build;agaspar;work;python;2.6.5;python;bltinmodule.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pycfunction_call;build;agaspar;work;python;2.6.5;object;methodobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;function_call;build;agaspar;work;python;2.6.5;object;funcobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;instancemethod_call;build;agaspar;work;python;2.6.5;object;classobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;slot_tp_call;build;agaspar;work;python;2.6.5;object;typeobject.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcode;build;agaspar;work;python;2.6.5;python;ceval.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_fileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_simplefileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_anyfileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;py_main;build;agaspar;work;python;2.6.5;module;main.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;main;build;agaspar;work;python;2.6.5;module;python.c;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;libc.so;cvmf;atlas.cern.ch;repo;software;slc5;gcc43;17.0.3;external;python;2.6.5;slc5;gcc43;python;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;py:rawtoesd;error;athena;receive;signal;exit;code;reset;athena;exit;code;task;data11_7tev;17.0.3.3;reco;fail;allocation;fail;error'),(86912,'atlas','atlasanalysis 	hsg2dpdutils	../x86_64-slc5-gcc43-opt/libhsg2dpdutils.so: no rule for such target.  stop.','hi in\r\n\r\natlasanalysis 	hsg2dpdutils	../x86_64-slc5-gcc43-opt/libhsg2dpdutils.so: no rule for such target.  stop.\r\n\r\nsee log\r\n\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x0val64bs5g4anlopt/nicos_log_rel_2/physicsanalysis_higgsphys_hsg2_hsg2dpdutils.html\r\n\r\n\r\ncheers\r\ndaniel','1216','5211','2011-09-20 08:44:27','5','none','closed','reconstruction',';atlasanalysis;hsg2dpdutil;x86_64;slc5;gcc43;libhsg2dpdutils.so;rule;target;stop;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x0val64bs5g4anlopt;nicos_log_rel_2;physicsanalysis_higgsphys_hsg2_hsg2dpdutils.html;daniel;atlasanalysis;hsg2dpdutil;x86_64;slc5;gcc43;libhsg2dpdutils.so;rule;target;stop;reconstruction'),(86947,'atlas','jetperformance	reconstruction/jet  		./src/jetperformancetoolsplots.cxx:1888: error: no matching function for call to \'t','hi in\r\n\r\njetperformance	reconstruction/jet \r\n		./src/jetperformancetoolsplots.cxx:1888: error: no matching function for call to \'tgraph::sethistogram(th1*&amp;)\'\r\n\r\nsee log\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x064bs5g4anlopt/nicos_log_rel_3/reconstruction_jet_jetperformance.html\r\n\r\ncheers\r\ndaniel','3114','5211','2011-09-21 09:00:44','5','none','closed','reconstruction',';jetperformance;reconstruction;jetperformancetoolsplots.cxx;error;match;function;tgraph;sethistogram;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x064bs5g4anlopt;nicos_log_rel_3;reconstruction_jet_jetperformance.html;daniel;jetperformance;reconstruction;jetperformancetoolsplots.cxx;error;match;function;reconstruction'),(86948,'atlas','generatorobjects compilation failed','hi in 	generatorobjects	generators\r\n		mceventdict_gen_rflx.cpp:6070: error: invalid use of incomplete type \'struct hepmc::constgenparticleendrange\'\r\n\r\nsee log \r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x0val64bs5g4evtopt/nicos_log_rel_3/generators_generatorobjects.html\r\n\r\ncheers\r\ndaniel','455','5211','2011-09-21 09:10:36','5','fixed','closed','reconstruction',';generatorobject;generator;mceventdict_gen_rflx.cpp;error;invalid;incomplete;type;struct;hepmc;constgenparticleendrange;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x0val64bs5g4evtopt;nicos_log_rel_3;generators_generatorobjects.html;daniel;generatorobject;compilation;fail;reconstruction'),(86949,'atlas','atlfastalgs compilation failed','hi in generatorobjects	generators  #86948\r\n		mceventdict_gen_rflx.cpp:6070: error: invalid use of incomplete type \'struct hepmc::constgenparticleendrange\'\r\n\r\nsee log\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x0val64bs5g4recopt/nicos_log_rel_3/simulation_atlfast_atlfastalgs.html\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-21 09:13:00','5','fixed','closed','reconstruction',';generatorobject;generator;mceventdict_gen_rflx.cpp;error;invalid;incomplete;type;struct;hepmc;constgenparticleendrange;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x0val64bs5g4recopt;nicos_log_rel_3;simulation_atlfast_atlfastalgs.html;daniel;atlfastalg;compilation;fail;reconstruction'),(87002,'atlas','mugirl	reconstruction/muonidentification mugirl.cxx:941: error: \'class mugirlns::iglobalfittool\' has no member named \'ms','hi in mugirl	reconstruction/muonidentification\r\n\r\n../src/mugirl.cxx: in member function \'virtual statuscode mugirl::execute()\':\r\n../src/mugirl.cxx:941: error: \'class mugirlns::iglobalfittool\' has no member named \'msfit\'\r\n../src/mugirl.cxx:962: error: \'class mugirlns::icandidatetool\' has no member named \'fillmstrack\'\r\n../src/mugirl.cxx:987: error: \'class mugirlns::icandidatetool\' has no member named \'fillmstrack\'\r\n../src/mugirl.cxx:990: error: \'class mugirlns::icandidatetool\' has no member named \'fillmstrack\'\r\n\r\nsee log\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x0val64bs5g4recopt/nicos_log_rel_4/reconstruction_muonidentification_mugirl.html\r\n\r\ncheers\r\ndaniel','8379','5211','2011-09-22 10:02:30','5','none','closed','reconstruction',';mugirl;reconstruction;muonidentification;mugirl.cxx;member;function;virtual;statuscode;mugirl;execute;mugirl.cxx;error;class;mugirln;iglobalfittool;member;name;msfit;mugirl.cxx;error;class;mugirln;icandidatetool;member;name;fillmstrack;mugirl.cxx;error;class;mugirln;icandidatetool;member;name;fillmstrack;mugirl.cxx;error;class;mugirln;icandidatetool;member;name;fillmstrack;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x0val64bs5g4recopt;nicos_log_rel_4;reconstruction_muonidentification_mugirl.html;daniel;mugirl;reconstruction;muonidentification;mugirl.cxx;error;class;mugirln;iglobalfittool;member;name;reconstruction'),(87005,'atlas','tct devval, rel_4: importerror: no module named mugirlconf ','hi there are many errors today in tct tests.\r\n\r\ndue to athacronym=an error occurred when importing a file\r\n\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_4/installarea/share/bin/reco_trf.py ami=q200 maxevents=500 outputesdfile=myesd_express_8.pool.root outputaodfile=myaod_express_8.aod.pool.root outputtagfile_a2t=mytag_express_8.root --uploadtoami=0.0\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=2\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 11000 (exit code 10)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: thu sep 22 07:47:50 2011 cest, vmem: 317.6171875 mb, rss: 103.7109375 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 00 hrs 00 mins 51 secs, cpu-time: 00 hrs 00 mins 25 secs, vmem: 1.1640625 mb, rss: 1.07421875 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_4/installarea/share/bin/reco_trf.py ami=q200 maxevents=500 outputesdfile=myesd_express_8.pool.root outputaodfile=myaod_express_8.aod.pool.root outputtagfile_a2t=mytag_express_8.root --uploadtoami=0.0\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=2\r\nathacronym=an error occurred when importing a file\r\nathcode=3\r\nrundiruseddisk=2356759941\r\nrundiravailabledisk=184824922112\r\nworkdir=/pool/lsf/rtt/171577279\r\nathcode=3\r\nrundiravailabledisk=184825196544\r\ndbrelease=\r\nathacronym=an error occurred when importing a file\r\nrundiruseddisk=2356544941\r\natlasrelease=rel_4\r\nwriting jobinfo_recotrf.xml file...\r\ntct_recotrf.py: starting 2011-09-22 07:45:42.030999\r\ntct_recotrf.py: working directory is /pool/lsf/rtt/171577279\r\nconfig {\'dohi_\': false, \'nfiles_\': 3, \'dryrun_\': false, \'ami\': \'q200\', \'preexec_r2e\': \'\', \'lb_\': \'550,552\', \'append_preexec\': \'\', \'maxevents\': 500, \'preinclude\': \'\', \'postexec\': \'\', \'uselatestrun_\': false, \'jobnum_\': 8, \'postexec_e2a\': \'\', \'dropntups_\': true, \'runnumber\': \'\', \'--ignoreerrors\': \'\', \'amitag\': \'\', \'--uploadtoami\': 0.0, \'postinclude\': \'\', \'dropdesds_\': true, \'readrdo_\': false, \'triggerconfig\': \'\', \'trigstream_\': \'express\', \'castordir_\': \'/castor/cern.ch/grid/atlas/tzero/prod1/perm/data11_7tev/express_express/00183021/data11_7tev.00183021.express_express.merge.raw\', \'autoconfiguration\': \'\', \'drophist_\': true, \'preexec\': \'\', \'runnum_\': \'00183021\', \'dropdraws_\': true}\r\ntct_recotrf.py: generate copy link script\r\ntct_recotrf.py : checktctcastor : starting...\r\ntct_recotrf.py : checktctcastor : check only files with lumiblocks within [550, 552]\r\ntct_recotrf.py : checktctcastor : /castor/cern.ch/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_express/data11_7tev.00183021.express_express.merge.raw._lb0550._sfo-all._0001.1 is good file, ngoodfiles = 1\r\ntct_recotrf.py : checktctcastor : /castor/cern.ch/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_express/data11_7tev.00183021.express_express.merge.raw._lb0551._sfo-all._0001.1 is good file, ngoodfiles = 2\r\ntct_recotrf.py : checktctcastor : /castor/cern.ch/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_express/data11_7tev.00183021.express_express.merge.raw._lb0552._sfo-all._0001.1 is good file, ngoodfiles = 3\r\ntct_recotrf.py : checktctcastor : test = 1 ...end\r\ntct_recotrf.py: start copying files... 2011-09-22 07:45:43.729178\r\ntct_recotrf.py : copytctcastor : stage_host=castoratlast3\r\ntct_recotrf.py : copytctcastor : stage_svcclass=atlascerngroupdisk\r\ntct_recotrf.py : copytctcastor : look for files in /castor/cern.ch/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_express\r\ntct_recotrf.py : copytctcastor : copying /castor/cern.ch/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_express/data11_7tev.00183021.express_express.merge.raw._lb0550._sfo-all._0001.1 to 0.data\r\ntct_recotrf.py : copytctcastor : copying /castor/cern.ch/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_express/data11_7tev.00183021.express_express.merge.raw._lb0551._sfo-all._0001.1 to 1.data\r\ntct_recotrf.py : copytctcastor : copying /castor/cern.ch/atlas/atlascerngroupdisk/proj-sit/tct/rec_input/00183021/physics_express/data11_7tev.00183021.express_express.merge.raw._lb0552._sfo-all._0001.1 to 2.data\r\nall files copied from castor are ok !\r\ntct_recotrf.py: ...files copying done  2011-09-22 07:46:28.465885\r\ntct_recotrf.py: set atlcoolt0setup.py\r\ntct_recotrf.py: print list of input raw data files :\r\ntct_recotrf.py : generaterecotrfcmd : found only 0 data files =&gt; set it as number of files!\r\n\r\ntct_recotrf.py: exact reco_trf.py command is &quot; reco_trf.py  ami=q200 maxevents=500 outputesdfile=myesd_express_8.pool.root outputaodfile=myaod_express_8.aod.pool.root outputtagfile_a2t=mytag_express_8.root --uploadtoami=0.0 &quot;\r\n\r\ntct_recotrf.py: starting reco_trf.py... 2011-09-22 07:46:37.850487\r\n\r\nsee log\r\nhttps://atlasrtt.cern.ch/tct/rel_4/devval/build/x86_64-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express8_collisions_withouthist/run00183021_express8_collisions_withouthist_log\r\n\r\ncheers\r\ndaniel','8379','5211','2011-09-22 10:26:02','5','duplicate','closed','reconstruction',';error;today;test;athacronym;error;occur;import;file;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;devval;atlasreconstruction;rel_4;installarea;share;reco_trf.py;maxevent;outputesdfile;myesd_express_8;pool.root;outputaodfile;myaod_express_8;aod.pool.root;outputtagfile_a2t;mytag_express_8;root;uploadtoamus;exitcode;errorcode;ath_failure;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;devval;atlasreconstruction;rel_4;installarea;share;reco_trf.py;maxevent;outputesdfile;myesd_express_8;pool.root;outputaodfile;myaod_express_8;aod.pool.root;outputtagfile_a2t;mytag_express_8;root;uploadtoamus;exitcode;errorcode;ath_failure;fatal;error;warning;athacronym;error;occur;import;file;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;error;occur;import;file;rundiruseddisk;atlasrelease;rel_4;write;jobinfo_recotrf.xml;file;tct_recotrf.py;start;tct_recotrf.py;work;directory;pool;config;dohi_;false;nfiles_;dryrun_;false;preexec_r2e;append_preexec;maxevent;preinclude;postexec;uselatestrun_;false;jobnum_;postexec_e2a;dropntups_;true;runnumber;ignoreerror;amitag;uploadtoamus;postinclude;dropdesds_;true;readrdo_;false;triggerconfig;trigstream_;express;castordir_;castor;grid;atla;tzero;prod1;perm;data11_7tev;express_express;data11_7tev;express_express.merge.raw;autoconfiguration;drophist_;true;preexec;runnum_;dropdraws_;true;tct_recotrf.py;generate;copy;link;script;tct_recotrf.py;checktctcastor;start;tct_recotrf.py;checktctcastor;check;file;lumiblock;tct_recotrf.py;checktctcastor;castor;atla;atlascerngroupdisk;proj;rec_input;physics_express;data11_7tev;express_express.merge.raw;good;file;ngoodfile;tct_recotrf.py;checktctcastor;castor;atla;atlascerngroupdisk;proj;rec_input;physics_express;data11_7tev;express_express.merge.raw;good;file;ngoodfile;tct_recotrf.py;checktctcastor;castor;atla;atlascerngroupdisk;proj;rec_input;physics_express;data11_7tev;express_express.merge.raw;good;file;ngoodfile;tct_recotrf.py;checktctcastor;test;tct_recotrf.py;start;copying;file;tct_recotrf.py;copytctcastor;stage_host;castoratlast3;tct_recotrf.py;copytctcastor;stage_svcclass;atlascerngroupdisk;tct_recotrf.py;copytctcastor;look;file;castor;atla;atlascerngroupdisk;proj;rec_input;physics_express;tct_recotrf.py;copytctcastor;copying;castor;atla;atlascerngroupdisk;proj;rec_input;physics_express;data11_7tev;express_express.merge.raw;datum;tct_recotrf.py;copytctcastor;copying;castor;atla;atlascerngroupdisk;proj;rec_input;physics_express;data11_7tev;express_express.merge.raw;datum;tct_recotrf.py;copytctcastor;copying;castor;atla;atlascerngroupdisk;proj;rec_input;physics_express;data11_7tev;express_express.merge.raw;datum;file;copy;castor;tct_recotrf.py;file;copying;tct_recotrf.py;atlcoolt0setup.py;tct_recotrf.py;print;list;input;datum;file;tct_recotrf.py;generaterecotrfcmd;datum;file;number;file;tct_recotrf.py;exact;reco_trf.py;command;reco_trf.py;maxevent;outputesdfile;myesd_express_8;pool.root;outputaodfile;myaod_express_8;aod.pool.root;outputtagfile_a2t;mytag_express_8;root;uploadtoamus;tct_recotrf.py;start;reco_trf.py;http;atlasrtt.cern.ch;rel_4;devval;build;x86_64;slc5;gcc43;offline;tier0chaintest;run00183021_express8_collisions_withouthist;run00183021_express8_collisions_withouthist_log;daniel;devval;rel_4;importerror;module;name;mugirlconf;reconstruction'),(87020,'atlas','segmentation fault in metrefalg','hi, \r\n\r\none job was aborted by segmentation fault in metrefalg. \r\ni will reproduce the problem. \r\n\r\ncheers, \r\nsusumu \r\n\r\n+verbatim+\r\ninput file(s):\r\n--------------------------------------------------------------------------------\r\nstage_svcclass=atlcal\r\n/castor/cern.ch/grid/atlas/tzero/prod1/perm/data11_7tev/physics_egamma/00189561/data11_7tev.00189561.physics_egamma.merge.raw/data11_7tev.00189561.physics_egamma.merge.raw._lb0327._sfo-4._0001.1\r\n\r\n\r\ntier-0 transformation command run:\r\n--------------------------------------------------------------------------------\r\nsource /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/setup/setuptrf.sh /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/patches/atlasproduction-17.0.3.6 atlasproduction-17.0.3.6 /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/setup/specialsetup_tier0.sh ; python -u `which reco_trf.py` --argdict=/afs/cern.ch/atlas/tzero/prod1/log2/13681/13681292_data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.3//data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.argdict.gpickle  &amp;&gt; data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.tmp.log\r\n\r\n\r\nami configuration tag (more details):\r\n--------------------------------------------------------------------------------\r\nsee http://ami.in2p3.fr:8080/opencms/opencms/ami/www/referencetables/ --&gt; configurationtag --&gt; interpret --&gt; f405\r\n\r\n\r\nlogfile location:\r\n--------------------------------------------------------------------------------\r\nstage_svcclass=t0merge\r\n/castor/cern.ch/grid/atlas/tzero/prod1/temp/data11_7tev/physics_egamma/00189561/data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task.tmp.log/data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.tmp.log.3\r\n\r\n\r\ntail of the log file (300 lines):\r\n--------------------------------------------------------------------------------\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63011207, run #189561 311 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012115, run #189561 311 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012115, run #189561 312 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63011722, run #189561 312 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63011722, run #189561 313 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63011538, run #189561 313 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63011538, run #189561 314 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012008, run #189561 314 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012008, run #189561 315 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63013206, run #189561 315 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63013206, run #189561 316 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012426, run #189561 316 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012426, run #189561 317 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012025, run #189561 317 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012025, run #189561 318 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012409, run #189561 318 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012409, run #189561 319 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012887, run #189561 319 events processed so far  &lt;&lt;&lt;===\r\nperfmonsvc:    info [evt: 319] cpu=31044.000 ms vmem=2192.328 mb dvmem=  23.398 mb alloc= +0 -0    0.000 kb  =&gt; [/evt/perfmonslice]\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012887, run #189561 320 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63013945, run #189561 320 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63013945, run #189561 321 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012800, run #189561 321 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012800, run #189561 322 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012082, run #189561 322 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012082, run #189561 323 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63013799, run #189561 323 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63013799, run #189561 324 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014300, run #189561 324 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014300, run #189561 325 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014717, run #189561 325 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014717, run #189561 326 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014117, run #189561 326 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014117, run #189561 327 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014508, run #189561 327 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014508, run #189561 328 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63013345, run #189561 328 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63013345, run #189561 329 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014312, run #189561 329 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014312, run #189561 330 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63013170, run #189561 330 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63013170, run #189561 331 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012962, run #189561 331 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012962, run #189561 332 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014646, run #189561 332 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014646, run #189561 333 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63012960, run #189561 333 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63012960, run #189561 334 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014718, run #189561 334 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014718, run #189561 335 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014849, run #189561 335 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014849, run #189561 336 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63016132, run #189561 336 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63016132, run #189561 337 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63015162, run #189561 337 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63015162, run #189561 338 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63015677, run #189561 338 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63015677, run #189561 339 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63015129, run #189561 339 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63015129, run #189561 340 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63016410, run #189561 340 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63016410, run #189561 341 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63015289, run #189561 341 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63015289, run #189561 342 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63014628, run #189561 342 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63014628, run #189561 343 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63016482, run #189561 343 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63016482, run #189561 344 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63016157, run #189561 344 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63016157, run #189561 345 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63015687, run #189561 345 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63015687, run #189561 346 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63016612, run #189561 346 events processed so far  &lt;&lt;&lt;===\r\ndataproxy                                         warning accessdata:  iconversionsvc ptr not set\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #63016612, run #189561 347 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #63016933, run #189561 347 events processed so far  &lt;&lt;&lt;===\r\n-------------------------------------------------------------------------------------\r\ncore dump from coredumpsvc on lxbst2703.cern.ch at thu sep 22 16:00:31 2011\r\n\r\ncaught signal 11(segmentation fault). details: \r\n  errno = 0, code = 1 (address not mapped to object)\r\n  pid   = 64, uid = 1061514144\r\n  value = (-1982750381, 0x89d1a153)\r\n  vmem = 2192.33 mb\r\n  rss  = 1817.1 mb\r\n  total-ram = 32049.5 mb\r\n  free-ram  = 2835.29 mb\r\n  buffer-ram= 482.801 mb\r\n  total-swap= 36867.8 mb\r\n  free-swap = 36705.1 mb\r\n  addr  = 0x40\r\n\r\nevent counter: 348\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [189561,63016933,327,1316522742:272214420,1067,0xfffffffffff7]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent algorithm: metrefalg\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   metrefalg\r\n-------------------------------------------------------------------------------------\r\n\r\n(pid=9776 ppid=9672) received fatal signal 11 (segmentation fault)\r\nsignal context:\r\n  signo  = 11, errno = 0, code = 1 (address not mapped to object)\r\n  pid    = 64, uid = 1061514144\r\n  value  = (-1982750381, 0x89d1a153)\r\n  addr   = 0x40\r\n  stack  = (2, 0, (nil))\r\n\r\n  eip: 0023:e3531721           eflags: 00210282\r\n  eax: 4ac09750   ebx: e36168d8   ecx: 0e9f7000   edx: 00000000\r\n  esi: 00000000   edi: 5909d378   ebp: ffb818d8   esp: ffb817f8\r\n   ds: 002b        es: 002b        fs: 0000        ss: 002b\r\n\r\n  signal esp: ffb817f8  trap: 14/4  oldmask: 00000000   cr2: 00000040\r\n\r\n  fpu:  control = ffff037f\r\n        status  = ffff0020\r\n        tag     = ffffffff\r\n        ip      = 0010:dc843266\r\n        data    = 002b:16dea7d0\r\n        state   = 00000020\r\n    %fp0 = [3ff9:40005e35c05bd8c0]\r\n    %fp1 = [0000:0000000000000000]\r\n    %fp2 = [3ffe:c8003e540a40d970]\r\n    %fp3 = [3fff:0000000000008000]\r\n    %fp4 = [4009:f0007791f040cbcf]\r\n    %fp5 = [400a:9000c64925ccd192]\r\n    %fp6 = [400a:9000c64925ccd192]\r\n    %fp7 = [400a:9000c64925ccd192]\r\n\r\nstack trace:\r\n 0xf41747a0 athena::debugaids::stacktrace(int)+0x60 ??:0   + 0x60 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0xf41757bf athena::signal::fataldump(int, siginfo*, void*)+0xff ??:0   + 0xff [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0xf4175ad1 athena::signal::fatal(int, siginfo*, void*)+0x181 ??:0   + 0x181 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0xf268b006 coredumpsvchandler::action(int, siginfo*, void*)+0x106 ??:0   + 0x106 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xffffe600 &lt;unknown function&gt;\r\n 0xe3531721 mettrackselectortool::select(rec::trackparticle const*, caloclustercontainer const*)+0xe1 ??:0   + 0xe1 [/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.6/installarea/i686-slc5-gcc43-opt/lib/libmissingetlib.so]\r\n 0xe350279a metrefinedclustertool::executeco(caloclustercontainer const*, missinget*, missingetcomposition*)+0x60a ??:0   + 0x60a [/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.6/installarea/i686-slc5-gcc43-opt/lib/libmissingetlib.so]\r\n 0xe3505b59 metrefinedclustertool::execute(missingetcomposition*)+0x259 ??:0   + 0x259 [/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.6/installarea/i686-slc5-gcc43-opt/lib/libmissingetlib.so]\r\n 0xe34c1170 metrefinedtoolbase::exectool(missingetcomposition*)+0x60 ??:0   + 0x60 [/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.6/installarea/i686-slc5-gcc43-opt/lib/libmissingetlib.so]\r\n 0xe355b9d2 metrefinedalgorithm::execute()+0x4d2 ??:0   + 0x4d2 [/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.6/installarea/i686-slc5-gcc43-opt/lib/libmissingetlib.so]\r\n 0xf51aaabd algorithm::sysexecute()+0x11d ??:0   + 0x11d [/afs/cern.ch/atlas/software/releases/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0xe79206fb athsequencer::execute()+0x25b ??:0   + 0x25b [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libgaudisequencer.so]\r\n 0xf51aaabd algorithm::sysexecute()+0x11d ??:0   + 0x11d [/afs/cern.ch/atlas/software/releases/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0xe79206fb athsequencer::execute()+0x25b ??:0   + 0x25b [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libgaudisequencer.so]\r\n 0xf51aaabd algorithm::sysexecute()+0x11d ??:0   + 0x11d [/afs/cern.ch/atlas/software/releases/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0xf274f90f athenaeventloopmgr::executealgorithms()+0x6f ??:0   + 0x6f [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf2755e66 athenaeventloopmgr::executeevent(void*)+0x6f6 ??:0   + 0x6f6 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf2753d52 athenaeventloopmgr::nextevent(int)+0x8d2 ??:0   + 0x8d2 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf2756359 athenaeventloopmgr::executerun(int)+0x29 ??:0   + 0x29 [/afs/cern.ch/atlas/software/releases/17.0.3/atlascore/17.0.3/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0xf2d5b2f9 applicationmgr::executerun(int)+0x79 ??:0   + 0x79 [/afs/cern.ch/atlas/software/releases/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudisvc.so]\r\n 0xf363bf2d _zn12_global__n_1l11method_4304epvs0_rkst6vectoris0_sais0_ees0_+0x3d ??:0   + 0xf363bf2d [/afs/cern.ch/atlas/software/releases/17.0.3/gaudi/v22r1p4/installarea/i686-slc5-gcc43-opt/lib/libgaudikerneldict.so]\r\n 0xf5c29a57 root::cintex::method_stub_with_context(root::cintex::stubcontext_t*, g__value*, char const*, g__param*, int)+0x157 ??:0   + 0x157 [/afs/cern.ch/atlas/software/releases/17.0.3/lcgcmt/lcgcmt_60c/installarea/i686-slc5-gcc43-opt/lib/libcintex.so]\r\n 0xf2ad8034 &lt;unknown function&gt;\r\n 0xf6ed8572 cint::g__callfunc::execute(void*)+0xd2 ??:0   + 0xd2 [/afs/cern.ch/atlas/software/releases/17.0.3/lcgcmt/lcgcmt_60c/installarea/i686-slc5-gcc43-opt/lib/libcint.so]\r\n 0xf7bb475c pyroot::trootobjectbyvalueexecutor::execute(cint::g__callfunc*, void*)+0x2c ??:0   + 0x2c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf7bbb134 pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::callsafe(void*)+0x64 ??:0   + 0x64 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf7bbb2cc pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::execute(void*)+0x6c ??:0   + 0x6c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf7bb97f3 pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::operator()(pyroot::objectproxy*, _object*, _object*, long)+0x153 ??:0   + 0x153 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0xf7bc0b2a pyroot::(anonymous namespace)::mp_call(pyroot::methodproxy*, _object*, _object*)+0x4aa ??:0   + 0xf7bc0b2a [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/app/releases/root/5.28.00e/i686-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d969d pyeval_evalframeex+0x2dfd /build/agaspar/work/python-2.6.5/python/ceval.c:3968   + 0x2dfd [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dc329 pyeval_evalframeex+0x5a89 /build/agaspar/work/python-2.6.5/python/ceval.c:3846   + 0x5a89 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dddd3 pyeval_evalcode+0x63 /build/agaspar/work/python-2.6.5/python/ceval.c:547   + 0x63 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe044 pyrun_fileexflags+0xb4 /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339   + 0xb4 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d4900 builtin_execfile+0x1d0 /build/agaspar/work/python-2.6.5/python/bltinmodule.c:797   + 0x80d4900 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x081391bd pycfunction_call+0x18d /build/agaspar/work/python-2.6.5/objects/methodobject.c:81   + 0x18d [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dbd09 pyeval_evalframeex+0x5469 /build/agaspar/work/python-2.6.5/python/ceval.c:3750   + 0x5469 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08137caa function_call+0xaa /build/agaspar/work/python-2.6.5/objects/funcobject.c:524   + 0x8137caa [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08064935 instancemethod_call+0x145 /build/agaspar/work/python-2.6.5/objects/classobject.c:2579   + 0x8064935 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080b29e3 slot_tp_call+0x83 /build/agaspar/work/python-2.6.5/objects/typeobject.c:5373   + 0x80b29e3 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d969d pyeval_evalframeex+0x2dfd /build/agaspar/work/python-2.6.5/python/ceval.c:3968   + 0x2dfd [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dddd3 pyeval_evalcode+0x63 /build/agaspar/work/python-2.6.5/python/ceval.c:547   + 0x63 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe044 pyrun_fileexflags+0xb4 /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339   + 0xb4 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d4900 builtin_execfile+0x1d0 /build/agaspar/work/python-2.6.5/python/bltinmodule.c:797   + 0x80d4900 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x081391bd pycfunction_call+0x18d /build/agaspar/work/python-2.6.5/objects/methodobject.c:81   + 0x18d [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dbd09 pyeval_evalframeex+0x5469 /build/agaspar/work/python-2.6.5/python/ceval.c:3750   + 0x5469 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08137caa function_call+0xaa /build/agaspar/work/python-2.6.5/objects/funcobject.c:524   + 0x8137caa [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08064935 instancemethod_call+0x145 /build/agaspar/work/python-2.6.5/objects/classobject.c:2579   + 0x8064935 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080b29e3 slot_tp_call+0x83 /build/agaspar/work/python-2.6.5/objects/typeobject.c:5373   + 0x80b29e3 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x0805d88c pyobject_call+0x5c /build/agaspar/work/python-2.6.5/objects/abstract.c:2493   + 0x5c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080d969d pyeval_evalframeex+0x2dfd /build/agaspar/work/python-2.6.5/python/ceval.c:3968   + 0x2dfd [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080ddc8c pyeval_evalcodeex+0x87c /build/agaspar/work/python-2.6.5/python/ceval.c:3000   + 0x87c [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080dddd3 pyeval_evalcode+0x63 /build/agaspar/work/python-2.6.5/python/ceval.c:547   + 0x63 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe044 pyrun_fileexflags+0xb4 /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339   + 0xb4 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe3a4 pyrun_simplefileexflags+0x204 /build/agaspar/work/python-2.6.5/python/pythonrun.c:935   + 0x204 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x080fe69a pyrun_anyfileexflags+0x7a /build/agaspar/work/python-2.6.5/python/pythonrun.c:739   + 0x7a [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08058d46 py_main+0xc56 /build/agaspar/work/python-2.6.5/modules/main.c:572   + 0xc56 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x08057e42 main+0x32 /build/agaspar/work/python-2.6.5/./modules/python.c:24   + 0x32 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\n 0x00383e9c __libc_start_main+0xdc ??:0   + 0xdc [/lib/libc.so.6]\r\n 0x08057d51 _start+0x21 ??:0   + 0x8057d51 [/afs/cern.ch/atlas/software/releases/17.0.3/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code -11.\r\npy:rawtoesd         error athena received signal 11. exit code reset to athena exit code 8.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/patches/atlasproduction-17.0.3.6/installarea/share/atlas_error_categories.db\r\ninfo: read 53 error patterns from file /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/patches/atlasproduction-17.0.3.6/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/tzero/prod1/inst/projects/data11/patches/atlasproduction-17.0.3.6/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: thu sep 22 16:00:42 2011 cest, vmem: 101.81640625 mb, rss: 68.37109375 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,17.0.3.6 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info done scanning 19182 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 0\r\npy:rawtoesd          info    warnings: 318\r\npy:rawtoesd          info    errors  : 0\r\npy:rawtoesd          info    fatals  : 2\r\nathenalogchecker stop [rawtoesd log] wall-time: thu sep 22 16:00:43 2011 cest, vmem: 101.81640625 mb, rss: 68.38671875 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 01 secs, cpu-time: 00 hrs 00 mins 01 secs, vmem: 0.0 mb, rss: 0.015625 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 60010 (exit code 6)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: thu sep 22 16:00:43 2011 cest, vmem: 101.81640625 mb, rss: 68.40625 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 01 hrs 06 mins 16 secs, cpu-time: 01 hrs 04 mins 51 secs, vmem: 0.44921875 mb, rss: 0.546875 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.6/installarea/share/bin/reco_trf.py --argdict=/afs/cern.ch/atlas/tzero/prod1/log2/13681/13681292_data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.3//data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.argdict.gpickle\r\nexitcode=6\r\nerrorcode=60010 (trf_segfault)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault (2 times)\r\nfatals=2\r\nerrors=0\r\nwarnings=8\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 60010 (exit code 6)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: thu sep 22 16:00:43 2011 cest, vmem: 101.96875 mb, rss: 68.58203125 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 01 hrs 06 mins 16 secs, cpu-time: 01 hrs 04 mins 51 secs, vmem: 0.6015625 mb, rss: 0.77734375 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.3/atlasproduction/17.0.3.6/installarea/share/bin/reco_trf.py --argdict=/afs/cern.ch/atlas/tzero/prod1/log2/13681/13681292_data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.3//data11_7tev.00189561.physics_egamma.merge.raw.f405.recon.task._lb0327._sfo-4._0001.job.argdict.gpickle\r\nexitcode=6\r\nerrorcode=60010 (trf_segfault)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault (2 times)\r\nfatals=2\r\nerrors=0\r\nwarnings=8\r\nerrorinfo\r\n  acronym=trf_segfault (60010) segmentation fault\r\n  message=caught signal 11(segmentation fault). details:\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_segfault (60010) segmentation fault\r\n  message=(pid=9776 ppid=9672) received fatal signal 11 (segmentation fault)\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=execute\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=1282457563\r\nrundiravailabledisk=1101254995968\r\nworkdir=/pool/lsf/atlast0/171700176\r\nathcode=8\r\nrundiravailabledisk=1101255323648\r\ndbrelease=current\r\nathacronym=an unknown exception occurred\r\n-verbatim-','4343','4045','2011-09-22 16:22:26','7','fixed','closed','missinget',';aborted;segmentation;fault;metrefalg;reproduce;problem;susumu;verbatim;input;file;stage_svcclass;atlcal;castor;grid;atla;tzero;prod1;perm;data11_7tev;physics_egamma;data11_7tev;physics_egamma.merge.raw;data11_7tev;physics_egamma.merge.raw;tier;transformation;command;source;atla;tzero;prod1;inst;project;data11;setup;setuptrf.sh;atla;tzero;prod1;inst;project;data11;patch;atlasproduction;17.0.3.6;atlasproduction;17.0.3.6;atla;tzero;prod1;inst;project;data11;setup;specialsetup_tier0;python;reco_trf.py;argdict;atla;tzero;prod1;log2;13681292_data11_7tev;physics_egamma.merge.raw.f405;recon.task;data11_7tev;physics_egamma.merge.raw.f405;recon.task;job.argdict.gpickle;data11_7tev;physics_egamma.merge.raw.f405;recon.task;job.tmp.log;configuration;detail;http;ami.in2p3;opencm;opencm;referencetable;configurationtag;interpret;logfile;location;stage_svcclass;t0merge;castor;grid;atla;tzero;prod1;temp;data11_7tev;physics_egamma;data11_7tev;physics_egamma.merge.raw.f405;recon.task.tmp.log;data11_7tev;physics_egamma.merge.raw.f405;recon.task;job.tmp.log;tail;file;line;dataproxy;warn;accessda;iconversionsvc;athenaeventloopmgr;info;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;perfmonslice;athenaeventloopmgr;info;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;processing;event;event;process;start;processing;event;event;process;mettrackselectortool;select;trackparticle;const;caloclustercontainer;const;atla;software;release;17.0.3;atlasproduction;17.0.3.6;installarea;slc5;gcc43;libmissingetlib.so;metrefinedclustertool;executeco;caloclustercontainer;const;missinget;missingetcomposition;atla;software;release;17.0.3;atlasproduction;17.0.3.6;installarea;slc5;gcc43;libmissingetlib.so;metrefinedclustertool;execute;missingetcomposition;atla;software;release;17.0.3;atlasproduction;17.0.3.6;installarea;slc5;gcc43;libmissingetlib.so;metrefinedtoolbase;exectool;missingetcomposition;atla;software;release;17.0.3;atlasproduction;17.0.3.6;installarea;slc5;gcc43;libmissingetlib.so;metrefinedalgorithm;execute;atla;software;release;17.0.3;atlasproduction;17.0.3.6;installarea;slc5;gcc43;libmissingetlib.so;algorithm;sysexecute;atla;software;release;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikernel.so;athsequencer;execute;atla;software;release;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libgaudisequencer.so;algorithm;sysexecute;atla;software;release;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikernel.so;athsequencer;execute;atla;software;release;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libgaudisequencer.so;algorithm;sysexecute;atla;software;release;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikernel.so;athenaeventloopmgr;executealgorithm;atla;software;release;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;athenaeventloopmgr;executeevent;void;atla;software;release;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;athenaeventloopmgr;nextevent;atla;software;release;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;athenaeventloopmgr;executerun;atla;software;release;17.0.3;atlascore;17.0.3;installarea;slc5;gcc43;libathenaservices.so;applicationmgr;executerun;atla;software;release;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudisvc.so;atla;software;release;17.0.3;gaudus;v22r1p4;installarea;slc5;gcc43;libgaudikerneldict.so;root;cintex;method_stub_with_context;root;cintex;stubcontext_t;g__value;char;const;g__param;atla;software;release;17.0.3;lcgcmt;lcgcmt_60c;installarea;slc5;gcc43;libcintex.so;cint;g__callfunc;execute;void;atla;software;release;17.0.3;lcgcmt;lcgcmt_60c;installarea;slc5;gcc43;libcint.so;pyroot;trootobjectbyvalueexecutor;execute;cint;g__callfunc;void;atla;software;release;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;callsafe;void;atla;software;release;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;execute;void;atla;software;release;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;operator;pyroot;objectproxy;long;atla;software;release;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyroot;anonymous;namespace;mp_call;pyroot;methodproxy;atla;software;release;17.0.3;release;root;5.28.00e;slc5;gcc43;root;libpyroot.so;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcode;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_fileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;builtin_execfile;build;agaspar;work;python;2.6.5;python;bltinmodule.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pycfunction_call;build;agaspar;work;python;2.6.5;object;methodobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;function_call;build;agaspar;work;python;2.6.5;object;funcobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;instancemethod_call;build;agaspar;work;python;2.6.5;object;classobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;slot_tp_call;build;agaspar;work;python;2.6.5;object;typeobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcode;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_fileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;builtin_execfile;build;agaspar;work;python;2.6.5;python;bltinmodule.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pycfunction_call;build;agaspar;work;python;2.6.5;object;methodobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;function_call;build;agaspar;work;python;2.6.5;object;funcobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;instancemethod_call;build;agaspar;work;python;2.6.5;object;classobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;slot_tp_call;build;agaspar;work;python;2.6.5;object;typeobject.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyobject_call;build;agaspar;work;python;2.6.5;object;abstract.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalframeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcodeex;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyeval_evalcode;build;agaspar;work;python;2.6.5;python;ceval.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_fileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_simplefileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;pyrun_anyfileexflags;build;agaspar;work;python;2.6.5;python;pythonrun.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;py_main;build;agaspar;work;python;2.6.5;module;main.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;main;build;agaspar;work;python;2.6.5;module;python.c;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;libc.so;atla;software;release;17.0.3;external;python;2.6.5;slc5;gcc43;python;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;py:rawtoesd;error;athena;receive;signal;exit;code;reset;athena;exit;code;info;read;error;category;file;atla;tzero;prod1;inst;project;data11;patch;atlasproduction;17.0.3.6;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;tzero;prod1;inst;project;data11;patch;atlasproduction;17.0.3.6;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;tzero;prod1;inst;project;data11;patch;atlasproduction;17.0.3.6;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;17.0.3.6;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;release;17.0.3;atlasproduction;17.0.3.6;installarea;share;reco_trf.py;argdict;atla;tzero;prod1;log2;13681292_data11_7tev;physics_egamma.merge.raw.f405;recon.task;data11_7tev;physics_egamma.merge.raw.f405;recon.task;job.argdict.gpickle;exitcode;errorcode;trf_segfault;errorcategory;trf_segfault;segmentation;fault;time;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;release;17.0.3;atlasproduction;17.0.3.6;installarea;share;reco_trf.py;argdict;atla;tzero;prod1;log2;13681292_data11_7tev;physics_egamma.merge.raw.f405;recon.task;data11_7tev;physics_egamma.merge.raw.f405;recon.task;job.argdict.gpickle;exitcode;errorcode;trf_segfault;errorcategory;trf_segfault;segmentation;fault;time;fatal;error;warning;errorinfo;acronym;trf_segfault;segmentation;fault;message;catch;signal;segmentation;fault;detail;severity;fatal;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_segfault;segmentation;fault;message;ppid;receive;fatal;signal;segmentation;fault;severity;fatal;producer;rawtoesd;stage;execute;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;atlast0;athcode;rundiravailabledisk;dbrelease;current;athacronym;unknown;exception;occur;verbatim;segmentation;fault;metrefalg;missinget'),(87055,'atlas','iluminositytool not found','\r\n   hello,\r\n\r\n   a number of rtt tests fail in rel_5 with the following message\r\n\r\n  toolsvc                                             error cannot create tool iluminositytool (no factory found)\r\n\r\n   for example https://atlasrtt.cern.ch/rtt_j_r/rel_5/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_allstreams_a/data_7tev_allstreams_a_log\r\n\r\n   compilation problem ?\r\n\r\n      sharka\r\n\r\n\r\n  https://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n         ','6358','5211','2011-09-23 13:16:54','5','none','closed','reconstruction',';hello;number;test;fail;rel_5;follow;message;toolsvc;error;create;tool;iluminositytool;factory;example;http;atlasrtt.cern.ch;rtt_j_r;rel_5;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;data_7tev_allstreams_a;data_7tev_allstreams_a_log;compilation;problem;sharka;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;iluminositytool;reconstruction'),(87084,'atlas','fpe in indettrtsegmentsfinder_ec_ctb','\r\n   hi,\r\n\r\n     a problem appeared in devval rtts rel_6 ( probably present for\r\n some time but shadowed by other problems )\r\n\r\n  recjobtransformtests/amitag_q120\r\n\r\n  log file location : https://atlasrtt.cern.ch/rtt_j_r/rel_6/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/amitag_q120/amitag_q120_log\r\n  \r\n    please investigate\r\n\r\n   sharka (validation shifter)\r\n   \r\n\r\n https://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation','100','5211','2011-09-24 17:16:16','5','duplicate','closed','reconstruction',';problem;appear;devval;rel_6;probably;present;time;shadow;problem;recjobtransformtest;amitag_q120;file;location;http;atlasrtt.cern.ch;rtt_j_r;rel_6;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;amitag_q120;amitag_q120_log;investigate;sharka;validation;shifter;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;indettrtsegmentsfinder_ec_ctb;reconstruction'),(87086,'atlas','error in &lt;tstreamerinfo::build&gt;: eventbookkeeper_p2','\r\n  hi,\r\n\r\n   the following error message appears in devval rtts\r\nrecperftests/all_bs, details_all_bs ..\r\n  \r\n\r\n    &quot;error in &lt;tstreamerinfo::build&gt;: eventbookkeeper_p2, unknown type: enum* m_childreneb &quot;\r\n\r\n   log file (example) : https://atlasrtt.cern.ch/rtt_j_r/rel_6/devval/build/x86_64-slc5-gcc43-opt/offline/recperftests/details_all_bs/details_all_bs_log\r\n\r\n   please have a look\r\n\r\n   sharka ( validation shifter )\r\n   \r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation','100','5211','2011-09-24 17:34:37','5','duplicate','closed','reconstruction',';follow;error;message;appear;devval;recperftest;all_bs;details_all_bs;error;eventbookkeeper_p2;unknown;type;enum;m_childreneb;file;example;http;atlasrtt.cern.ch;rtt_j_r;rel_6;devval;build;x86_64;slc5;gcc43;offline;recperftest;details_all_bs;details_all_bs_log;look;sharka;validation;shifter;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;error;eventbookkeeper_p2;reconstruction'),(87100,'atlas','fpe in singleelectronstream_lowettaufilter','\r\n\r\n hi\r\n\r\n  there is a crash in dev rtt recexanatest/mc11_all \r\n\r\n   log file :\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_0/dev/build/x86_64-slc5-gcc43-opt/offline/recexanatest/mc11_all/mc11_all_log\r\n\r\n   \r\n  sharka ( validation shifter )\r\n\r\n  https://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation ','3245','5211','2011-09-25 17:06:56','5','duplicate','closed','reconstruction',';crash;recexanatest;mc11_all;file;http;atlasrtt.cern.ch;rtt_j_r;rel_0;build;x86_64;slc5;gcc43;offline;recexanatest;mc11_all;mc11_all_log;sharka;validation;shifter;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;singleelectronstream_lowettaufilter;reconstruction'),(87101,'atlas','muonstandalonemonmanager crashes in finalization (segfault)','\r\n  hi,\r\n\r\n   the crash is seen in devval rel_0\r\n\r\n  log file :\r\n\r\n  https://atlasrtt.cern.ch/rtt_j_r/rel_0/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_nocalo/data_7tev_nocalo_log\r\n\r\n  \r\n   please have a look,\r\n\r\n   sharka (validation shifter)\r\n\r\n  https://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n  ','3518','5211','2011-09-25 17:25:55','5','fixed','closed','reconstruction',';crash;devval;rel_0;file;http;atlasrtt.cern.ch;rtt_j_r;rel_0;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;data_7tev_nocalo;data_7tev_nocalo_log;look;sharka;validation;shifter;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;muonstandalonemonmanager;crash;finalization;segfault;reconstruction'),(87102,'atlas','eflowobjectcontainer_p4_eflowobjects_taumode is irreproducible','hi mark, \r\n\r\nthe size of eflowobjectcontainer_p4_eflowobjects_taumode is irreproducible. please fix it immediately or please remove the container from 17.0.x.y. \r\nthere is no tag difference between 17.0.x.y-val,rel_6 and 17.0.x.y,rel_0. \r\n+verbatim+\r\nget-tag-diff.py --ref=17.0.x.y-val,rel_6 --chk=17.0.x.y,rel_0\r\n::: setup reference env. [17.0.x.y-val,rel_6]...\r\n::: setup check env. [17.0.x.y,rel_0]...\r\n::: found [0] tags which are different\r\n-verbatim-\r\nbut there are differences in eflowobjectcontainer_p4_eflowobjects_taumode container sizes in tct. \r\n+verbatim+\r\ncomparetcts.py --nref=17.0.x.y-val --rref=rel_6 --nval=17.0.x.y --rval=rel_0\r\n\r\n================================================================================\r\n::: comparing pool files...\r\n ref : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_6/17.0.x.y-val/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_minbias0_collisions/myaod_minbias_0.aod.pool.root\r\n chk : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_0/17.0.x.y/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_minbias0_collisions/myaod_minbias_0.aod.pool.root\r\n--------------------------------------------------------------------------------\r\n::: comparing common content (mem-size)...\r\n[err]     1302.630 kb (ref) ==&gt;     1302.411 kb (chk) | eflowobjectcontainer_p4_eflowobjects_taumode\r\n================================================================================\r\n\r\n================================================================================\r\n::: comparing pool files...\r\n ref : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_6/17.0.x.y-val/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express5_collisions/myaod_express_5.aod.pool.root\r\n chk : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_0/17.0.x.y/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express5_collisions/myaod_express_5.aod.pool.root\r\n--------------------------------------------------------------------------------\r\n::: comparing common content (mem-size)...\r\n[err]     3407.266 kb (ref) ==&gt;     3407.484 kb (chk) | eflowobjectcontainer_p4_eflowobjects_taumode\r\n================================================================================\r\n\r\n================================================================================\r\n::: comparing pool files...\r\n ref : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_6/17.0.x.y-val/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_muons_collisions_withouthist/myaod_muons_2.aod.pool.root\r\n chk : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_0/17.0.x.y/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_muons_collisions_withouthist/myaod_muons_2.aod.pool.root\r\n--------------------------------------------------------------------------------\r\n::: comparing common content (mem-size)...\r\n[err]     3119.646 kb (ref) ==&gt;     3119.755 kb (chk) | eflowobjectcontainer_p4_eflowobjects_taumode\r\n================================================================================\r\n\r\n================================================================================\r\n::: comparing pool files...\r\n ref : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_6/17.0.x.y-val/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_muons_collisions_withouthist/myesd_muons_2.pool.root\r\n chk : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_0/17.0.x.y/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_muons_collisions_withouthist/myesd_muons_2.pool.root\r\n--------------------------------------------------------------------------------\r\n::: comparing common content (mem-size)...\r\n[err]     3685.434 kb (ref) ==&gt;     3685.562 kb (chk) | eflowobjectcontainer_p4_eflowobjects_taumode\r\n================================================================================\r\n\r\n================================================================================\r\n::: comparing pool files...\r\n ref : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_6/17.0.x.y-val/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express1_collisions/myesd_express_1.pool.root\r\n chk : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_0/17.0.x.y/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express1_collisions/myesd_express_1.pool.root\r\n--------------------------------------------------------------------------------\r\n::: comparing common content (mem-size)...\r\n[err]     3466.251 kb (ref) ==&gt;     3465.867 kb (chk) | eflowobjectcontainer_p4_eflowobjects_taumode\r\n================================================================================\r\n\r\n================================================================================\r\n::: comparing pool files...\r\n ref : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_6/17.0.x.y-val/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express2_collisions_with_3rd_muon_chain/myesd_express_2.pool.root\r\n chk : root://eosatlas//eos/atlas/atlascerngroupdisk/proj-sit/rtt/prod/tct/rel_0/17.0.x.y/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express2_collisions_with_3rd_muon_chain/myesd_express_2.pool.root\r\n--------------------------------------------------------------------------------\r\n::: comparing common content (mem-size)...\r\n[err]     3482.310 kb (ref) ==&gt;     3482.054 kb (chk) | eflowobjectcontainer_p4_eflowobjects_taumode\r\n================================================================================\r\n-verbatim-\r\n\r\ncheers, \r\nsusumu  ','1160','4045','2011-09-25 17:27:52','9','fixed','closed','eflow',';mark;size;eflowobjectcontainer_p4_eflowobjects_taumode;irreproducible;immediately;remove;container;difference;rel_6;rel_0;verbatim;diff.py;rel_6;rel_0;setup;reference;rel_6;setup;check;rel_0;different;verbatim;difference;eflowobjectcontainer_p4_eflowobjects_taumode;container;size;verbatim;comparetcts.py;nref;rref;rel_6;nval;rval;rel_0;compare;pool;file;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_6;slc5;gcc43;offline;tier0chaintest;run00183021_minbias0_collisions;myaod_minbias_0;aod.pool.root;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_0;slc5;gcc43;offline;tier0chaintest;run00183021_minbias0_collisions;myaod_minbias_0;aod.pool.root;compare;common;content;size;eflowobjectcontainer_p4_eflowobjects_taumode;compare;pool;file;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_6;slc5;gcc43;offline;tier0chaintest;run00183021_express5_collisions;myaod_express_5;aod.pool.root;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_0;slc5;gcc43;offline;tier0chaintest;run00183021_express5_collisions;myaod_express_5;aod.pool.root;compare;common;content;size;eflowobjectcontainer_p4_eflowobjects_taumode;compare;pool;file;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_6;slc5;gcc43;offline;tier0chaintest;run00183021_muons_collisions_withouthist;myaod_muons_2;aod.pool.root;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_0;slc5;gcc43;offline;tier0chaintest;run00183021_muons_collisions_withouthist;myaod_muons_2;aod.pool.root;compare;common;content;size;eflowobjectcontainer_p4_eflowobjects_taumode;compare;pool;file;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_6;slc5;gcc43;offline;tier0chaintest;run00183021_muons_collisions_withouthist;myesd_muons_2;pool.root;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_0;slc5;gcc43;offline;tier0chaintest;run00183021_muons_collisions_withouthist;myesd_muons_2;pool.root;compare;common;content;size;eflowobjectcontainer_p4_eflowobjects_taumode;compare;pool;file;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_6;slc5;gcc43;offline;tier0chaintest;run00183021_express1_collisions;myesd_express_1;pool.root;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_0;slc5;gcc43;offline;tier0chaintest;run00183021_express1_collisions;myesd_express_1;pool.root;compare;common;content;size;eflowobjectcontainer_p4_eflowobjects_taumode;compare;pool;file;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_6;slc5;gcc43;offline;tier0chaintest;run00183021_express2_collisions_with_3rd_muon_chain;myesd_express_2;pool.root;root;eosatla;atla;atlascerngroupdisk;proj;prod;rel_0;slc5;gcc43;offline;tier0chaintest;run00183021_express2_collisions_with_3rd_muon_chain;myesd_express_2;pool.root;compare;common;content;size;eflowobjectcontainer_p4_eflowobjects_taumode;verbatim;susumu;eflowobjectcontainer_p4_eflowobjects_taumode;irreproducible;eflow'),(87123,'atlas','many unprotected debug msgs in egdetailcontainercnv each event','\r\nduring a reco job, we are seeing many unprotected debug messages from egdetailcontainercnv each event which should either be protected or removed:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect debug output.\r\n\r\ncheers,\r\nthomas\r\n','3732','916','2011-09-26 14:14:12','5','fixed','closed','none',';reco;unprotected;debug;message;egdetailcontainercnv;event;protect;remove;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;debug;output;thoma;unprotected;debug;egdetailcontainercnv;event'),(87124,'atlas','many unprotected debug msgs in photoncontainercnv each event','\r\nduring a reco job, we are seeing many unprotected debug messages from photoncontainercnv each event which should either be protected or removed:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect debug output.\r\n\r\ncheers,\r\nthomas\r\n\r\nps. a priori the fix is just needed in dev.\r\n','3732','916','2011-09-26 14:16:29','5','fixed','closed','none',';reco;unprotected;debug;message;photoncontainercnv;event;protect;remove;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;debug;output;thoma;priori;just;need;unprotected;debug;photoncontainercnv;event'),(87126,'atlas','many unprotected verbose msgs in toolsvc.jetvtxtrackhelper each event','\r\nduring a reco job, we are seeing many unprotected verbose messages from toolsvc.jetvtxtrackhelper each event which should either be protected or removed:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect verbose output.\r\n\r\ncheers,\r\nthomas\r\n\r\nps. a priori the fix is just needed in dev.\r\n(','1693','916','2011-09-26 14:17:59','5','fixed','closed','none',';reco;unprotected;verbose;message;toolsvc.jetvtxtrackhelper;event;protect;remove;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;verbose;output;thoma;priori;just;need;unprotected;verbose;toolsvc.jetvtxtrackhelper;event'),(87127,'atlas','many unprotected debug msgs in toolsvc.softmuontag each event','\r\nduring a reco job, we are seeing many unprotected debug messages from toolsvc.softmuontag each event which should either be protected or removed:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect debug output.\r\n\r\ncheers,\r\nthomas\r\n\r\nps. a priori the fix is just needed in dev.\r\n','3896','916','2011-09-26 14:19:12','5','fixed','closed','none',';reco;unprotected;debug;message;toolsvc.softmuontag;event;protect;remove;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;debug;output;thoma;priori;just;need;unprotected;debug;toolsvc.softmuontag;event'),(87128,'atlas','many unprotected verbose msgs in toolsvc.muonisolationtool each event','\r\nduring a reco job, we are seeing many unprotected verbose messages from toolsvc.muonisolationtool each event which should either be protected or removed:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect verbose output.\r\n\r\ncheers,\r\nthomas\r\n\r\nps. a priori the fix is just needed in dev.\r\n','3518','916','2011-09-26 14:20:10','5','fixed','closed','muonidentification/muonisolationtool',';reco;unprotected;verbose;message;toolsvc.muonisolationtool;event;protect;remove;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;verbose;output;thoma;priori;just;need;unprotected;verbose;toolsvc.muonisolationtool;event;muonidentification;muonisolationtool'),(87129,'atlas','many unprotected verbose msgs in toolsvc.muoncombinepatterntool each event','\r\nduring a reco job, we are seeing many unprotected verbose messages from toolsvc.muoncombinepatterntool each event which should either be protected or removed:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect verbose output.\r\n\r\ncheers,\r\nthomas\r\n\r\nps. a priori the fix is just needed in dev.\r\n','1682','916','2011-09-26 14:20:46','5','fixed','closed','none',';reco;unprotected;verbose;message;toolsvc.muoncombinepatterntool;event;protect;remove;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;verbose;output;thoma;priori;just;need;unprotected;verbose;toolsvc.muoncombinepatterntool;event'),(87133,'atlas','many unprotected debug msgs in  toolsvc.egammaswtool.calorunclustercorrectionsegele... each event','during a reco job, we are seeing many unprotected debug messages from toolsvc.egammaswtool.calorunclustercorrectionsegele372.calhits_global_ele37eg2/debug and toolsvc.egammaswtool.calorunclustercorrectionsegele552.calhits_global_ele55eg2/debug each event which should either be protected or removed:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect debug output.\r\n\r\ncheers,\r\nthomas\r\n\r\nps. a priori the fix is just needed in dev.\r\n\r\npps. likely related (but i will open separate bug reports and post links here afterwards) are the unprotected debug messages in:\r\n\r\n  emtopo35clustermaker.calorunclustercorrectionstopo35gam35.removebad_gam35topo35/debug\r\nemtopo35clustermaker.calorunclustercorrectionstopo35gam35.layers_gam35topo35/debug\r\nemtopo35clustermaker.calorunclustercorrectionstopo35gam35.time_gam35topo35/debug\r\nemtopo35clustermaker.calorunclustercorrectionstopo35gam35.larupdate_gam35topo35/debug\r\nlarclustermaker.calorunclustercorrectionsele35.calhits_global_ele35/debug\r\n\r\n','3732','916','2011-09-26 14:31:10','5','duplicate','closed','none',';reco;unprotected;debug;message;toolsvc.egammaswtool.calorunclustercorrectionsegele372;calhits_global_ele37eg2;debug;toolsvc.egammaswtool.calorunclustercorrectionsegele552;calhits_global_ele55eg2;debug;event;protect;remove;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;debug;output;thoma;priori;just;need;likely;related;open;separate;report;post;link;unprotected;debug;message;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;removebad_gam35topo35;debug;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;layers_gam35topo35;debug;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;time_gam35topo35;debug;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;larupdate_gam35topo35;debug;larclustermaker.calorunclustercorrectionsele35;calhits_global_ele35;debug;unprotected;debug;toolsvc.egammaswtool.calorunclustercorrectionsegele;event'),(87134,'atlas','many unprotected debug msgs in  emtopo35clustermaker.calorunclustercorrectionstopo35gam35 each event','\r\nduring a reco job, we are seeing many unprotected debug messages each event, which should either be protected or removed, coming from:\r\n\r\nemtopo35clustermaker.calorunclustercorrectionstopo35gam35.removebad_gam35topo35/debug\r\nemtopo35clustermaker.calorunclustercorrectionstopo35gam35.layers_gam35topo35/debug\r\nemtopo35clustermaker.calorunclustercorrectionstopo35gam35.time_gam35topo35/debug\r\nemtopo35clustermaker.calorunclustercorrectionstopo35gam35.larupdate_gam35topo35/debug\r\n\r\nseen here:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/arch-mon/?test=msg/dev__x86_64-slc5-gcc43-dbg#plot_totalmsgsvcabusesduringashortjob\r\n\r\nthis is seen in dev (devval is also available on the page via the same menu). find more information on the link if you are not aware of what it means to protect debug output.\r\n\r\ncheers,\r\nthomas\r\n\r\nps. a priori the fix is just needed in dev.\r\n\r\npps. likely related (but i will open separate bug reports and post links here afterwards) are the unprotected debug messages in: \r\n toolsvc.egammaswtool.calorunclustercorrectionsegele372.calhits_global_ele37eg2/debug\r\ntoolsvc.egammaswtool.calorunclustercorrectionsegele552.calhits_global_ele55eg2/debug\r\nlarclustermaker.calorunclustercorrectionsele35.calhits_global_ele35/debug\r\n\r\n\r\n\r\n','3732','916','2011-09-26 14:34:11','5','duplicate','closed','none',';reco;unprotected;debug;message;event;protect;remove;come;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;removebad_gam35topo35;debug;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;layers_gam35topo35;debug;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;time_gam35topo35;debug;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;larupdate_gam35topo35;debug;http;atla;pmb.web.cern.ch;atla;arch;test;dev__x86_64;slc5;gcc43;plot_totalmsgsvcabusesduringashortjob;devval;available;page;menu;information;link;aware;means;protect;debug;output;thoma;priori;just;need;likely;related;open;separate;report;post;link;unprotected;debug;message;toolsvc.egammaswtool.calorunclustercorrectionsegele372;calhits_global_ele37eg2;debug;toolsvc.egammaswtool.calorunclustercorrectionsegele552;calhits_global_ele55eg2;debug;larclustermaker.calorunclustercorrectionsele35;calhits_global_ele35;debug;unprotected;debug;emtopo35clustermaker.calorunclustercorrectionstopo35gam35;event'),(87187,'atlas','rtt mc10_rdotobs 		 transform output file corrupted','hi in mc10_rdotobs\r\n		transform output file corrupted\r\n\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/share/bin/reco_trf.py inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/mc10_7tev.105200.t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00/rdo.197091._000001.pool.root.1 outputbsfile=t1_mcatnlo_jimmy.data autoconfiguration=everything maxevents=300\r\nexitcode=41\r\nerrorcode=64140 (trf_outfile_corrup)\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_outfile_corrup (64140) transform output file corrupted\r\n  message=output file t1_mcatnlo_jimmy.data failed validation [8]. file corrupt. argument outputbsfile\r\n  severity=fatal\r\n  producer=rdotobs\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/trf.py&quot;, line 1858, in dopostrunactions\r\n              cmd.postrunaction()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1197, in postrunaction\r\n              self.validatefile()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1184, in validatefile\r\n              self._filetype.validatefile( self, **self.__validationdict )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/16.6.x.y/atlasproduction/rel_2/installarea/python/pyjobtransformscore/trfutil.py&quot;, line 1114, in validatefile\r\n              raise transformvalidationerror( fname, \'failed validation [%s]. file corrupt. argument %s\' % ( sc, argname ) )\r\nathacronym=successful run\r\nathcode=0\r\nrundiruseddisk=646912982\r\nrundiravailabledisk=192940535808\r\nworkdir=/pool/lsf/rtt/172969669\r\nathcode=0\r\nrundiravailabledisk=192940773376\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=646687578\r\natlasrelease=rel_2\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\nsee log\r\nhttps://atlasrtt.cern.ch/rtt/rel_2/16.6.x.y/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/mc10_rdotobs/mc10_rdotobs_log\r\n\r\ncheers\r\ndaniel','100','5211','2011-09-27 14:14:25','5','wont fix','closed','reconstruction',';mc10_rdotobs;transform;output;file;corrupt;report;produce;recotrf;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;reco_trf.py;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;mc10_7tev;t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00;pool.root;outputbsfile;t1_mcatnlo_jimmy.data;autoconfiguration;maxevent;exitcode;errorcode;trf_outfile_corrup;fatal;error;warning;errorinfo;acronym;trf_outfile_corrup;transform;output;file;corrupt;message;output;file;t1_mcatnlo_jimmy.data;fail;validation;file;corrupt;argument;outputbsfile;severity;fatal;producer;rdotob;backtrace;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;trf.py;line;dopostrunaction;cmd.postrunaction;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;basic_trfarg.py;line;postrunaction;self.validatefile;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;basic_trfarg.py;line;validatefile;self;self;self;file;atla;software;build;nightly;atlasproduction;rel_2;installarea;python;pyjobtransformscore;trfutil.py;line;validatefile;raise;transformvalidationerror;fname;fail;validation;file;corrupt;argument;argname;athacronym;successful;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_2;write;jobinfo_recotrf.xml;file;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;recjobtransformtest;mc10_rdotobs;mc10_rdotobs_log;daniel;mc10_rdotobs;transform;output;file;corrupt;reconstruction'),(87189,'atlas','mis-configuration of csc_physval_mon','hi in 	recoonlytopvalid\r\n		holder can\'t be done, no predefined storage found for clid: 1271751257\r\n\r\n===== job report produced by csc_physval_mon version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y/atlasproduction/rel_2/installarea/share/bin/csc_physval_mon_trf.py inputaodfile=/afs/cern.ch/atlas/project/rtt/prod/results/fct/chainstore/17.0.x.y/i686-slc5-gcc43-opt/atlasproduction/tools/fullchaintests/recoonlytopchainjob/toprecolong_pll1.aod.pool.root,/afs/cern.ch/atlas/project/rtt/prod/results/fct/chainstore/17.0.x.y/i686-slc5-gcc43-opt/atlasproduction/tools/fullchaintests/recoonlytopchainjob/toprecolong_pll2.aod.pool.root histogramfile=toprecolong.histos.pool.root maxevents=1000 skipevents=0 geometryversion=atlas-geo-16-00-00 conditionstag=oflcond-sdr-bs7t-04-13 triggerconfig=on dbrelease=/afs/cern.ch/atlas/www/groups/database/pacman4/dbrelease/dbrelease-pcache-current.tar.gz --athenaopts=--stdcmalloc\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (248 times)\r\nfatals=0\r\nerrors=2\r\nwarnings=9\r\nerrorinfo (count=124)\r\n  acronym=trf_unknown (69999)\r\n  message=createholder: holder can\'t be done, no predefined storage found for clid: 1271751257\r\n  severity=error\r\n  who=toolsvc.trig::t...\r\n  producer=csc_physval_mon\r\n  stage=execute\r\nerrorinfo (count=124)\r\n  acronym=trf_unknown (69999)\r\n  message=deserialize: could not create holder for clid 1271751257 this probably means that the package holding the class for this clid was not compiled against the trignavigation package in use!\r\n  severity=error\r\n  who=toolsvc.trig::t...\r\n  producer=csc_physval_mon\r\n  stage=execute\r\nworkdir=/pool/lsf/rtt/172959190\r\nathcode=0\r\nrundiravailabledisk=737239633920\r\ndbrelease=17.1.1\r\nathacronym=successful run\r\nrundiruseddisk=1308375052\r\natlasrelease=rel_2\r\nwriting jobinfo_csc_physval_mon.xml file...\r\n\r\n\r\nsee log\r\nhttps://atlasrtt.cern.ch/fct/rel_2/17.0.x.y/build/i686-slc5-gcc43-opt/offline/fullchaintests/recoonlytopvalid/recoonlytopvalid_log\r\n\r\ncheers\r\ndaniel','1160','5211','2011-09-27 14:33:20','5','wont fix','closed','reconstruction',';recoonlytopvalid;holder;can\'t;predefine;storage;clid;report;produce;csc_physval_mon;version;unknown;command;atla;software;build;nightly;atlasproduction;rel_2;installarea;share;csc_physval_mon_trf.py;inputaodfile;atla;project;prod;result;chainstore;slc5;gcc43;atlasproduction;tool;fullchaintest;recoonlytopchainjob;toprecolong_pll1;aod.pool.root;atla;project;prod;result;chainstore;slc5;gcc43;atlasproduction;tool;fullchaintest;recoonlytopchainjob;toprecolong_pll2;aod.pool.root;histogramfile;toprecolong.histos.pool.root;maxevent;skipevent;geometryversion;atla;conditionstag;oflcond;bs7t;triggerconfig;dbrelease;atla;group;database;pacman4;dbrelease;dbrelease;pcache;current.tar.gz;athenaopt;stdcmalloc;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;count;acronym;trf_unknown;message;createholder;holder;can\'t;predefine;storage;clid;severity;error;toolsvc.trig;producer;csc_physval_mon;stage;execute;errorinfo;count;acronym;trf_unknown;message;deserialize;create;holder;clid;probably;means;package;hold;class;clid;compile;trignavigation;package;severity;error;toolsvc.trig;producer;csc_physval_mon;stage;execute;workdir;pool;athcode;rundiravailabledisk;dbrelease;17.1.1;athacronym;successful;rundiruseddisk;atlasrelease;rel_2;write;jobinfo_csc_physval_mon.xml;file;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;fullchaintest;recoonlytopvalid;recoonlytopvalid_log;daniel;configuration;csc_physval_mon;reconstruction'),(87523,'atlas','unitialised variables in jettagmonitoring','\r\ni noticed that rerunning the same job twice gave difference in some jet tag monitoring histograms. i think the reason is the incomplete resetting in:\r\n\r\nhttps://svnweb.cern.ch/trac/atlasoff/browser/physicsanalysis/jettagging/jettagmonitoring/trunk/jettagmonitoring/runsummary.h#l107\r\n\r\nfor instance jfcnn_pos and jfcnn_neg are not reset here, and thus never initialised (there might be more of course so please doublecheck all variables).\r\n\r\ncheers,\r\nthomas','3402','916','2011-10-07 10:11:26','5','fixed','closed','none',';notice;rerun;twice;give;difference;monitoring;histogram;think;reason;incomplete;reset;http;svnweb.cern.ch;trac;atlasoff;browser;physicsanalysis;jettagging;jettagmonitoring;trunk;jettagmonitoring;runsummary.h;instance;jfcnn_pos;jfcnn_neg;reset;initialise;course;doublecheck;variable;thoma;unitialised;variable;jettagmonitoring'),(87195,'atlas','keyerror in esd_memleakana, 17.x.0, input file not found','rtt: keyerror in esd_memleakana, 17.x.0(rel_2)\r\n\r\nlogfile snippet:\r\n-----------------\r\ntraceback (most recent call last):\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_2/installarea/python/recexconfig/inputfilepeeker.py&quot;, line 48, in _setup\r\n    fi = athfile.fopen(infile)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_2/installarea/python/pyutils/athfile/__init__.py&quot;, line 160, in fopen\r\n    return self.server.fopen(fnames, evtmax)\r\n  file &quot;&lt;string&gt;&quot;, line 2, in fopen\r\n  file &quot;/afs/cern.ch/sw/lcg/external/pytools/1.5_python2.6/x86_64-slc5-gcc43-opt/lib/python2.6/site-packages/multiprocessing-2.6.2.1-py2.6-linux-x86_64.egg/multiprocessing/managers.py&quot;, line 743, in _callmethod\r\n    raise convert_to_error(kind, result)\r\nkeyerror: -11\r\n\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_2/devval/build/x86_64-slc5-gcc43-opt/offline/recexanatest/esd_memleakana/esd_memleakana_log\r\n\r\ncommand:\r\n---------\r\njp.athenacommonflags.evtmax=-1; rec.docbnt=false ;jp.athenacommonflags.filesinput=[\'/afs/cern.ch/atlas/project/rig/referencefiles/mc/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e835_s1256_s1258_d566_tid464353_00/rdo.464353._000001.pool.root.1\']\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','1658','5211','2011-09-27 17:23:12','5','fixed','closed','none',';keyerror;esd_memleakana;rel_2;logfile;snippet;traceback;recent;file;atla;software;build;nightly;devval;atlasevent;rel_2;installarea;python;recexconfig;inputfilepeeker.py;line;athfile.fopen;infile;file;atla;software;build;nightly;devval;atlascore;rel_2;installarea;python;pyutil;athfile;line;fopen;return;self.server.fopen;fname;evtmax;file;line;fopen;file;external;pytool;1.5_python2.6;x86_64;slc5;gcc43;python2.6;site;package;multiprocess;2.6.2.1;py2.6;linux;x86_64;multiprocess;managers.py;line;raise;convert_to_error;kind;result;keyerror;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_2;devval;build;x86_64;slc5;gcc43;offline;recexanatest;esd_memleakana;esd_memleakana_log;command;jp.athenacommonflags.evtmax;rec.docbnt;false;jp.athenacommonflags.filesinput;atla;project;referencefile;valid1.105200;t1_mcatnlo_jimmy.digit.rdo.e835_s1256_s1258_d566_tid464353_00;pool.root;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;keyerror;esd_memleakana;input;file'),(87196,'atlas','trig_rdo error in rawtoesd part','dear experts,\r\n\r\nin todays 17.0.x.y (x86_64-slc5-gcc43-opt) rel_2 the trig_rdo job crashed with this error:\r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_2/17.0.x.y/build/x86_64-slc5-gcc43-opt/offline/recperftests/trig_rdo/trig_rdo_log\r\n\r\npmonsd =========================== semi-detailed perfmon info / end ===========================\r\nperfmonsvc: warning problem on python side during finalize() !!\r\ntraceback (most recent call last):\r\n  file &quot;/afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/python/perfmoncomps/pyperfmon.py&quot;, line 231, in finalize\r\n    _msg = self.msg\r\n  file &quot;/afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/python/perfmoncomps/pyperfmon.py&quot;, line 167, in msg\r\n    import athenacommon.logging as l\r\nimporterror: no module named athenacommon.logging\r\nexception runtimeerror: \'(file &quot;&quot;, line 0) python exception (c++ exception)\' in &lt;bound method appmgr.__del__ of &lt;gaudipython.bindings.appmgr object at 0x1ecb9dd0&gt;&gt; ignored\r\ncoralapplication info delete the cool coralapplication...\r\ncoralapplication info delete the cool database service\r\nraldatabasesvc info delete the raldatabasesvc...\r\nraldatabasesvc info purge the connection pool\r\nraldatabasesvc info reset the ics pointer\r\nraldatabasesvc info delete the raldatabasesvc... done\r\ncoralapplication info delete the coral connection service\r\ncoralapplication info delete the cool coralapplication... done\r\ncoral/services/connectionservice info deleting the connectionpool\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code 30.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/share/atlas_error_categories.db\r\ninfo: read 53 error patterns from file /afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: tue sep 27 04:47:56 2011 cest, vmem: 281.75 mb, rss: 95.78125 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_2 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info athena finalise()...\r\npy:rawtoesd          info done scanning 73117 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 6\r\npy:rawtoesd          info    warnings: 470\r\npy:rawtoesd          info    errors  : 4\r\npy:rawtoesd          info    fatals  : 5\r\nathenalogchecker stop [rawtoesd log] wall-time: tue sep 27 04:48:04 2011 cest, vmem: 281.88671875 mb, rss: 95.83203125 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 07 secs, cpu-time: 00 hrs 00 mins 07 secs, vmem: 0.13671875 mb, rss: 0.05078125 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: tue sep 27 04:48:04 2011 cest, vmem: 281.88671875 mb, rss: 95.87109375 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 48 mins 10 secs, cpu-time: 00 hrs 46 mins 25 secs, vmem: 0.26953125 mb, rss: 0.34765625 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.4/atlasreconstruction/17.0.4/installarea/share/bin/rawtoesd_trf.py inputrdofile=/afs/cern.ch/atlas/project/rig/referencefiles/rttinputfiles/rdo.039414._00001.pool.root.1 maxevents=250 autoconfiguration=everything --athenaopts=--stdcmalloc preexec=from@perfmoncomps.perfmonflags@import@jobproperties@as@pmjp,,pmjp.perfmonflags.domallocmonitoring=true,,pmjp.perfmonflags.dopostprocessing=true,,pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (9 times)\r\nfatals=5\r\nerrors=4\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=out of memory: saving summary ...\r\n  severity=fatal\r\n  who=athenasummarysvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=detailedtracktruthmaker\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=std::bad_alloc\r\n  severity=error\r\n  who=detailedtracktruthmaker\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=athalgseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=std::exception\r\n  severity=error\r\n  who=athalgseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=athmasterseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=std::exception\r\n  severity=error\r\n  who=athmasterseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=standard std::exception is caught\r\n  severity=fatal\r\n  who=trigconf2coolsyncsvc\r\n  producer=rawtoesd\r\n  stage=finalise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=python exception\r\n  severity=error\r\n  who=trigconf2coolsyncsvc\r\n  producer=rawtoesd\r\n  stage=finalise\r\nworkdir=/pool/lsf/rtt/172923852\r\nathcode=30\r\nrundiravailabledisk=138985598976\r\ndbrelease=\r\nathacronym=unknown\r\nrundiruseddisk=19115400\r\natlasrelease=rel_2\r\nwriting jobinfo_rawtoesd.xml file...','100','5211','2011-09-27 17:27:23','5','none','closed','none',';dear;expert;today;x86_64;slc5;gcc43;rel_2;trig_rdo;crash;error;http;atlasrtt.cern.ch;rel_2;build;x86_64;slc5;gcc43;offline;recperftest;trig_rdo;trig_rdo_log;pmonsd;semus;detailed;perfmon;info;perfmonsvc;warn;problem;python;finalize;traceback;recent;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;python;perfmoncomp;pyperfmon.py;line;finalize;self.msg;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;python;perfmoncomp;pyperfmon.py;line;import;athenacommon.logging;importerror;module;name;athenacommon.logging;exception;runtimeerror;file;line;python;exception;exception;ignore;coralapplication;info;delete;cool;coralapplication;coralapplication;info;delete;cool;database;service;raldatabasesvc;info;delete;raldatabasesvc;raldatabasesvc;info;purge;connection;pool;raldatabasesvc;info;reset;pointer;raldatabasesvc;info;delete;raldatabasesvc;coralapplication;info;delete;coral;connection;service;coralapplication;info;delete;cool;coralapplication;coral;service;connectionservice;info;delete;connectionpool;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;info;read;error;category;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_2;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;athena;finalise;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;release;17.0.4;atlasreconstruction;17.0.4;installarea;share;rawtoesd_trf.py;inputrdofile;atla;project;referencefile;rttinputfile;pool.root;maxevent;autoconfiguration;athenaopt;stdcmalloc;preexec;perfmoncomps.perfmonflags;import;jobproperty;pmjp;pmjp.perfmonflags.domallocmonitoring;true;pmjp.perfmonflags.dopostprocessing;true;pmjp.perfmonflags.dosemidetailedmonitoringfullprint;true;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;memory;save;summary;severity;fatal;athenasummarysvc;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;detailedtracktruthmaker;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;bad_alloc;severity;error;detailedtracktruthmaker;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;athalgseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;exception;severity;error;athalgseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;athmasterseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;exception;severity;error;athmasterseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;standard;exception;catch;severity;fatal;trigconf2coolsyncsvc;producer;rawtoesd;stage;finalise;errorinfo;acronym;trf_unknown;message;python;exception;severity;error;trigconf2coolsyncsvc;producer;rawtoesd;stage;finalise;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;rundiruseddisk;atlasrelease;rel_2;write;jobinfo_rawtoesd.xml;file;trig_rdo;error;rawtoesd'),(87198,'atlas','in mc10_all_nofpe:error: could not load triggerrelease/jobofragment_triggercbnt.py','rtt:in mc10_all_nofpe:error: could not load triggerrelease/jobofragment_triggercbnt.py, 17.x.0-val(rel_2) \r\n\r\nlogfile snippet:\r\n-----------------\r\ntraceback (most recent call last):\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_2/installarea/python/athenacommon/resilience.py&quot;, line 25, in protect\r\n    return fn (*args, **kw)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_2/installarea/python/athenacommon/include.py&quot;, line 168, in __call__\r\n    execfile( name, self._workspace, self._workspace )\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlastrigger/rel_2/installarea/joboptions/triggerrelease/jobofragment_triggercbnt.py&quot;, line 31, in &lt;module&gt;\r\n    include(&quot;trigntcalo/jobofragment_trigntcalo.py&quot;)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_2/installarea/python/athenacommon/include.py&quot;, line 128, in __call__\r\n    raise includeerror( \'include file %s can not be found\' % fn )\r\nincludeerror: include file trigntcalo/jobofragment_trigntcalo.py can not be found\r\n\r\nerror: could not load triggerrelease/jobofragment_triggercbnt.py\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_2/devval/build/x86_64-slc5-gcc43-opt/offline/recexanatest/mc10_all_nofpe/mc10_all_nofpe_log\r\n\r\ncommand:\r\n---------\r\n\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','314','5211','2011-09-27 17:33:57','5','fixed','closed','none',';rtt:in;mc10_all_nofpe:error;load;triggerrelease;jobofragment_triggercbnt.py;rel_2;logfile;snippet;traceback;recent;file;atla;software;build;nightly;devval;atlascore;rel_2;installarea;python;athenacommon;resilience.py;line;protect;return;file;atla;software;build;nightly;devval;atlascore;rel_2;installarea;python;athenacommon;include.py;line;execfile;self;self;file;atla;software;build;nightly;devval;atlastrigger;rel_2;installarea;joboption;triggerrelease;jobofragment_triggercbnt.py;line;include;trigntcalo;jobofragment_trigntcalo.py;file;atla;software;build;nightly;devval;atlascore;rel_2;installarea;python;athenacommon;include.py;line;raise;includeerror;include;file;includeerror;include;file;trigntcalo;jobofragment_trigntcalo.py;error;load;triggerrelease;jobofragment_triggercbnt.py;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_2;devval;build;x86_64;slc5;gcc43;offline;recexanatest;mc10_all_nofpe;mc10_all_nofpe_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;mc10_all_nofpe:error;load;triggerrelease;jobofragment_triggercbnt.py'),(87199,'atlas','error in cosmis_allstream','dear expert,\r\n\r\ntodays cosmis_allstream 17.x.0 (i686-slc5-gcc43-opt) rel_2 was crashed with the following error:\r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_2/dev/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/cosmis_allstream/cosmis_allstream_log\r\n\r\n=====================================================\r\npy:athena            info leaving with code 65: &quot;failure in an algorithm execute&quot;\r\ncoralapplication info delete the cool coralapplication...\r\ncoralapplication info delete the cool database service\r\nraldatabasesvc info delete the raldatabasesvc...\r\nraldatabasesvc info purge the connection pool\r\nraldatabasesvc info reset the ics pointer\r\nraldatabasesvc info delete the raldatabasesvc... done\r\ncoralapplication info delete the coral connection service\r\ncoralapplication info delete the cool coralapplication... done\r\ncoralapplication info delete the cool coralapplication...\r\ncoralapplication info delete the cool database service\r\nraldatabasesvc info delete the raldatabasesvc...\r\nraldatabasesvc info purge the connection pool\r\nraldatabasesvc info reset the ics pointer\r\nraldatabasesvc info delete the raldatabasesvc... done\r\ncoralapplication info delete the coral connection service\r\ncoralapplication info delete the cool coralapplication... done\r\ncoral/services/connectionservice info deleting the connectionpool\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code 65.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_2/installarea/share/atlas_error_categories.db\r\ninfo: read 53 error patterns from file /afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_2/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_2/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: tue sep 27 16:31:33 2011 cest, vmem: 107.59375 mb, rss: 67.73828125 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_2 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info athena finalise()...\r\npy:rawtoesd          info done scanning 18561 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 14\r\npy:rawtoesd          info    warnings: 319\r\npy:rawtoesd          info    errors  : 49\r\npy:rawtoesd          info    fatals  : 4\r\nathenalogchecker stop [rawtoesd log] wall-time: tue sep 27 16:31:35 2011 cest, vmem: 107.59375 mb, rss: 67.7890625 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 01 secs, cpu-time: 00 hrs 00 mins 01 secs, vmem: 0.0 mb, rss: 0.05078125 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: tue sep 27 16:31:35 2011 cest, vmem: 107.59375 mb, rss: 67.8125 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 09 mins 18 secs, cpu-time: 00 hrs 03 mins 58 secs, vmem: 0.14453125 mb, rss: 0.3515625 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_2/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputbsfile=/afs/cern.ch/user/c/cote/gencomm/nightlytestinput/data09_cos.00135664.physics_idcosmic.daq.raw._lb0000._sfo-2._0001.data maxevents=300 trigstream=idcosmic autoconfiguration=everything outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (53 times)\r\nfatals=4\r\nerrors=10\r\nwarnings=10\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: tue sep 27 16:31:35 2011 cest, vmem: 107.7578125 mb, rss: 67.984375 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 00 hrs 09 mins 18 secs, cpu-time: 00 hrs 03 mins 58 secs, vmem: 0.30859375 mb, rss: 0.546875 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_2/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputbsfile=/afs/cern.ch/user/c/cote/gencomm/nightlytestinput/data09_cos.00135664.physics_idcosmic.daq.raw._lb0000._sfo-2._0001.data maxevents=300 trigstream=idcosmic autoconfiguration=everything outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (53 times)\r\nfatals=4\r\nerrors=10\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot find /tdaq/olc/calibrations in detectorstore\r\n  severity=error\r\n  who=toolsvc.lumiblockmutool\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot find /tdaq/olc/lhc/fillparams in detectorstore\r\n  severity=error\r\n  who=toolsvc.lumiblockmutool\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot find /tdaq/olc/bunchlumis in detectorstore\r\n  severity=error\r\n  who=toolsvc.lumiblockmutool\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=incorrect object/count: indet::pixelgangedclusterambiguities_splitclusterambiguitymap, 3 should be: 4\r\n  severity=error\r\n  who=streamesd\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo (count=2)\r\n  acronym=trf_unknown (69999)\r\n  message=pool commitandhold failed 0xe2a10e8\r\n  severity=error\r\n  who=poolsvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo (count=2)\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed to commitandhold outputstream.\r\n  severity=error\r\n  who=athenapoolcnvsvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=unable to commit output myesd.pool.root\r\n  severity=error\r\n  who=streamesd.streamesdtool\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed.\r\n  severity=fatal\r\n  who=streamesd\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=terminating event processing loop due to errors\r\n  severity=error\r\n  who=athenaeventloopmgr\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=athenapoolconverter createrep failed to get token, key = streamesd\r\n  severity=error\r\n  who=athenapoolconverter\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=could not create rep for dataobject (clid/key):167728019 streamesd\r\n  severity=error\r\n  who=streamesd_fh.streamesd_fhtool\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=streamobjects failed.\r\n  severity=fatal\r\n  who=streamesd_fh\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed.\r\n  severity=fatal\r\n  who=streamesd_fh\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot disconnect stream: 1\r\n  severity=fatal\r\n  who=poolsvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nathacronym=failure in an algorithm execute\r\nathcode=65\r\nrundiruseddisk=9422845\r\nrundiravailabledisk=189744144384\r\nworkdir=/pool/lsf/rtt/173020222\r\nathcode=65\r\nrundiravailabledisk=189744369664\r\ndbrelease=\r\nathacronym=failure in an algorithm execute\r\nrundiruseddisk=9198791\r\natlasrelease=rel_2\r\nwriting jobinfo_recotrf.xml file...\r\n\r\ncheers michael','100','5211','2011-09-27 17:40:29','5','fixed','closed','none',';dear;expert;today;cosmis_allstream;slc5;gcc43;rel_2;crash;follow;error;http;atlasrtt.cern.ch;rel_2;build;slc5;gcc43;offline;recjobtransformtest;cosmis_allstream;cosmis_allstream_log;py:athena;info;leave;code;failure;algorithm;execute;coralapplication;info;delete;cool;coralapplication;coralapplication;info;delete;cool;database;service;raldatabasesvc;info;delete;raldatabasesvc;raldatabasesvc;info;purge;connection;pool;raldatabasesvc;info;reset;pointer;raldatabasesvc;info;delete;raldatabasesvc;coralapplication;info;delete;coral;connection;service;coralapplication;info;delete;cool;coralapplication;coralapplication;info;delete;cool;coralapplication;coralapplication;info;delete;cool;database;service;raldatabasesvc;info;delete;raldatabasesvc;raldatabasesvc;info;purge;connection;pool;raldatabasesvc;info;reset;pointer;raldatabasesvc;info;delete;raldatabasesvc;coralapplication;info;delete;coral;connection;service;coralapplication;info;delete;cool;coralapplication;coral;service;connectionservice;info;delete;connectionpool;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;info;read;error;category;file;atla;software;build;nightly;atlascore;rel_2;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;build;nightly;atlascore;rel_2;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;build;nightly;atlascore;rel_2;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_2;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;athena;finalise;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasreconstruction;rel_2;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;user;cote;gencomm;nightlytestinput;data09_cos;physics_idcosmic.daq.raw;datum;maxevent;trigstream;idcosmic;autoconfiguration;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;atlasreconstruction;rel_2;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;user;cote;gencomm;nightlytestinput;data09_cos;physics_idcosmic.daq.raw;datum;maxevent;trigstream;idcosmic;autoconfiguration;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;tdaq;calibration;detectorstore;severity;error;toolsvc.lumiblockmutool;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;tdaq;fillparam;detectorstore;severity;error;toolsvc.lumiblockmutool;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;tdaq;bunchlumi;detectorstore;severity;error;toolsvc.lumiblockmutool;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;incorrect;object;count;indet;pixelgangedclusterambiguities_splitclusterambiguitymap;severity;error;streamesd;producer;rawtoesd;stage;execute;errorinfo;count;acronym;trf_unknown;message;pool;commitandhold;fail;severity;error;poolsvc;producer;rawtoesd;stage;execute;errorinfo;count;acronym;trf_unknown;message;commitoutput;fail;commitandhold;outputstream;severity;error;athenapoolcnvsvc;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;unable;commit;output;myesd.pool.root;severity;error;streamesd.streamesdtool;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;commitoutput;fail;severity;fatal;streamesd;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;terminate;event;processing;loop;error;severity;error;athenaeventloopmgr;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;athenapoolconverter;createrep;fail;token;streamesd;severity;error;athenapoolconverter;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;create;dataobject;clid;streamesd;severity;error;streamesd_fh.streamesd_fhtool;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;streamobject;fail;severity;fatal;streamesd_fh;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;commitoutput;fail;severity;fatal;streamesd_fh;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;disconnect;stream;severity;fatal;poolsvc;producer;rawtoesd;stage;execute;athacronym;failure;algorithm;execute;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;failure;algorithm;execute;rundiruseddisk;atlasrelease;rel_2;write;jobinfo_recotrf.xml;file;michael;error;cosmis_allstream'),(87203,'atlas','seg fault in data_heavy_ion stream of 17.x.0-val(rel_2)','rtt:in data_heavy_ion:segmentation fault, 17.x.0-val(rel_2)\r\n\r\nlogfile snippet:\r\n-----------------\r\ncore dump from coredumpsvc on lxb8768.cern.ch at tue sep 27 09:51:38 2011\r\n\r\ncaught signal 11(segmentation fault). details: \r\n  errno = 0, code = 1 (address not mapped to object)\r\n  pid   = 382552, uid = 0\r\n  value = (418100144, 0x2b0418ebb3b0)\r\n  vmem = 954.508 mb\r\n  rss  = 548.965 mb\r\n  total-ram = 15925.7 mb\r\n  free-ram  = 2596.71 mb\r\n  buffer-ram= 177.777 mb\r\n  total-swap= 20481.2 mb\r\n  free-swap = 20382.1 mb\r\n  addr  = 0x5d658\r\n\r\nevent counter: 1\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [152166,451943,206,1269948495:250508165,1,0x1fffffffffff7]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent algorithm: streamaod\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   streamaod\r\n\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_2/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/data_heavy_ion/data_heavy_ion_log\r\n\r\ncommand:\r\n---------\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-09-27 21:17:46','5','none','closed','none',';rtt:in;data_heavy_ion:segmentation;fault;rel_2;logfile;snippet;core;dump;coredumpsvc;lxb8768;catch;signal;segmentation;fault;detail;errno;code;address;object;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;algorithm;streamaod;algorithm;stack;athmasterseq;athalgseq;streamaod;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_2;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;data_heavy_ion;data_heavy_ion_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;fault;data_heavy_ion;stream;rel_2'),(87204,'atlas','mc10_stdcmalloc_physvalmon: transform unknown exceptions','rtt:in mc10_stdcmalloc_physvalmon: transform unknown exceptions, 17.x.0-val(rel_2) \r\n\r\nlogfile snippet: \r\n----------------- \r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_2/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputesdfile=t1_mcatnlo_jimmy_stdm.esd.pool.root outputhist_physvalmonfile=pvm_std.root maxevents=-1 autoconfiguration=everything --athenaopts=--stdcmalloc\r\nexitcode=34\r\nerrorcode=63400 (trf_exc)\r\nerrorcategory=trf_exc,	63400,	transform unknown exceptions\r\nfatals=1\r\nerrors=0\r\nwarnings=0\r\nerrorinfo\r\n  acronym=trf_exc (63400) transform unknown exceptions\r\n  message=keyerror: (\'attempting to add a report (task esdtoaod) with missing athcode/athacronym.\',)\r\n  severity=fatal\r\n  producer=reco_trf\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_2/installarea/python/pyjobtransformscore/trf.py&quot;, line 2189, in execute\r\n              self._jobreport.addreport( self.runjob(), \'merge\' )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_2/installarea/share/bin/reco_trf.py&quot;, line 162, in runjob\r\n              report.addreport( reportaod )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_2/installarea/python/pyjobtransformscore/jobreport.py&quot;, line 808, in addreport\r\n              raise keyerror( \'attempting to add a report (task %s) with missing athcode/athacronym.\' % report._task.taskname() )\r\nrundiruseddisk=115568\r\nrundiravailabledisk=221728669696\r\n\r\nlogfile: \r\n--------- \r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_2/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/mc10_stdcmalloc_physvalmon/mc10_stdcmalloc_physvalmon_log\r\n\r\ncommand: \r\n--------- \r\n/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_2/installarea/share/bin/reco_trf.py --uploadtoami=1.0\r\n\r\nplease consult: \r\n---------------- \r\nhttps://twiki.cern.ch/twiki/bin/vie... \r\ncheers, \r\nrobert (valshift)','100','5211','2011-09-27 21:38:11','5','duplicate','closed','none',';rtt:in;mc10_stdcmalloc_physvalmon;transform;unknown;exception;rel_2;logfile;snippet;command;atla;software;build;nightly;devval;atlasreconstruction;rel_2;installarea;share;reco_trf.py;uploadtoamus;inputesdfile;t1_mcatnlo_jimmy_stdm.esd.pool.root;outputhist_physvalmonfile;pvm_std.root;maxevent;autoconfiguration;athenaopt;stdcmalloc;exitcode;errorcode;trf_exc;errorcategory;trf_exc;transform;unknown;exception;fatal;error;warning;errorinfo;acronym;trf_exc;transform;unknown;exception;message;keyerror;attempt;report;task;esdtoaod;miss;athcode;athacronym;severity;fatal;producer;reco_trf;backtrace;file;atla;software;build;nightly;devval;atlascore;rel_2;installarea;python;pyjobtransformscore;trf.py;line;execute;self;self.runjob;merge;file;atla;software;build;nightly;devval;atlasreconstruction;rel_2;installarea;share;reco_trf.py;line;runjob;report.addreport;reportaod;file;atla;software;build;nightly;devval;atlascore;rel_2;installarea;python;pyjobtransformscore;jobreport.py;line;addreport;raise;keyerror;attempt;report;task;miss;athcode;athacronym;report;rundiruseddisk;rundiravailabledisk;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_2;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;mc10_stdcmalloc_physvalmon;mc10_stdcmalloc_physvalmon_log;command;atla;software;build;nightly;devval;atlasreconstruction;rel_2;installarea;share;reco_trf.py;uploadtoamus;consult;http;twiki.cern.ch;twikus;robert;valshift;mc10_stdcmalloc_physvalmon;transform;unknown;exception'),(87205,'atlas','cannot create tool bytestreammetadatatool (rtt esd_notrig_memleak)','rtt:in esd_notrig_memleak, 17.x.0(rel_2) \r\nlogfile snippet: \r\n----------------- \r\ntoolsvc                                             error cannot create tool bytestreammetadatatool (no factory found)\r\nmetadatasvc                                         fatal cannot get metadatatools = publictoolhandlearray([\'iovdbmetadatatool\',\'bytestreammetadatatool\'])\r\nservicemanager                                      error unable to initialize service &quot;metadatasvc&quot;\r\nproxyprovidersvc                                    error servicelocatorhelper::service: can not locate service metadatasvc/metadatasvc\r\nproxyprovidersvc                                    error  getting address provider metadatasvc\r\njoboptionssvc                                       error unable to set the property \'providernames\' of \'proxyprovidersvc\'. check option and algorithm names, type and bounds.\r\nservicemanager                                      error unable to initialize service &quot;proxyprovidersvc&quot;\r\ndetectorstore                                       error servicelocatorhelper::service: can not locate service proxyprovidersvc\r\ndetectorstore                                       error could not get pointer to proxyprovider service\r\nservicemanager                                      error unable to initialize service &quot;detectorstore&quot;\r\ngeomodelsvc                                         error servicelocatorhelper::service: can not locate service detectorstore\r\ngeomodelsvc                                         fatal detectorstore service not found!\r\nservicemanager                                      error unable to initialize service: geomodelsvc\r\npy:athena            info leaving with code 33: &quot;failure in initialization&quot;\r\napplicationmgr                                       info application manager terminated successfully\r\n\r\nlogfile: \r\n--------- \r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_2/dev/build/x86_64-slc5-gcc43-opt/offline/recexanatest/esd_notrig_memleak/esd_notrig_memleak_log\r\n\r\ncommand: \r\n--------- \r\n\r\nplease consult: \r\n---------------- \r\nhttps://twiki.cern.ch/twiki/bin/vie... \r\ncheers, \r\nrobert (valshift)','100','5211','2011-09-27 22:31:13','5','none','closed','none',';rtt:in;esd_notrig_memleak;rel_2;logfile;snippet;toolsvc;error;create;tool;bytestreammetadatatool;factory;metadatasvc;fatal;metadatatool;publictoolhandlearray;iovdbmetadatatool;bytestreammetadatatool;servicemanager;error;unable;initialize;service;metadatasvc;proxyprovidersvc;error;servicelocatorhelper;service;locate;service;metadatasvc;metadatasvc;proxyprovidersvc;error;address;provider;metadatasvc;joboptionssvc;error;unable;property;providername;proxyprovidersvc;check;option;algorithm;name;type;bound;servicemanager;error;unable;initialize;service;proxyprovidersvc;detectorstore;error;servicelocatorhelper;service;locate;service;proxyprovidersvc;detectorstore;error;pointer;proxyprovider;service;servicemanager;error;unable;initialize;service;detectorstore;geomodelsvc;error;servicelocatorhelper;service;locate;service;detectorstore;geomodelsvc;fatal;detectorstore;service;servicemanager;error;unable;initialize;service;geomodelsvc;py:athena;info;leave;code;failure;initialization;applicationmgr;info;application;manager;terminate;successfully;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_2;build;x86_64;slc5;gcc43;offline;recexanatest;esd_notrig_memleak;esd_notrig_memleak_log;command;consult;http;twiki.cern.ch;twikus;robert;valshift;create;tool;bytestreammetadatatool;esd_notrig_memleak'),(87210,'atlas','issue in photon term in reffinal-like etmiss','hi,\r\n\r\nthis comes from the susy photon channel analyses. we found some potentially undesirable behavior in how certain photons are counted in reffinal like versions of the etmiss. the source of the \'issue\' is in reconstruction/missinget/src/metrefinedeletool.cxx:\r\n\r\n437        else if ( pegamma-&gt;author(egammaparameters::authorphoton )  ||  pegamma-&gt;author(egammaparameters::authorrconv ) ) \r\n438       { \r\n439         // loop on electrons and check if there is overlap between rconv and electrons (in recon from aod overalp had to be removed)    \r\n440         bool isinele = false;\r\n441         egammacontainer::const_iterator fele(elecoll-&gt;begin());\r\n442         egammacontainer::const_iterator lele(elecoll-&gt;end());\r\n443         for ( ; fele != lele; fele++, ie++ )\r\n444         {   \r\n445           // find photons that are also in ele collection \r\n446           if ( (*fele)-&gt;hassameathenabarcodeexceptversion(**fegamma)) { isinele=true ;}   \r\n447         } \r\n448         // go on only if photon is not in ele collection\r\n449         //if (isinele == true) { std::cout &lt;&lt; &quot; found photon in ele coll&quot; &lt;&lt; std::endl; }\r\n450         if (isinele == false) {\r\n...\r\n\r\n\r\nthe problem here is they reject the photon if it has a matching electron, but with *no* quality cuts on the electron. this means *all* recovered photons are excluded. they should only be excluded if the electron it matches passes quality cuts so that it would have been added to the missing et as an electron (the purpose of this cut, i would guess, is only to avoid double counting).\r\n\r\nthere can be cases where the recovered photon (authorrconv) has an electron that is only electronloose isem (likely only loose because it fails the tighter track requirements of tight electron isem defs*) while the photon is photontight. currently then this object would be added as a jet, which is probably not what is desired. for authorphoton photons what the code is doing appears to be ok.\r\n\r\nwould it be possible to make this bit of the met tool configurable, so that photons have matching electrons would be rejected only if the electron passes the quality cuts used for selecting electrons in the met?\r\n\r\ndan','416','6308','2011-09-28 04:23:09','5','none','closed','missinget',';come;susy;photon;channel;analysis;potentially;undesirable;behavior;certain;photon;count;reffinal;like;version;etmiss;source;issue;reconstruction;missinget;metrefinedeletool.cxx;pegamma;author;egammaparameter;authorphoton;pegamma;author;egammaparameter;authorrconv;loop;electron;check;overlap;rconv;electron;recon;overalp;remove;bool;isinele;false;egammacontainer;const_iterator;fele;elecoll;begin;egammacontainer;const_iterator;lele;elecoll;fele;lele;fele;photon;collection;fele;hassameathenabarcodeexceptversion;fegamma;isinele;true;photon;collection;isinele;true;cout;photon;coll;endl;isinele;false;problem;reject;photon;match;electron;quality;electron;means;recover;photon;exclude;exclude;electron;match;pass;quality;miss;electron;purpose;guess;avoid;double;count;case;recover;photon;authorrconv;electron;electronloose;isem;likely;loose;fail;tighter;track;requirement;tight;electron;isem;photon;photontight;currently;object;probably;desire;authorphoton;photon;code;appear;possible;make;meet;tool;configurable;photon;match;electron;reject;electron;pass;quality;select;electron;meet;issue;photon;term;reffinal;like;etmiss;missinget'),(87239,'atlas','fatal in: recjobtransformtests/aodtotag_test, 17.x.0(rel_3)','rtt:fatal with command: aodtotag_trf.py \r\nrecjobtransformtests/aodtotag_test/jobofragment_triggercbnt.py, 17.x.0(rel_3)\r\n\r\nlogfile snippet: \r\n----------------- \r\n===== job report produced by aodtotag version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/aodtotag_trf.py inputaodfile=/afs/cern.ch/atlas/maxidisk/d33/releases/latest.aod.pool.root maxevents=300 autoconfiguration=everything\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (9 times)\r\nerrorcategory=trf_svrinit,	61200,	servicemanager unable to initialize service (4 times)\r\nfatals=2\r\nerrors=10\r\nwarnings=3\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot create tool bytestreammetadatatool (no factory found)\r\n  severity=error\r\n  who=toolsvc\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot get metadatatools = publictoolhandlearray([\'iovdbmetadatatool\',\'lumiblockmetadatatool\',\'bytestreammetadatatool\'])\r\n  severity=fatal\r\n  who=metadatasvc\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_svrinit (61200) servicemanager unable to initialize service\r\n  message=error unable to initialize service &quot;metadatasvc&quot;\r\n  severity=error\r\n  who=servicemanager\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=servicelocatorhelper::service: can not locate service metadatasvc/metadatasvc\r\n  severity=error\r\n  who=proxyprovidersvc\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=getting address provider metadatasvc\r\n  severity=error\r\n  who=proxyprovidersvc\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=unable to set the property \'providernames\' of \'proxyprovidersvc\'. check option and algorithm names, type and bounds.\r\n  severity=error\r\n  who=joboptionssvc\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_svrinit (61200) servicemanager unable to initialize service\r\n  message=error unable to initialize service &quot;proxyprovidersvc&quot;\r\n  severity=error\r\n  who=servicemanager\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=servicelocatorhelper::service: can not locate service proxyprovidersvc\r\n  severity=error\r\n  who=detectorstore\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=could not get pointer to proxyprovider service\r\n  severity=error\r\n  who=detectorstore\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_svrinit (61200) servicemanager unable to initialize service\r\n  message=error unable to initialize service &quot;detectorstore&quot;\r\n  severity=error\r\n  who=servicemanager\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=servicelocatorhelper::service: can not locate service detectorstore\r\n  severity=error\r\n  who=geomodelsvc\r\n  producer=aodtotag\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=detectorstore service not found!\r\n  severity=fatal\r\n  who=geomodelsvc\r\n  producer=aodtotag\r\n  stage=initialise\r\nworkdir=/pool/lsf/rtt/173273776\r\nathcode=33\r\nrundiravailabledisk=143043702784\r\ndbrelease=\r\nathacronym=failure in initialization\r\nrundiruseddisk=491428\r\natlasrelease=rel_3\r\nwriting jobinfo_aodtotag.xml file...\r\n\r\nlogfile: \r\n--------- \r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_3/dev/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/aodtotag_test/aodtotag_test_log \r\n\r\ncommand: \r\n--------- \r\n/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/aodtotag_trf.py\r\n\r\nplease consult: \r\n---------------- \r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-09-28 16:05:43','5','none','closed','none',';rtt:fatal;command;aodtotag_trf.py;recjobtransformtest;aodtotag_test;jobofragment_triggercbnt.py;rel_3;logfile;snippet;report;produce;aodtotag;version;unknown;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;aodtotag_trf.py;inputaodfile;atla;maxidisk;release;latest.aod.pool.root;maxevent;autoconfiguration;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;errorcategory;trf_svrinit;servicemanager;unable;initialize;service;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;create;tool;bytestreammetadatatool;factory;severity;error;toolsvc;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;metadatatool;publictoolhandlearray;iovdbmetadatatool;lumiblockmetadatatool;bytestreammetadatatool;severity;fatal;metadatasvc;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_svrinit;servicemanager;unable;initialize;service;message;error;unable;initialize;service;metadatasvc;severity;error;servicemanager;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;servicelocatorhelper;service;locate;service;metadatasvc;metadatasvc;severity;error;proxyprovidersvc;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;address;provider;metadatasvc;severity;error;proxyprovidersvc;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;unable;property;providername;proxyprovidersvc;check;option;algorithm;name;type;bound;severity;error;joboptionssvc;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_svrinit;servicemanager;unable;initialize;service;message;error;unable;initialize;service;proxyprovidersvc;severity;error;servicemanager;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;servicelocatorhelper;service;locate;service;proxyprovidersvc;severity;error;detectorstore;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;pointer;proxyprovider;service;severity;error;detectorstore;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_svrinit;servicemanager;unable;initialize;service;message;error;unable;initialize;service;detectorstore;severity;error;servicemanager;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;servicelocatorhelper;service;locate;service;detectorstore;severity;error;geomodelsvc;producer;aodtotag;stage;initialise;errorinfo;acronym;trf_unknown;message;detectorstore;service;severity;fatal;geomodelsvc;producer;aodtotag;stage;initialise;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;failure;initialization;rundiruseddisk;atlasrelease;rel_3;write;jobinfo_aodtotag.xml;file;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_3;build;x86_64;slc5;gcc43;offline;recjobtransformtest;aodtotag_test;aodtotag_test_log;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;aodtotag_trf.py;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;fatal;recjobtransformtest;aodtotag_test;rel_3'),(87241,'atlas','include error in mc10_default_short','dear experts,\r\n\r\nin todays 17.x.0 (i686-slc5-gcc43-opt) rel_3 there was an include error. see logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_3/dev/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/mc10_default_short/mc10_default_short_log\r\n\r\nshortened traceback (most recent user call last):\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/joboptions/recjobtransforms/skeleton.rawtoesd.py&quot;, line 134, in &lt;module&gt;\r\n    else: include( &quot;recexcommon/recexcommon_topoptions.py&quot; )\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/joboptions/recexcommon/recexcommon_topoptions.py&quot;, line 889, in &lt;module&gt;\r\n    include (&quot;lumiblockathenapool/lumiblockathenapool_joboptions.py&quot;)\r\nincludeerror: include file lumiblockathenapool/lumiblockathenapool_joboptions.py can not be found\r\npy:athena            info leaving with code 2: &quot;an error occurred when including a file&quot;\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_3/installarea/share/atlas_error_categories.db\r\n/afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_3/installarea/share/bin/athena.py -b runargs.rawtoesd.py recjobtransforms/skeleton.rawtoesd.py rawtoesd_messagesvc_joboptions.py --drop-and-reload - exit code 2\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code 2.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_3/installarea/share/atlas_error_categories.db\r\ninfo: read 53 error patterns from file /afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_3/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_3/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: wed sep 28 10:15:22 2011 cest, vmem: 112.37890625 mb, rss: 67.7578125 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_3 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info done scanning 5335 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 0\r\npy:rawtoesd          info    warnings: 9\r\npy:rawtoesd          info    errors  : 0\r\npy:rawtoesd          info    fatals  : 1\r\nathenalogchecker stop [rawtoesd log] wall-time: wed sep 28 10:15:22 2011 cest, vmem: 112.37890625 mb, rss: 67.80859375 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 00 secs, cpu-time: 00 hrs 00 mins 00 secs, vmem: 0.0 mb, rss: 0.05078125 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 10510 (exit code 5)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: wed sep 28 10:15:22 2011 cest, vmem: 112.37890625 mb, rss: 67.8359375 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 04 mins 29 secs, cpu-time: 00 hrs 03 mins 13 secs, vmem: 0.12890625 mb, rss: 0.3671875 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/mc10_7tev.105200.t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00/rdo.197091._000001.pool.root.1 outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root outputhistfile=t1_mcatnlo_jimmy.hist.root maxevents=25 autoconfiguration=everything\r\nexitcode=5\r\nerrorcode=10510 (ath_jop_notfound)\r\nerrorcategory=ath_jop_notfound,	10510,	joboptions file not found\r\nfatals=1\r\nerrors=0\r\nwarnings=9\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 10510 (exit code 5)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: wed sep 28 10:15:22 2011 cest, vmem: 112.54296875 mb, rss: 68.00390625 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 00 hrs 04 mins 29 secs, cpu-time: 00 hrs 03 mins 13 secs, vmem: 0.29296875 mb, rss: 0.5625 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/mc10_7tev.105200.t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00/rdo.197091._000001.pool.root.1 outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root outputhistfile=t1_mcatnlo_jimmy.hist.root maxevents=25 autoconfiguration=everything\r\nexitcode=5\r\nerrorcode=10510 (ath_jop_notfound)\r\nerrorcategory=ath_jop_notfound,	10510,	joboptions file not found\r\nfatals=1\r\nerrors=0\r\nwarnings=9\r\nerrorinfo\r\n  acronym=ath_jop_notfound (10510) joboptions file not found\r\n  message=includeerror: include file lumiblockathenapool/lumiblockathenapool_joboptions.py can not be found\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=initialise\r\nathacronym=an error occurred when including a file\r\nathcode=2\r\nrundiruseddisk=3532278\r\nrundiravailabledisk=186008879104\r\nworkdir=/pool/lsf/rtt/173281366\r\nathcode=2\r\nrundiravailabledisk=186009108480\r\ndbrelease=\r\nathacronym=an error occurred when including a file\r\nrundiruseddisk=3313061\r\natlasrelease=rel_3\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\n\r\ncheers michael','100','5211','2011-09-28 16:13:01','5','none','closed','none',';dear;expert;today;slc5;gcc43;rel_3;include;error;logfile;http;atlasrtt.cern.ch;rel_3;build;slc5;gcc43;offline;recjobtransformtest;mc10_default_short;mc10_default_short_log;shorten;traceback;recent;user;file;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;joboption;recjobtransform;skeleton.rawtoesd.py;line;include;recexcommon;recexcommon_topoptions.py;file;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;joboption;recexcommon;recexcommon_topoptions.py;line;include;lumiblockathenapool;lumiblockathenapool_joboptions.py;includeerror;include;file;lumiblockathenapool;lumiblockathenapool_joboptions.py;py:athena;info;leave;code;error;occur;include;file;info;read;error;category;file;atla;software;build;nightly;atlascore;rel_3;installarea;share;atlas_error_categories.db;atla;software;build;nightly;atlascore;rel_3;installarea;share;athena.py;runargs.rawtoesd.py;recjobtransform;skeleton.rawtoesd.py;rawtoesd_messagesvc_joboptions.py;drop;reload;exit;code;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;info;read;error;category;file;atla;software;build;nightly;atlascore;rel_3;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;build;nightly;atlascore;rel_3;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;build;nightly;atlascore;rel_3;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_3;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;reco_trf.py;uploadtoamus;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;mc10_7tev;t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00;pool.root;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;outputhistfile;t1_mcatnlo_jimmy.hist.root;maxevent;autoconfiguration;exitcode;errorcode;ath_jop_notfound;errorcategory;ath_jop_notfound;joboption;file;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;reco_trf.py;uploadtoamus;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;mc10_7tev;t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00;pool.root;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;outputhistfile;t1_mcatnlo_jimmy.hist.root;maxevent;autoconfiguration;exitcode;errorcode;ath_jop_notfound;errorcategory;ath_jop_notfound;joboption;file;fatal;error;warning;errorinfo;acronym;ath_jop_notfound;joboption;file;message;includeerror;include;file;lumiblockathenapool;lumiblockathenapool_joboptions.py;severity;fatal;producer;rawtoesd;stage;initialise;athacronym;error;occur;include;file;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;error;occur;include;file;rundiruseddisk;atlasrelease;rel_3;write;jobinfo_recotrf.xml;file;michael;include;error;mc10_default_short'),(87272,'atlas','error in recexanatest/esd_memleakana,  17.x.0(rel_3)','rtt: error in recexanatest/esd_memleakana,  17.x.0(rel_3)\r\n\r\nlogfile snippet:\r\n-----------------\r\ntraceback (most recent call last):\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasevent/rel_3/installarea/python/recexconfig/inputfilepeeker.py&quot;, line 48, in _setup\r\n    fi = athfile.fopen(infile)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_3/installarea/python/pyutils/athfile/__init__.py&quot;, line 160, in fopen\r\n    return self.server.fopen(fnames, evtmax)\r\n  file &quot;&lt;string&gt;&quot;, line 2, in fopen\r\n  file &quot;/afs/cern.ch/sw/lcg/external/pytools/1.5_python2.6/x86_64-slc5-gcc43-opt/lib/python2.6/site-packages/multiprocessing-2.6.2.1-py2.6-linux-x86_64.egg/multiprocessing/managers.py&quot;, line 743, in _callmethod\r\n    raise convert_to_error(kind, result)\r\nkeyerror: -11\r\npy:inputfilepeeker   error unable to build inputfilesummary from any of the specified input files. there is probably a problem.\r\npy:autoconfiguration   error no runnumber stored in inputfile!\r\nshortened traceback (most recent user call last):\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasanalysis/rel_3/installarea/joboptions/recexanatest/recexanatest_rtt_all.py&quot;, line 4, in &lt;module&gt;\r\n    include (&quot;recexcommon/recexcommon_topoptions.py&quot;)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/joboptions/recexcommon/recexcommon_topoptions.py&quot;, line 89, in &lt;module&gt;\r\n    include ( &quot;recexcond/recexcommon_flags.py&quot; )\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasevent/rel_3/installarea/joboptions/recexcond/recexcommon_flags.py&quot;, line 235, in &lt;module&gt;\r\n    configurefromlistofkeys(rec.autoconfiguration())\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasevent/rel_3/installarea/python/recexconfig/autoconfiguration.py&quot;, line 891, in configurefromlistofkeys\r\n    rec.projectname=getprojectname()\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasevent/rel_3/installarea/python/recexconfig/autoconfiguration.py&quot;, line 286, in getprojectname\r\n    from recexconfig.getcool import cool\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasevent/rel_3/installarea/python/recexconfig/getcool.py&quot;, line 32, in &lt;module&gt;\r\n    cool=_setup()\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasevent/rel_3/installarea/python/recexconfig/getcool.py&quot;, line 19, in _setup\r\n    lb=getlbnumber()\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasevent/rel_3/installarea/python/recexconfig/autoconfiguration.py&quot;, line 63, in getlbnumber\r\n    if inputfilesummary[\'nentries\']==0:\r\nkeyerror: \'nentries\'\r\npy:athena            info leaving with code 8: &quot;an unknown exception occurred&quot;\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_3/dev/build/x86_64-slc5-gcc43-opt/offline/recexanatest/esd_memleakana/esd_memleakana_log\r\n\r\ncomments:\r\n---------\r\nthere were multiple crashes related to this file:\r\n/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/joboptions/recexcommon/recexcommon_topoptions.py\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','1658','5211','2011-09-29 14:31:33','5','fixed','closed','none',';error;recexanatest;esd_memleakana;rel_3;logfile;snippet;traceback;recent;file;atla;software;build;nightly;atlasevent;rel_3;installarea;python;recexconfig;inputfilepeeker.py;line;athfile.fopen;infile;file;atla;software;build;nightly;atlascore;rel_3;installarea;python;pyutil;athfile;line;fopen;return;self.server.fopen;fname;evtmax;file;line;fopen;file;external;pytool;1.5_python2.6;x86_64;slc5;gcc43;python2.6;site;package;multiprocess;2.6.2.1;py2.6;linux;x86_64;multiprocess;managers.py;line;raise;convert_to_error;kind;result;keyerror;py:inputfilepeeker;error;unable;build;inputfilesummary;specify;input;file;probably;problem;py:autoconfiguration;error;runnumber;store;inputfile;shorten;traceback;recent;user;file;atla;software;build;nightly;atlasanalysis;rel_3;installarea;joboption;recexanatest;recexanatest_rtt_all.py;line;include;recexcommon;recexcommon_topoptions.py;file;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;joboption;recexcommon;recexcommon_topoptions.py;line;include;recexcond;recexcommon_flags.py;file;atla;software;build;nightly;atlasevent;rel_3;installarea;joboption;recexcond;recexcommon_flags.py;line;configurefromlistofkey;rec.autoconfiguration;file;atla;software;build;nightly;atlasevent;rel_3;installarea;python;recexconfig;autoconfiguration.py;line;configurefromlistofkey;rec.projectname;getprojectname;file;atla;software;build;nightly;atlasevent;rel_3;installarea;python;recexconfig;autoconfiguration.py;line;getprojectname;recexconfig.getcool;import;cool;file;atla;software;build;nightly;atlasevent;rel_3;installarea;python;recexconfig;getcool.py;line;cool;file;atla;software;build;nightly;atlasevent;rel_3;installarea;python;recexconfig;getcool.py;line;getlbnumber;file;atla;software;build;nightly;atlasevent;rel_3;installarea;python;recexconfig;autoconfiguration.py;line;getlbnumber;inputfilesummary;nentry;keyerror;nentry;py:athena;info;leave;code;unknown;exception;occur;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_3;build;x86_64;slc5;gcc43;offline;recexanatest;esd_memleakana;esd_memleakana_log;comment;multiple;crash;related;file;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;joboption;recexcommon;recexcommon_topoptions.py;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recexanatest;esd_memleakana;rel_3'),(87273,'atlas','error in recexanatest/mc10_chain_prod_esdtoaod,  17.x.0(rel_3)','rtt: error in recexanatest/mc10_chain_prod_esdtoaod,  17.x.0(rel_3)\r\n\r\nlogfile snippet:\r\n-----------------\r\ntoolsvc                                             error cannot create tool bytestreammetadatatool (no factory found)\r\nmetadatasvc                                         fatal cannot get metadatatools = publictoolhandlearray([\'iovdbmetadatatool\',\'lumiblockmetadatatool\',\'bytestreammetadatatool\'])\r\nservicemanager                                      error unable to initialize service &quot;metadatasvc&quot;\r\nproxyprovidersvc                                    error servicelocatorhelper::service: can not locate service metadatasvc/metadatasvc\r\nproxyprovidersvc                                    error  getting address provider metadatasvc\r\njoboptionssvc                                       error unable to set the property \'providernames\' of \'proxyprovidersvc\'. check option and algorithm names, type and bounds.\r\nservicemanager                                      error unable to initialize service &quot;proxyprovidersvc&quot;\r\ndetectorstore                                       error servicelocatorhelper::service: can not locate service proxyprovidersvc\r\ndetectorstore                                       error could not get pointer to proxyprovider service\r\nservicemanager                                      error unable to initialize service &quot;detectorstore&quot;\r\ngeomodelsvc                                         error servicelocatorhelper::service: can not locate service detectorstore\r\ngeomodelsvc                                         fatal detectorstore service not found!\r\nservicemanager                                      error unable to initialize service: geomodelsvc\r\npy:athena            info leaving with code 33: &quot;failure in initialization&quot;\r\napplicationmgr                                       info application manager terminated successfully\r\nsummarizing all message counts\r\n=====================================================\r\n message source              |   level |    count\r\n-----------------------------+---------+-------------\r\n detectorstore               |   error |        2\r\n geomodelsvc                 |   error |        1\r\n geomodelsvc                 |   fatal |        1\r\n joboptionssvc               |   error |        1\r\n metadatasvc                 |   fatal |        1\r\n proxyprovidersvc            |   error |        2\r\n servicemanager              |   error |        4\r\n toolsvc                     |   error |        1\r\n toolsvc.goodrunslistselectortool| warning |        1\r\n=====================================================\r\ncoral/services/connectionservice info deleting the connectionpool\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_3/dev/build/x86_64-slc5-gcc43-opt/offline/recexanatest/mc10_chain_prod_esdtoaod/mc10_chain_prod_esdtoaod_log\r\n\r\ncommand:\r\n---------\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-09-29 14:36:22','5','none','closed','none',';error;recexanatest;mc10_chain_prod_esdtoaod;rel_3;logfile;snippet;toolsvc;error;create;tool;bytestreammetadatatool;factory;metadatasvc;fatal;metadatatool;publictoolhandlearray;iovdbmetadatatool;lumiblockmetadatatool;bytestreammetadatatool;servicemanager;error;unable;initialize;service;metadatasvc;proxyprovidersvc;error;servicelocatorhelper;service;locate;service;metadatasvc;metadatasvc;proxyprovidersvc;error;address;provider;metadatasvc;joboptionssvc;error;unable;property;providername;proxyprovidersvc;check;option;algorithm;name;type;bound;servicemanager;error;unable;initialize;service;proxyprovidersvc;detectorstore;error;servicelocatorhelper;service;locate;service;proxyprovidersvc;detectorstore;error;pointer;proxyprovider;service;servicemanager;error;unable;initialize;service;detectorstore;geomodelsvc;error;servicelocatorhelper;service;locate;service;detectorstore;geomodelsvc;fatal;detectorstore;service;servicemanager;error;unable;initialize;service;geomodelsvc;py:athena;info;leave;code;failure;initialization;applicationmgr;info;application;manager;terminate;successfully;summarize;message;count;message;source;level;count;detectorstore;error;geomodelsvc;error;geomodelsvc;fatal;joboptionssvc;error;metadatasvc;fatal;proxyprovidersvc;error;servicemanager;error;toolsvc;error;toolsvc.goodrunslistselectortool;warn;coral;service;connectionservice;info;delete;connectionpool;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_3;build;x86_64;slc5;gcc43;offline;recexanatest;mc10_chain_prod_esdtoaod;mc10_chain_prod_esdtoaod_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recexanatest;mc10_chain_prod_esdtoaod;rel_3'),(87275,'atlas','error in recjobtransformtests,  17.x.0(rel_3)','rtt: error in recjobtransformtests,  17.x.0(rel_3)\r\n\r\nthis error is in almost all the jobs in the branch!\r\n\r\nlogfile snippet:\r\n-----------------\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputbsfile=/afs/cern.ch/atlas/project/rig/referencefiles/datastreams_high_mu/high_mu-data11_7tev.00179725.physics_jettauetmiss.merge.raw._lb0021.data,/afs/cern.ch/atlas/project/rig/referencefiles/datastreams_high_mu/high_mu-data11_7tev.00180481.physics_jettauetmiss.merge.raw._lb0172.data outputesdfile=myesd.pool.root outputaodfile=myaod.pool.root outputtagfile=mytag.pool.root outputhistfile=myhist.pool.root\r\nexitcode=5\r\nerrorcode=10510 (ath_jop_notfound)\r\nerrorcategory=ath_jop_notfound,	10510,	joboptions file not found\r\nfatals=1\r\nerrors=0\r\nwarnings=2\r\nerrorinfo\r\n  acronym=ath_jop_notfound (10510) joboptions file not found\r\n  message=includeerror: include file lumiblockathenapool/lumiblockathenapool_joboptions.py can not be found\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=initialise\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/builds/nightlies/dev/atlascore/rel_3/installarea/python/pyjobtransformscore/trf.py&quot;, line 2043, in runjob\r\n              maxevents = maxeventsarg.value()\r\nathacronym=an error occurred when including a file\r\nathcode=2\r\nrundiruseddisk=592973\r\nrundiravailabledisk=735037177856\r\nworkdir=/pool/lsf/rtt/173274204\r\nathcode=2\r\nrundiravailabledisk=735037407232\r\ndbrelease=\r\nathacronym=an error occurred when including a file\r\nrundiruseddisk=379758\r\natlasrelease=rel_3\r\nwriting jobinfo_recotrf.xml file...\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_3/dev/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_diffinput/data_7tev_diffinput_log\r\n\r\ncommand:\r\n---------\r\n/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/reco_trf.py --uploadtoami=1.0\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-09-29 14:55:24','5','none','closed','none',';error;recjobtransformtest;rel_3;error;branch;logfile;snippet;report;produce;recotrf;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;atla;project;referencefile;datastreams_high_mu;high_mu;data11_7tev;physics_jettauetmiss.merge.raw;datum;atla;project;referencefile;datastreams_high_mu;high_mu;data11_7tev;physics_jettauetmiss.merge.raw;datum;outputesdfile;myesd.pool.root;outputaodfile;myaod.pool.root;outputtagfile;mytag.pool.root;outputhistfile;myhist.pool.root;exitcode;errorcode;ath_jop_notfound;errorcategory;ath_jop_notfound;joboption;file;fatal;error;warning;errorinfo;acronym;ath_jop_notfound;joboption;file;message;includeerror;include;file;lumiblockathenapool;lumiblockathenapool_joboptions.py;severity;fatal;producer;rawtoesd;stage;initialise;backtrace;file;atla;software;build;nightly;atlascore;rel_3;installarea;python;pyjobtransformscore;trf.py;line;runjob;maxevent;maxeventsarg.value;athacronym;error;occur;include;file;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;error;occur;include;file;rundiruseddisk;atlasrelease;rel_3;write;jobinfo_recotrf.xml;file;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_3;build;x86_64;slc5;gcc43;offline;recjobtransformtest;data_7tev_diffinput;data_7tev_diffinput_log;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;reco_trf.py;uploadtoamus;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recjobtransformtest;rel_3'),(87276,'atlas','fatal in: recexanatest/mc10_all_nofpe, 17.x.0-val(rel_4)','rtt: fatal in: recexanatest/mc10_all_nofpe, 17.x.0-val(rel_4)\r\n\r\nlogfile snippet:\r\n-----------------\r\ncore dump from coredumpsvc on lxbrb2206.cern.ch at thu sep 29 13:39:36 2011\r\n\r\ncaught signal 8(floating point exception). details: \r\n  errno = 0, code = 4 (floating point overflow)\r\n  pid   = -820098103, uid = 10922\r\n  value = (0, 0)\r\n  vmem = 3345.43 mb\r\n  rss  = 2674.11 mb\r\n  total-ram = 15929.1 mb\r\n  free-ram  = 85.2148 mb\r\n  buffer-ram= 92.5547 mb\r\n  total-swap= 4094.55 mb\r\n  free-swap = 4094.32 mb\r\n  addr  = 0x2aaacf1e4bc9\r\n\r\nevent counter: 67\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [105200,31967,3,0:0,0,0x0]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent trigger chain: ef_e10_nocut\r\ncurrent algorithm: trigegammarec_egamma\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   trigsteer_ef\r\n   trigegammarec_egamma\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_4/devval/build/x86_64-slc5-gcc43-opt/offline/recexanatest/mc10_all_nofpe/mc10_all_nofpe_log\r\n\r\ncommand:\r\n---------\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','5259','5211','2011-09-29 15:12:15','5','duplicate','closed','none',';fatal;recexanatest;mc10_all_nofpe;rel_4;logfile;snippet;core;dump;coredumpsvc;lxbrb2206;catch;signal;float;point;exception;detail;errno;code;float;point;overflow;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;trigger;chain;ef_e10_nocut;current;algorithm;trigegammarec_egamma;algorithm;stack;athmasterseq;athalgseq;trigsteer_ef;trigegammarec_egamma;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_4;devval;build;x86_64;slc5;gcc43;offline;recexanatest;mc10_all_nofpe;mc10_all_nofpe_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;fatal;recexanatest;mc10_all_nofpe;rel_4'),(87281,'atlas','error in recperftests/all_bs, 17.x.0(rel_3)','rtt: error in recperftests/all_bs, 17.x.0(rel_3)\r\n\r\nthis error is seemingly related to:\r\nhttps://savannah.cern.ch/bugs/?87196\r\n\r\nthe reply to this bug was &quot;there was not enough memory and this happens&quot;. this error is slightly different, not sure if it falls under this same prescription.\r\n\r\nlogfile snippet:\r\n-----------------\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/rawtoesd_trf.py inputbsfile=/afs/cern.ch/atlas/project/pmb/data/data11_7tev.00182726.physics_jettauetmiss.merge.raw._lb0215._sfo-5._0001.1_1000evts maxevents=250 autoconfiguration=everything --athenaopts=--stdcmalloc preexec=from@perfmoncomps.perfmonflags@import@jobproperties@as@pmjp,,pmjp.perfmonflags.domallocmonitoring=true,,pmjp.perfmonflags.dopostprocessing=true,,pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true outputdqmonitorfile=mymon.root outputesdfile=myesd.pool.root outputntup_muoncalibfile=myntup_muoncalib.root outputdraw_zmumufile=mydraw_zmumu.data outputdraw_zeefile=mydraw_zee.data\r\nexitcode=5\r\nerrorcode=10510 (ath_jop_notfound)\r\nerrorcategory=ath_jop_notfound,	10510,	joboptions file not found\r\nfatals=1\r\nerrors=0\r\nwarnings=2\r\nerrorinfo\r\n  acronym=ath_jop_notfound (10510) joboptions file not found\r\n  message=includeerror: include file lumiblockathenapool/lumiblockathenapool_joboptions.py can not be found\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=initialise\r\nworkdir=/pool/lsf/rtt/173274194\r\nathcode=2\r\nrundiravailabledisk=185634140160\r\ndbrelease=\r\nathacronym=an error occurred when including a file\r\nrundiruseddisk=406774\r\natlasrelease=rel_3\r\nwriting jobinfo_rawtoesd.xml file...\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_3/dev/build/x86_64-slc5-gcc43-opt/offline/recperftests/all_bs/all_bs_log\r\n\r\ncommand:\r\n---------\r\n/afs/cern.ch/atlas/software/builds/nightlies/dev/atlasreconstruction/rel_3/installarea/share/bin/rawtoesd_trf.py\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-09-29 16:51:41','5','none','closed','none',';error;recperftest;all_bs;rel_3;error;seemingly;related;http;savannah.cern.ch;reply;memory;happen;error;slightly;different;sure;fall;prescription;logfile;snippet;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;rawtoesd_trf.py;inputbsfile;atla;project;datum;data11_7tev;physics_jettauetmiss.merge.raw;maxevent;autoconfiguration;athenaopt;stdcmalloc;preexec;perfmoncomps.perfmonflags;import;jobproperty;pmjp;pmjp.perfmonflags.domallocmonitoring;true;pmjp.perfmonflags.dopostprocessing;true;pmjp.perfmonflags.dosemidetailedmonitoringfullprint;true;outputdqmonitorfile;mymon.root;outputesdfile;myesd.pool.root;outputntup_muoncalibfile;myntup_muoncalib.root;outputdraw_zmumufile;mydraw_zmumu.data;outputdraw_zeefile;mydraw_zee.data;exitcode;errorcode;ath_jop_notfound;errorcategory;ath_jop_notfound;joboption;file;fatal;error;warning;errorinfo;acronym;ath_jop_notfound;joboption;file;message;includeerror;include;file;lumiblockathenapool;lumiblockathenapool_joboptions.py;severity;fatal;producer;rawtoesd;stage;initialise;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;error;occur;include;file;rundiruseddisk;atlasrelease;rel_3;write;jobinfo_rawtoesd.xml;file;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_3;build;x86_64;slc5;gcc43;offline;recperftest;all_bs;all_bs_log;command;atla;software;build;nightly;atlasreconstruction;rel_3;installarea;share;rawtoesd_trf.py;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recperftest;all_bs;rel_3'),(87310,'atlas','fatal error in mc_default_short','dear experts,\r\n\r\nin todays 17.x.0-val (i686-slc5-gcc43-dbg) rel_5, error in the eventloopmgr for mc_default_short.\r\n\r\nsee logfile:\r\nhttps://atlasrtt.cern.ch/rtt/rel_5/devval/build/i686-slc5-gcc43-dbg/offline/recjobtransformtests/mc10_default_short/mc10_default_short_log\r\n\r\n---------------------------------------------------------------------------------\r\ncore dump from coredumpsvc on lxb8773.cern.ch at fri sep 30 14:39:29 2011\r\n\r\ncaught signal 11(segmentation fault). details: \r\n  errno = 0, code = 1 (address not mapped to object)\r\n  pid   = 0, uid = 1609383097\r\n  value = (0, 0)\r\n  vmem = 2388.09 mb\r\n  rss  = 1919.02 mb\r\n  total-ram = 15925.7 mb\r\n  free-ram  = 3652.96 mb\r\n  buffer-ram= 877.398 mb\r\n  total-swap= 4094.55 mb\r\n  free-swap = 4053.46 mb\r\n  addr  = 0\r\n\r\nevent counter: 3\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [105200,31853,3,0:0,0,0x0]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent trigger chain: &lt;none&gt;\r\ncurrent algorithm: streamesd\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   streamesd\r\n-------------------------------------------------------------------------------------\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code -11.\r\npy:rawtoesd         error athena received signal 11. exit code reset to athena exit code 8.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_5/installarea/share/atlas_error_categories.db\r\ninfo: read 53 error patterns from file /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_5/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_5/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: fri sep 30 14:39:29 2011 cest, vmem: 126.00390625 mb, rss: 81.01953125 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_5 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info done scanning 59792 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 0\r\npy:rawtoesd          info    warnings: 111\r\npy:rawtoesd          info    errors  : 0\r\npy:rawtoesd          info    fatals  : 1\r\nathenalogchecker stop [rawtoesd log] wall-time: fri sep 30 14:39:37 2011 cest, vmem: 126.14453125 mb, rss: 81.109375 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 07 secs, cpu-time: 00 hrs 00 mins 07 secs, vmem: 0.140625 mb, rss: 0.08984375 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 60010 (exit code 6)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: fri sep 30 14:39:37 2011 cest, vmem: 126.14453125 mb, rss: 81.1328125 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 17 mins 06 secs, cpu-time: 00 hrs 14 mins 46 secs, vmem: 0.53125 mb, rss: 0.65234375 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_5/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/mc10_7tev.105200.t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00/rdo.197091._000001.pool.root.1 outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root outputhistfile=t1_mcatnlo_jimmy.hist.root maxevents=25 autoconfiguration=everything\r\nexitcode=6\r\nerrorcode=60010 (trf_segfault)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 60010 (exit code 6)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: fri sep 30 14:39:37 2011 cest, vmem: 126.2890625 mb, rss: 81.26953125 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 00 hrs 17 mins 06 secs, cpu-time: 00 hrs 14 mins 46 secs, vmem: 0.93359375 mb, rss: 0.90625 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_5/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/mc10_7tev.105200.t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00/rdo.197091._000001.pool.root.1 outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root outputhistfile=t1_mcatnlo_jimmy.hist.root maxevents=25 autoconfiguration=everything\r\nexitcode=6\r\nerrorcode=60010 (trf_segfault)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_segfault (60010) segmentation fault\r\n  message=caught signal 11(segmentation fault). details:\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=execute\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=18675131\r\nrundiravailabledisk=194569949184\r\nworkdir=/pool/lsf/rtt/173873515\r\nathcode=8\r\nrundiravailabledisk=194570088448\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=18538688\r\natlasrelease=rel_5\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\ncheers michael ','1658','5211','2011-09-30 16:28:48','5','none','closed','none',';dear;expert;today;slc5;gcc43;rel_5;error;eventloopmgr;mc_default_short;logfile;http;atlasrtt.cern.ch;rel_5;devval;build;slc5;gcc43;offline;recjobtransformtest;mc10_default_short;mc10_default_short_log;core;dump;coredumpsvc;lxb8773;frus;catch;signal;segmentation;fault;detail;errno;code;address;object;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;trigger;chain;current;algorithm;streamesd;algorithm;stack;athmasterseq;athalgseq;streamesd;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;py:rawtoesd;error;athena;receive;signal;exit;code;reset;athena;exit;code;info;read;error;category;file;atla;software;build;nightly;devval;atlascore;rel_5;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;build;nightly;devval;atlascore;rel_5;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;build;nightly;devval;atlascore;rel_5;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;frus;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_5;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;frus;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;frus;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;devval;atlasreconstruction;rel_5;installarea;share;reco_trf.py;uploadtoamus;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;mc10_7tev;t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00;pool.root;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;outputhistfile;t1_mcatnlo_jimmy.hist.root;maxevent;autoconfiguration;exitcode;errorcode;trf_segfault;errorcategory;trf_segfault;segmentation;fault;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;frus;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;devval;atlasreconstruction;rel_5;installarea;share;reco_trf.py;uploadtoamus;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;mc10_7tev;t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00;pool.root;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;outputhistfile;t1_mcatnlo_jimmy.hist.root;maxevent;autoconfiguration;exitcode;errorcode;trf_segfault;errorcategory;trf_segfault;segmentation;fault;fatal;error;warning;errorinfo;acronym;trf_segfault;segmentation;fault;message;catch;signal;segmentation;fault;detail;severity;fatal;producer;rawtoesd;stage;execute;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_5;write;jobinfo_recotrf.xml;file;michael;fatal;error;mc_default_short'),(87318,'atlas','some valgrindrttjobs	seem to be failed by patjobtransforms update (17.0.x.y)','hi, \r\n\r\nsince 17.0.x.y rel_3, \r\ncallgrindpileuprecodatajobeventloop\r\ncallgrindpileuprecodatajobinifirstevent\r\ncallgrindpileuprecomcjobeventloop\r\ncallgrindpileuprecomcjobinifirstevent\r\ndhatcollisionsrawtoesd_calo\r\ndhatcollisionsrawtoesd_id\r\ndhatcollisionsrawtoesd_muon\r\nfailed with \r\n+verbatim+\r\npy:valgrind          info  not vetoed, so we will run valgrind !\r\npy:valgrind         error during execution of valgrind, exception caught: local variable \'athenareport\' referenced before assignment\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/share/atlas_error_categories.db\r\npy:valgrind          info jobtransform completed for valgrind with error code 63400 (exit code 34)\r\npy:valgrind          info transform timer stop [valgrind] wall-time: wed sep 28 04:45:20 2011 cest, vmem: 15.15625 mb, rss: 10.35546875 mb\r\npy:valgrind          info transform timer report [valgrind] wall-time: 00 hrs 00 mins 00 secs, cpu-time: 00 hrs 00 mins 00 secs, vmem: 0.1328125 mb, rss: 0.1875 mb\r\n===== job report produced by valgrind version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.4/atlasreconstruction/17.0.4/installarea/share/bin/valgrind_trf.py valgrindtool=callgrind recotrf=reco_trf.py+inputbsfile=/afs/cern.ch/user/s/seuster/rig/referencefiles/datastreams_10events/data11_7tev.00178044.physics_jettauetmiss.merge.raw._lb0092._sfo-4._0001.1_25events+outputesdfile=myesd.pool.root+maxevents=5+autoconfiguration=everything+preexec=rec.doperfmon=false,,rec.dodetailedperfmon=false,,rec.dosemidetailedperfmon=false,,\'rec.domonitoring.set_value_and_lock(true)\',,&quot;rec.useralgs=[\\&quot;valgrindrttjobs/vgopts.py\\&quot;]&quot;+postexec=topsequence.timeout=0+--ignoreunknown stacktracedepth=5 trackorigins=false dumpinstr=true instratstart=false valgrindversion=3.6.0 reqcmtconfig=i686-slc5-gcc43\r\nexitcode=34\r\nerrorcode=63400 (trf_exc)\r\nerrorcategory=trf_exc,	63400,	transform unknown exceptions\r\nfatals=1\r\nerrors=0\r\nwarnings=0\r\nerrorinfo\r\n  acronym=trf_exc (63400) transform unknown exceptions\r\n  message=unboundlocalerror: (&quot;local variable \'athenareport\' referenced before assignment&quot;,)\r\n  severity=fatal\r\n  producer=valgrind\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/python/pyjobtransformscore/trf.py&quot;, line 2189, in execute\r\n              self._jobreport.addreport( self.runjob(), \'merge\' )\r\n            file &quot;/afs/cern.ch/atlas/software/releases/17.0.4/atlasreconstruction/17.0.4/installarea/share/bin/valgrind_trf.py&quot;, line 541, in runjob\r\n              athenareport.setproducer(self.name(),self.version())\r\n-verbatim-\r\nhttps://atlasrtt.cern.ch/rtt/rel_3/17.0.x.y/build/i686-slc5-gcc43-opt/offline/valgrindrttjobs/callgrindpileuprecodatajobeventloop/callgrindpileuprecodatajobeventloop_log\r\nhttps://atlasrtt.cern.ch/rtt/query.php?q=-p%20atlasproduction%20-b%2017.0.x.y%20-c%20i686-slc5-gcc43-opt%20-p%20valgrindrttjobs%20-r%20rel_0%20rel_1%20rel_2%20rel_3%20rel_4%20rel_5%20rel_6%20-v%20jobs\r\n\r\nthe tag differences between 17.0.x.y rel_2 and 17.0.x.y rel_3 are\r\n+verbatim+\r\n::: found [8] tags which are different\r\nref                  | chk                  | pkg-name\r\n--------------------------------------------------------------------------------\r\n00-09-14             | 00-09-15             | reconstruction/muonidentification/muonisolationtools\r\n00-03-62             | 00-03-63             | physicsanalysis/patjobtransforms\r\n00-03-07             | 00-03-11             | trigger/trigvalidation/triganalysistest\r\n00-03-30             | 00-03-34             | trigger/trigvalidation/trigvaltools\r\n00-01-55             | 00-01-60             | physicsanalysis/d3pdmaker/triggerd3pdmaker\r\n00-17-88-01          | 00-17-88-02          | trigger/triggercommon/triggermenupython\r\n00-05-12             | 00-05-20             | trigger/trigvalidation/triggertest\r\n00-00-18             | 00-00-19             | reconstruction/egamma/egammabremrec\r\n--------------------------------------------------------------------------------\r\n-verbatim-\r\ni think only patjobtransforms can cause this problem. \r\nsvn diff of patjobtransforms are \r\nhttps://svnweb.cern.ch/trac/atlasoff/changeset?old_path=%2fphysicsanalysis%2fpatjobtransforms%2ftags%2fpatjobtransforms-00-03-62&amp;old=461059&amp;new_path=%2fphysicsanalysis%2fpatjobtransforms%2ftags%2fpatjobtransforms-00-03-63&amp;new=461059\r\n\r\ncheers, \r\nsusumu ','1658','4045','2011-09-30 18:50:00','5','none','closed','none',';rel_3;callgrindpileuprecodatajobeventloop;callgrindpileuprecodatajobinifirstevent;callgrindpileuprecomcjobeventloop;callgrindpileuprecomcjobinifirstevent;dhatcollisionsrawtoesd_calo;dhatcollisionsrawtoesd_id;dhatcollisionsrawtoesd_muon;fail;verbatim;py:valgrind;info;veto;valgrind;py:valgrind;error;execution;valgrind;exception;catch;local;variable;athenareport;reference;assignment;info;read;error;category;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;share;atlas_error_categories.db;py:valgrind;info;jobtransform;complete;valgrind;error;code;exit;code;py:valgrind;info;transform;timer;stop;valgrind;wall;time;cest;vmem;py:valgrind;info;transform;timer;report;valgrind;wall;time;time;vmem;report;produce;valgrind;version;unknown;command;atla;software;release;17.0.4;atlasreconstruction;17.0.4;installarea;share;valgrind_trf.py;valgrindtool;callgrind;recotrf;reco_trf.py;inputbsfile;user;seuster;referencefile;datastreams_10events;data11_7tev;physics_jettauetmiss.merge.raw;outputesdfile;myesd.pool.root;maxevent;autoconfiguration;preexec;rec.doperfmon;false;rec.dodetailedperfmon;false;rec.dosemidetailedperfmon;false;rec.domonitoring.set_value_and_lock;true;rec.useralgs;valgrindrttjob;vgopts.py;postexec;topsequence.timeout;ignoreunknown;stacktracedepth;trackorigin;false;dumpinstr;true;instratstart;false;valgrindversion;3.6.0;reqcmtconfig;slc5;gcc43;exitcode;errorcode;trf_exc;errorcategory;trf_exc;transform;unknown;exception;fatal;error;warning;errorinfo;acronym;trf_exc;transform;unknown;exception;message;unboundlocalerror;local;variable;athenareport;reference;assignment;severity;fatal;producer;valgrind;backtrace;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;python;pyjobtransformscore;trf.py;line;execute;self;self.runjob;merge;file;atla;software;release;17.0.4;atlasreconstruction;17.0.4;installarea;share;valgrind_trf.py;line;runjob;athenareport.setproducer;self.name;self.version;verbatim;http;atlasrtt.cern.ch;rel_3;build;slc5;gcc43;offline;valgrindrttjob;callgrindpileuprecodatajobeventloop;callgrindpileuprecodatajobeventloop_log;http;atlasrtt.cern.ch;query.php;20atlasproduction;20i686;slc5;gcc43;20valgrindrttjobs;20rel_0;20rel_1;20rel_2;20rel_3;20rel_4;20rel_5;20rel_6;20jobs;difference;rel_2;rel_3;verbatim;different;reconstruction;muonidentification;muonisolationtool;physicsanalysis;patjobtransform;trigger;trigvalidation;triganalysistest;trigger;trigvalidation;trigvaltool;physicsanalysis;d3pdmaker;triggerd3pdmaker;trigger;triggercommon;triggermenupython;trigger;trigvalidation;triggertest;reconstruction;egamma;egammabremrec;verbatim;think;patjobtransform;cause;problem;diff;patjobtransform;http;svnweb.cern.ch;trac;atlasoff;changeset;old_path;2fphysicsanalysis;2fpatjobtransforms;2ftags;2fpatjobtransforms;new_path;2fphysicsanalysis;2fpatjobtransforms;2ftags;2fpatjobtransforms;susumu;valgrindrttjob;fail;patjobtransform;update'),(87327,'atlas','error in recjobtransformtests/mc10_default_short; 17.x.0-val(rel_6)','rtt: error in recjobtransformtests/mc10_default_short; 17.x.0-val(rel_6)\r\n\r\nthis error looks related to the previously reported memory error:\r\nhttps://savannah.cern.ch/bugs/?func=detailitem&amp;item_id=87196\r\n\r\nlogfile snippet:\r\n-----------------\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_6/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputrdofile=root://castoratlas//castor/cern.ch/atlas/atlascerngroupdisk/phys-rig/mc10_7tev.105200.t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00/rdo.197091._000001.pool.root.1 outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root outputhistfile=t1_mcatnlo_jimmy.hist.root maxevents=25 autoconfiguration=everything\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=10\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=17151238\r\nrundiravailabledisk=157373685760\r\nworkdir=/pool/lsf/rtt/174000096\r\nathcode=8\r\nrundiravailabledisk=157373927424\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=16925294\r\natlasrelease=rel_6\r\nwriting jobinfo_recotrf.xml file...\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_6/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/mc10_default_short/mc10_default_short_log\r\n\r\ncommand:\r\n---------\r\n/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_6/installarea/share/bin/reco_trf.py --uploadtoami=1.0 \r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-10-01 17:12:37','5','none','closed','none',';error;recjobtransformtest;mc10_default_short;rel_6;error;look;related;previously;report;memory;error;http;savannah.cern.ch;func;detailitem;item_id;logfile;snippet;report;produce;recotrf;command;atla;software;build;nightly;devval;atlasreconstruction;rel_6;installarea;share;reco_trf.py;uploadtoamus;inputrdofile;root;castoratla;castor;atla;atlascerngroupdisk;mc10_7tev;t1_mcatnlo_jimmy.digit.rdo.e598_s933_s946_d366_tid197091_00;pool.root;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;outputhistfile;t1_mcatnlo_jimmy.hist.root;maxevent;autoconfiguration;exitcode;errorcode;ath_failure;fatal;error;warning;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_6;write;jobinfo_recotrf.xml;file;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_6;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;mc10_default_short;mc10_default_short_log;command;atla;software;build;nightly;devval;atlasreconstruction;rel_6;installarea;share;reco_trf.py;uploadtoamus;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recjobtransformtest;mc10_default_short;rel_6'),(87336,'atlas','error in recexrecotest: 14 crashes related to this bug, 17.x.0-val(rel_6)','rtt: error in recexrecotest, 17.x.0-val(rel_6)\r\n\r\nthis may be related to https://savannah.cern.ch/bugs/?87306\r\n\r\nlogfile snippet:\r\n-----------------\r\ncore dump from coredumpsvc on lxbrb1013.cern.ch at sat oct  1 17:35:52 2011\r\n\r\ncaught signal 11(segmentation fault). details: \r\n  errno = 0, code = 128 (kernel)\r\n  pid   = 0, uid = 0\r\n  value = (0, 0)\r\n  vmem = 2524.18 mb\r\n  rss  = 1931.98 mb\r\n  total-ram = 15929.1 mb\r\n  free-ram  = 914.402 mb\r\n  buffer-ram= 180.355 mb\r\n  total-swap= 4094.55 mb\r\n  free-swap = 3970.87 mb\r\n  addr  = 0\r\n\r\nevent counter: 2\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [179725,631961,21,1302875848:457412525,1093,0x1fffffffffff7]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent algorithm: muoncombitrackbuilder\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   muoncombitrackbuilder\r\n-------------------------------------------------------------------------------------\r\n\r\n(pid=12066 ppid=32547) received fatal signal 11 (segmentation fault)\r\nsignal context:\r\n  signo  = 11, errno = 0, code = 128 (kernel)\r\n  pid    = 0, uid = 0\r\n  value  = (0, (nil))\r\n  addr   = (nil)\r\n  stack  = (2, 0, (nil))\r\n\r\n  03000000 00000000 01000000 00000000 00a082e5 aa2a0000 f014bdf4 ff7f0000\r\n  08000000 00000000 f015bdf4 ff7f0000 01000000 00000000 c0cceca6 072b0000\r\n  0080a802 00000000 01000000 00000000 48d0f51b 00000000 00000000 00388f40\r\n  30000000 00000000 b1000000 00000000 30000000 00000000 2015bdf4 ff7f0000\r\n  b625cca6 072b0000 06020100 00000000 33000000 00000000 00000000 00000000\r\n  0d000000 00000000 00000000 00000000 00000000 00000000 a012bdf4 ff7f0000\r\n  fe3c9db1 10000000 ffffffff 01000000 f1955217 36eb323f 00000000 0000f03f\r\n  b9aaecbb 226a5a40 38726419 00000000 08020000 00000000 00000000 0000f03f\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_6/devval/build/x86_64-slc5-gcc43-opt/offline/recexrecotest/data11_7tev_high_mureco/data11_7tev_high_mureco_log\r\n\r\ncommand:\r\n---------\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','721','5211','2011-10-02 15:13:23','5','duplicate','closed','none',';error;recexrecotest;rel_6;related;http;savannah.cern.ch;logfile;snippet;core;dump;coredumpsvc;lxbrb1013;catch;signal;segmentation;fault;detail;errno;code;kernel;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;algorithm;muoncombitrackbuilder;algorithm;stack;athmasterseq;athalgseq;muoncombitrackbuilder;ppid;receive;fatal;signal;segmentation;fault;signal;context;signo;errno;code;kernel;value;addr;stack;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_6;devval;build;x86_64;slc5;gcc43;offline;recexrecotest;data11_7tev_high_mureco;data11_7tev_high_mureco_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recexrecotest;crash;related;rel_6'),(87344,'atlas','error in recextrigtest, 17.x.0-val(rel_6)','rtt: error in recextrigtest/mc11_esdtrig, 17.x.0-val(rel_6)\r\n\r\nerror in two chains:\r\nmc11_esdtrig\r\nmc10_esdtrig\r\n\r\nlooks related to recent muon issues.\r\n\r\nlogfile snippet:\r\n-----------------\r\ncore dump from coredumpsvc on lxb8765.cern.ch at sat oct  1 17:11:37 2011\r\n\r\ncaught signal 11(segmentation fault). details: \r\n  errno = 0, code = 128 (kernel)\r\n  pid   = 0, uid = 0\r\n  value = (1, 0x1)\r\n  vmem = 2731.4 mb\r\n  rss  = 2132.08 mb\r\n  total-ram = 15925.7 mb\r\n  free-ram  = 83.0312 mb\r\n  buffer-ram= 189.422 mb\r\n  total-swap= 4094.55 mb\r\n  free-swap = 4007.38 mb\r\n  addr  = 0\r\n\r\nevent counter: 1\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [180164,35001,68,1303554730:0,201,0x0]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent trigger chain: ef_mu4_dimu_fs_noos\r\ncurrent algorithm: trigmuoneftrackbuilderconfig_fs\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   trigsteer_ef\r\n   trigmuoneftrackbuilderconfig_fs\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_6/devval/build/x86_64-slc5-gcc43-opt/offline/recextrigtest/mc10_esdtrig/mc10_esdtrig_log\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_6/devval/build/x86_64-slc5-gcc43-opt/offline/recextrigtest/mc11_esdtrig/mc11_esdtrig_log\r\n\r\n\r\ncommand:\r\n---------\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-10-02 21:45:46','5','none','closed','none',';error;recextrigtest;mc11_esdtrig;rel_6;error;chain;mc11_esdtrig;mc10_esdtrig;look;related;recent;muon;issue;logfile;snippet;core;dump;coredumpsvc;lxb8765;catch;signal;segmentation;fault;detail;errno;code;kernel;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;trigger;chain;ef_mu4_dimu_fs_noos;current;algorithm;trigmuoneftrackbuilderconfig_fs;algorithm;stack;athmasterseq;athalgseq;trigsteer_ef;trigmuoneftrackbuilderconfig_fs;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_6;devval;build;x86_64;slc5;gcc43;offline;recextrigtest;mc10_esdtrig;mc10_esdtrig_log;http;atlasrtt.cern.ch;rtt_j_r;rel_6;devval;build;x86_64;slc5;gcc43;offline;recextrigtest;mc11_esdtrig;mc11_esdtrig_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recextrigtest;rel_6'),(87348,'atlas','fpe in deltarassociationtool (recextrigtest/mc11_esdtrig, 17.x.0 (rel_0))','rtt: error in recextrigtest/mc11_esdtrig, 17.x.0 (rel_0)\r\n\r\nlogfile snippet:\r\n-----------------\r\ncore dump from coredumpsvc on lxb8501.cern.ch at sun oct  2 20:41:57 2011\r\n\r\ncaught signal 8(floating point exception). details: \r\n  errno = 0, code = 7 (floating point invalid operation)\r\n  pid   = -642134953, uid = 10922\r\n  value = (0, 0)\r\n  vmem = 3598.57 mb\r\n  rss  = 2893.93 mb\r\n  total-ram = 15925.7 mb\r\n  free-ram  = 3235.57 mb\r\n  buffer-ram= 122.574 mb\r\n  total-swap= 4094.55 mb\r\n  free-swap = 3330.33 mb\r\n  addr  = 0x2aaad9b9cc57\r\n\r\nevent counter: 89\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [180164,35089,68,1303554730:0,187,0x0]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent trigger chain: &lt;none&gt;\r\ncurrent algorithm: singleelectronstream_lowettaufilter\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   singleelectronstream_lowettaufilter\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_0/dev/build/x86_64-slc5-gcc43-opt/offline/recextrigtest/mc11_esdtrig/mc11_esdtrig_log\r\n\r\ncommand:\r\n---------\r\n\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','455','5211','2011-10-03 00:00:21','5','fixed','closed','none',';error;recextrigtest;mc11_esdtrig;rel_0;logfile;snippet;core;dump;coredumpsvc;lxb8501;catch;signal;float;point;exception;detail;errno;code;float;point;invalid;operation;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;trigger;chain;current;algorithm;singleelectronstream_lowettaufilter;algorithm;stack;athmasterseq;athalgseq;singleelectronstream_lowettaufilter;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_0;build;x86_64;slc5;gcc43;offline;recextrigtest;mc11_esdtrig;mc11_esdtrig_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;deltarassociationtool;recextrigtest;mc11_esdtrig;rel_0'),(87350,'atlas','error in recexanatest/mc10_all_nofpe, 17.x.0-val(rel_0)','rtt: error in recexanatest/mc10_all_nofpe, 17.x.0-val(rel_0)\r\n\r\nlogfile snippet:\r\n-----------------\r\nrootdbase.open success aod.pool.root file version:53000\r\nstoragesvc info building shape according to reflection information using shape id for:\r\nstoragesvc info eflowobjectcontainer_p4 [75cc8e85-4ca8-4fe9-a001-26856e3d44c6]\r\nclassidsvc                                           info  getregistryentries: read 44696 clidregistry entries for module all\r\nsrc/tcmalloc.cc:1908] allocation failed: 12\r\nsrc/tcmalloc.cc:1919]  aborting!\r\n\r\n(pid=14470 ppid=2738) received fatal signal 6 (aborted)\r\nsignal context:\r\n  signo  = 6, errno = 0, code = -6 (*unknown reason*)\r\n  pid    = 14470, uid = 24039\r\n  value  = (0, 0x200000000)\r\n  stack  = (2, 0, (nil))\r\n\r\n  80000000 00000000 01010101 01010101 08000000 00000000 02020000 00000000\r\n  08000000 00000000 40afd5a8 7f2b0000 c8d9d7a8 7f2b0000 28cc980f ff7f0000\r\n  86380000 00000000 86380000 00000000 e07c2da9 7f2b0000 00cb980f ff7f0000\r\n  06000000 00000000 00000000 00000000 ffffffff ffffffff 58ca980f ff7f0000\r\n  6502e33c 38000000 02020000 00000000 33000000 00000000 00000000 00000000\r\n  0d000000 00000000 00000000 00000000 00000000 00000000 d0c7980f ff7f0000\r\n  00a1f13c 38000000 05000000 ab2a0000 6014153d 38000000 00000000 00000000\r\n  01000000 00000000 7022a03d 38000000 90c7980f ff7f0000 ed26b5a8 7f2b0000\r\n\r\nlogfile:\r\n---------\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_0/devval/build/x86_64-slc5-gcc43-opt/offline/recexanatest/mc10_all_nofpe/mc10_all_nofpe_log\r\n\r\ncommand:\r\n---------\r\n\r\nplease consult:\r\n----------------\r\nhttps://twiki.cern.ch/twiki/bin/viewauth/atlas/recbuginvestigation\r\n\r\ncheers, \r\nrobert (valshift)','100','5211','2011-10-03 00:49:42','5','none','closed','none',';error;recexanatest;mc10_all_nofpe;rel_0;logfile;snippet;rootdbase.open;success;aod.pool.root;file;version;storagesvc;info;building;shape;accord;reflection;information;shape;storagesvc;info;eflowobjectcontainer_p4;classidsvc;info;getregistryentry;read;clidregistry;entry;module;tcmalloc.cc;allocation;fail;tcmalloc.cc;abort;ppid;receive;fatal;signal;aborted;signal;context;signo;errno;code;unknown;reason;value;stack;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_0;devval;build;x86_64;slc5;gcc43;offline;recexanatest;mc10_all_nofpe;mc10_all_nofpe_log;command;consult;http;twiki.cern.ch;twikus;viewauth;atla;recbuginvestigation;robert;valshift;error;recexanatest;mc10_all_nofpe;rel_0'),(87382,'atlas','rtt:	data_7tev_memleak 			error athena received signal -9 = sigkill. athena was killed, job will terminate. ','hi in\r\n\r\ndata_7tev_memleak 			error athena received signal -9 = sigkill. athena was killed, job will terminate. \r\n\r\n\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_1/installarea/share/bin/reco_trf.py inputbsfile=/afs/cern.ch/atlas/project/rig/referencefiles/datastreams_high_mu/high_mu-data11_7tev.00179725.physics_jettauetmiss.merge.raw._lb0021.data maxevents=300 autoconfiguration=everything outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root --athenaopts=--leak-check-execute\r\nexitcode=10\r\nerrorcode=11000 (ath_failure)\r\nfatals=0\r\nerrors=0\r\nwarnings=10\r\nathacronym=an unknown exception occurred\r\nathcode=8\r\nrundiruseddisk=511463410\r\nrundiravailabledisk=749436219392\r\nworkdir=/pool/lsf/rtt/174521861\r\nathcode=8\r\nrundiravailabledisk=749436452864\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=511249680\r\natlasrelease=rel_1\r\nwriting jobinfo_recotrf.xml file...\r\n\r\n\r\nsee log\r\n\r\nhttps://atlasrtt.cern.ch/rtt/rel_1/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/data_7tev_memleak/data_7tev_memleak_log\r\n\r\ncheers\r\ndaniel\r\n\r\n','100','5211','2011-10-03 14:19:49','5','none','closed','reconstruction',';data_7tev_memleak;error;athena;receive;signal;sigkill;athena;kill;terminate;command;atla;software;build;nightly;atlasproduction;rel_1;installarea;share;reco_trf.py;inputbsfile;atla;project;referencefile;datastreams_high_mu;high_mu;data11_7tev;physics_jettauetmiss.merge.raw;datum;maxevent;autoconfiguration;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;athenaopt;leak;check;execute;exitcode;errorcode;ath_failure;fatal;error;warning;athacronym;unknown;exception;occur;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_1;write;jobinfo_recotrf.xml;file;http;atlasrtt.cern.ch;rel_1;build;slc5;gcc43;offline;recjobtransformtest;data_7tev_memleak;data_7tev_memleak_log;daniel;data_7tev_memleak;error;athena;receive;signal;sigkill;athena;kill;terminate;reconstruction'),(87383,'atlas','excessive warnings reading rel16 aods','\r\nhello,\r\n\r\nthere is a huge number of warnings generated when reading rel16 aods using atlasproduction,17.3.1.1:\r\n\r\nidentifier::identifier - warning constructing 64-bit id with empty upper and non-empty lower: 0 71202000\r\nidentifier::identifier - warning constructing 64-bit id with empty upper and non-empty lower: 0 71204000\r\nidentifier::identifier - warning constructing 64-bit id with empty upper and non-empty lower: 0 71205000\r\n\r\nand this command:\r\n\r\n$ reco_trf.py inputaodfile=data11_7tev.00186877.physics_muons.merge.aod.f394_m934._lb0301._0001.1 outputtagfile=tag.pool.root autoconfiguration=everything\r\n\r\ni posted the input file here:\r\n\r\nlxplus421: /tmp/data11_7tev.00186877.physics_muons.merge.aod.f394_m934._lb0301._0001.1\r\n\r\ndoes this mean that i can not use release 17 to read release 16 aods?\r\n\r\nthanks,\r\nrustem','100','1974','2011-10-03 14:33:59','5','invalid','closed','reconstruction',';hello;huge;number;warning;generate;reading;rel16;atlasproduction;17.3.1.1;identifier;identifier;warn;construct;upper;lower;identifier;identifier;warn;construct;upper;lower;identifier;identifier;warn;construct;upper;lower;command;reco_trf.py;inputaodfile;data11_7tev;physics_muons.merge.aod.f394_m934;outputtagfile;tag.pool.root;autoconfiguration;post;input;file;lxplus421;data11_7tev;physics_muons.merge.aod.f394_m934;mean;release;read;release;rustem;excessive;warning;reading;rel16;reconstruction'),(87388,'atlas','error in athenaapp_all_rdo myesd.pool.root info database being retired...','dear experts,\r\n\r\nin todays 17.0.x.y (x86_64-slc5-gcc43-opt) rel_1 there was an\r\nerror in athenaapp_all_rdo myesd.pool.root info database being retired...\r\n\r\nsee logfile:\r\nhttps://atlasrtt.cern.ch/rtt/rel_1/17.0.x.y/build/x86_64-slc5-gcc43-opt/offline/recperftests/athenaapp_all_rdo/athenaapp_all_rdo_log\r\n\r\npersistencysvc::globaltransaction error pfn = myesd.pool.root\r\npoolsvc                                             error pool commit failed 0xda74ab0\r\nmyesd.pool.root info database being retired...\r\npoolsvc                                             fatal in sysstop(): standard std::exception is caught\r\npoolsvc                                             error std::bad_alloc\r\nservicemanager                                      error unable to stop service: poolsvc\r\napplicationmgr                                      fatal finalize: invalid state &quot;running&quot;\r\napplicationmgr                                      fatal terminate: invalid state &quot;running&quot;\r\npy:athena            info leaving with code 97: &quot;failure in finalization&quot;\r\nmyesd.pool.root info database being retired...\r\npoolsvc                                             fatal in sysstop(): standard std::exception is caught\r\npoolsvc                                             error std::bad_alloc\r\nservicemanager                                      error unable to stop service: poolsvc\r\ncoralapplication info delete the cool coralapplication...\r\ncoralapplication info delete the cool database service\r\nraldatabasesvc info delete the raldatabasesvc...\r\nraldatabasesvc info purge the connection pool\r\nraldatabasesvc info reset the ics pointer\r\nraldatabasesvc info delete the raldatabasesvc... done\r\ncoralapplication info delete the coral connection service\r\ncoralapplication info delete the cool coralapplication... done\r\ncoral/services/connectionservice info deleting the connectionpool\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code 97.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/share/atlas_error_categories.db\r\ninfo: read 53 error patterns from file /afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/releases/17.0.4/atlascore/17.0.4/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: mon oct 03 12:35:57 2011 cest, vmem: 281.7734375 mb, rss: 95.71484375 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_1 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info done scanning 55137 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 0\r\npy:rawtoesd          info    warnings: 252\r\npy:rawtoesd          info    errors  : 53\r\npy:rawtoesd          info    fatals  : 8\r\nathenalogchecker stop [rawtoesd log] wall-time: mon oct 03 12:36:03 2011 cest, vmem: 281.7734375 mb, rss: 95.71484375 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 05 secs, cpu-time: 00 hrs 00 mins 05 secs, vmem: 0.0 mb, rss: 0.0 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: mon oct 03 12:36:03 2011 cest, vmem: 281.7734375 mb, rss: 95.75 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 11 mins 07 secs, cpu-time: 00 hrs 10 mins 57 secs, vmem: 0.0 mb, rss: 0.21875 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.4/atlasreconstruction/17.0.4/installarea/share/bin/rawtoesd_trf.py --useathenaapp=true inputrdofile=/afs/cern.ch/atlas/project/rig/referencefiles/rttinputfiles/rdo.039414._00001.pool.root.1 maxevents=250 autoconfiguration=everything --athenaopts=--stdcmalloc preexec=from@perfmoncomps.perfmonflags@import@jobproperties@as@pmjp,,pmjp.perfmonflags.domallocmonitoring=true,,pmjp.perfmonflags.dopostprocessing=true,,pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true outputdqmonitorfile=mymon.root outputesdfile=myesd.pool.root outputntup_muoncalibfile=myntup_muoncalib.root\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (61 times)\r\nfatals=7\r\nerrors=10\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=out of memory: saving summary ...\r\n  severity=fatal\r\n  who=athenasummarysvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput - caught exception: std::bad_alloc\r\n  severity=error\r\n  who=athenapoolcnvsvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=unable to commit output myesd.pool.root\r\n  severity=error\r\n  who=streamesd.streamesdtool\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed.\r\n  severity=fatal\r\n  who=streamesd\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=terminating event processing loop due to errors\r\n  severity=error\r\n  who=athenaeventloopmgr\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=failed to &quot;executerun(250)&quot;\r\n  severity=error\r\n  who=py:athena\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=athenapoolconverter createrep failed to get token, key = streamesd\r\n  severity=error\r\n  who=athenapoolconverter\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=could not create rep for dataobject (clid/key):167728019 streamesd\r\n  severity=error\r\n  who=streamesd_fh.streamesd_fhtool\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=athenapoolconverter createrep failed to get token, key = incompletelumiblocks\r\n  severity=error\r\n  who=athenapoolconverter\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=could not create rep for dataobject (clid/key):210948284 incompletelumiblocks\r\n  severity=error\r\n  who=streamesd_fh.streamesd_fhtool\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=athenapoolconverter createrep failed to get token, key = eventbookkeepers\r\n  severity=error\r\n  who=athenapoolconverter\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=could not create rep for dataobject (clid/key):1094136276 eventbookkeepers\r\n  severity=error\r\n  who=streamesd_fh.streamesd_fhtool\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=streamobjects failed.\r\n  severity=fatal\r\n  who=streamesd_fh\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=commitoutput failed.\r\n  severity=fatal\r\n  who=streamesd_fh\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo (count=2)\r\n  acronym=trf_unknown (69999)\r\n  message=in sysstop(): standard std::exception is caught\r\n  severity=fatal\r\n  who=poolsvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=finalize: invalid state &quot;running&quot;\r\n  severity=fatal\r\n  who=applicationmgr\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=terminate: invalid state &quot;running&quot;\r\n  severity=fatal\r\n  who=applicationmgr\r\n  producer=rawtoesd\r\n  stage=execute\r\nworkdir=/pool/lsf/rtt/174530242\r\nathcode=97\r\nrundiravailabledisk=132535676928\r\ndbrelease=\r\nathacronym=failure in finalization\r\nrundiruseddisk=29396600\r\natlasrelease=rel_1\r\nwriting jobinfo_rawtoesd.xml file...\r\n','100','5211','2011-10-03 16:18:40','5','none','closed','none',';dear;expert;today;x86_64;slc5;gcc43;rel_1;error;athenaapp_all_rdo;myesd.pool.root;info;database;retire;logfile;http;atlasrtt.cern.ch;rel_1;build;x86_64;slc5;gcc43;offline;recperftest;athenaapp_all_rdo;athenaapp_all_rdo_log;persistencysvc;globaltransaction;error;myesd.pool.root;poolsvc;error;pool;commit;fail;myesd.pool.root;info;database;retire;poolsvc;fatal;sysstop;standard;exception;catch;poolsvc;error;bad_alloc;servicemanager;error;unable;stop;service;poolsvc;applicationmgr;fatal;finalize;invalid;state;applicationmgr;fatal;terminate;invalid;state;py:athena;info;leave;code;failure;finalization;myesd.pool.root;info;database;retire;poolsvc;fatal;sysstop;standard;exception;catch;poolsvc;error;bad_alloc;servicemanager;error;unable;stop;service;poolsvc;coralapplication;info;delete;cool;coralapplication;coralapplication;info;delete;cool;database;service;raldatabasesvc;info;delete;raldatabasesvc;raldatabasesvc;info;purge;connection;pool;raldatabasesvc;info;reset;pointer;raldatabasesvc;info;delete;raldatabasesvc;coralapplication;info;delete;coral;connection;service;coralapplication;info;delete;cool;coralapplication;coral;service;connectionservice;info;delete;connectionpool;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;info;read;error;category;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;release;17.0.4;atlascore;17.0.4;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_1;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;release;17.0.4;atlasreconstruction;17.0.4;installarea;share;rawtoesd_trf.py;useathenaapp;true;inputrdofile;atla;project;referencefile;rttinputfile;pool.root;maxevent;autoconfiguration;athenaopt;stdcmalloc;preexec;perfmoncomps.perfmonflags;import;jobproperty;pmjp;pmjp.perfmonflags.domallocmonitoring;true;pmjp.perfmonflags.dopostprocessing;true;pmjp.perfmonflags.dosemidetailedmonitoringfullprint;true;outputdqmonitorfile;mymon.root;outputesdfile;myesd.pool.root;outputntup_muoncalibfile;myntup_muoncalib.root;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;memory;save;summary;severity;fatal;athenasummarysvc;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;commitoutput;catch;exception;bad_alloc;severity;error;athenapoolcnvsvc;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;unable;commit;output;myesd.pool.root;severity;error;streamesd.streamesdtool;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;commitoutput;fail;severity;fatal;streamesd;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;terminate;event;processing;loop;error;severity;error;athenaeventloopmgr;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;fail;executerun;severity;error;py:athena;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;athenapoolconverter;createrep;fail;token;streamesd;severity;error;athenapoolconverter;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;create;dataobject;clid;streamesd;severity;error;streamesd_fh.streamesd_fhtool;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;athenapoolconverter;createrep;fail;token;incompletelumiblock;severity;error;athenapoolconverter;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;create;dataobject;clid;incompletelumiblock;severity;error;streamesd_fh.streamesd_fhtool;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;athenapoolconverter;createrep;fail;token;eventbookkeeper;severity;error;athenapoolconverter;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;create;dataobject;clid;eventbookkeeper;severity;error;streamesd_fh.streamesd_fhtool;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;streamobject;fail;severity;fatal;streamesd_fh;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;commitoutput;fail;severity;fatal;streamesd_fh;producer;rawtoesd;stage;execute;errorinfo;count;acronym;trf_unknown;message;sysstop;standard;exception;catch;severity;fatal;poolsvc;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;finalize;invalid;state;severity;fatal;applicationmgr;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;terminate;invalid;state;severity;fatal;applicationmgr;producer;rawtoesd;stage;execute;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;failure;finalization;rundiruseddisk;atlasrelease;rel_1;write;jobinfo_rawtoesd.xml;file;error;athenaapp_all_rdo;myesd.pool.root;info;database;retire'),(87389,'atlas','rtt fatal error in all_rdo (trf_segfault)','dear experts,\r\n\r\nin todays rel_1 there was an error in all_rdo trf_segfault \r\n\r\nsee logfile:\r\nhttps://atlasrtt.cern.ch/rtt/rel_1/17.0.x.y-val/build/x86_64-slc5-gcc43-opt/offline/recperftests/all_rdo/all_rdo_log\r\n\r\nstoragesvc info trigpassbitscollection_p1 [e6e4f396-d696-4c1c-a0c4-7766e0af9bb5]\r\n-------------------------------------------------------------------------------------\r\ncore dump from coredumpsvc on lxb8775.cern.ch at mon oct  3 13:15:02 2011\r\n\r\ncaught signal 11(segmentation fault). details: \r\n  errno = 0, code = 1 (address not mapped to object)\r\n  pid   = 24, uid = 0\r\n  value = (-1436295248, 0x7fffaa63dfb0)\r\n  vmem = 3000 mb\r\n  rss  = 2344.42 mb\r\n  total-ram = 15925.7 mb\r\n  free-ram  = 1873.36 mb\r\n  buffer-ram= 213.277 mb\r\n  total-swap= 20481.2 mb\r\n  free-swap = 20481 mb\r\n  addr  = 0x18\r\n\r\nevent counter: 1\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [5200,30002,0,0:0,0,0x0]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent algorithm: streamesd\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   streamesd\r\n-------------------------------------------------------------------------------------\r\n\r\n(pid=26072 ppid=25914) received fatal signal 11 (segmentation fault)\r\nsignal context:\r\n  signo  = 11, errno = 0, code = 1 (address not mapped to object)\r\n  pid    = 24, uid = 0\r\n  value  = (-1436295248, 0x7fffaa63dfb0)\r\n  addr   = 0x18\r\n  stack  = (2, 0, (nil))\r\n\r\n  ffffffff 00000000 00000000 00000000 22000000 00000000 46020000 00000000\r\n  30e363aa ff7f0000 87210000 00000000 e0e363aa ff7f0000 2fe463aa ff7f0000\r\n  00000000 00000000 18010000 00000000 e0e1824d d62a0000 50034d34 ab2a0000\r\n  18000000 00000000 00000000 00000000 b0ffffff ffffffff 90e163aa ff7f0000\r\n  73cc824d d62a0000 06020100 00000000 33000000 00000000 06000000 00000000\r\n  0e000000 00000000 00000000 00000000 18000000 00000000 10df63aa ff7f0000\r\n  8090224d d62a0000 b0e263aa ff7f0000 ce5c834d d62a0000 7008c81f 00000000\r\n  00000000 ff7f0000 dbdf63aa ff7f0000 2e4ee73c 38000000 d8e44c34 ab2a0000\r\n\r\nstack trace: .....\r\n .......\r\nathenalogchecker start [rawtoesd log] wall-time: mon oct 03 13:15:31 2011 cest, vmem: 281.78515625 mb, rss: 95.7265625 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_1 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info athena execute()...\r\npy:rawtoesd          info done scanning 54795 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 0\r\npy:rawtoesd          info    warnings: 68\r\npy:rawtoesd          info    errors  : 0\r\npy:rawtoesd          info    fatals  : 2\r\nathenalogchecker stop [rawtoesd log] wall-time: mon oct 03 13:15:37 2011 cest, vmem: 281.78515625 mb, rss: 95.7265625 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 05 secs, cpu-time: 00 hrs 00 mins 05 secs, vmem: 0.0 mb, rss: 0.0 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 60010 (exit code 6)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: mon oct 03 13:15:37 2011 cest, vmem: 281.78515625 mb, rss: 95.76171875 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 08 mins 12 secs, cpu-time: 00 hrs 08 mins 35 secs, vmem: 0.0 mb, rss: 0.21484375 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/releases/17.0.4/atlasreconstruction/17.0.4/installarea/share/bin/rawtoesd_trf.py inputrdofile=/afs/cern.ch/atlas/project/rig/referencefiles/rttinputfiles/rdo.039414._00001.pool.root.1 maxevents=250 autoconfiguration=everything --athenaopts=--stdcmalloc preexec=from@perfmoncomps.perfmonflags@import@jobproperties@as@pmjp,,pmjp.perfmonflags.domallocmonitoring=true,,pmjp.perfmonflags.dopostprocessing=true,,pmjp.perfmonflags.dosemidetailedmonitoringfullprint=true outputdqmonitorfile=mymon.root outputesdfile=myesd.pool.root outputntup_muoncalibfile=myntup_muoncalib.root\r\nexitcode=6\r\nerrorcode=60010 (trf_segfault)\r\nerrorcategory=trf_segfault,	60010,	segmentation fault (2 times)\r\nfatals=2\r\nerrors=0\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_segfault (60010) segmentation fault\r\n  message=caught signal 11(segmentation fault). details:\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_segfault (60010) segmentation fault\r\n  message=(pid=26072 ppid=25914) received fatal signal 11 (segmentation fault)\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  stage=execute\r\nworkdir=/pool/lsf/rtt/174536641\r\nathcode=8\r\nrundiravailabledisk=140175110144\r\ndbrelease=\r\nathacronym=an unknown exception occurred\r\nrundiruseddisk=14203471\r\natlasrelease=rel_1\r\nwriting jobinfo_rawtoesd.xml file...\r\n\r\n\r\n\r\ncheers michael','100','5211','2011-10-03 16:25:59','5','none','closed','none',';dear;expert;today;rel_1;error;all_rdo;trf_segfault;logfile;http;atlasrtt.cern.ch;rel_1;build;x86_64;slc5;gcc43;offline;recperftest;all_rdo;all_rdo_log;storagesvc;info;trigpassbitscollection_p1;core;dump;coredumpsvc;lxb8775;catch;signal;segmentation;fault;detail;errno;code;address;object;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;algorithm;streamesd;algorithm;stack;athmasterseq;athalgseq;streamesd;ppid;receive;fatal;signal;segmentation;fault;signal;context;signo;errno;code;address;object;value;addr;stack;stack;trace;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_1;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;athena;execute;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;release;17.0.4;atlasreconstruction;17.0.4;installarea;share;rawtoesd_trf.py;inputrdofile;atla;project;referencefile;rttinputfile;pool.root;maxevent;autoconfiguration;athenaopt;stdcmalloc;preexec;perfmoncomps.perfmonflags;import;jobproperty;pmjp;pmjp.perfmonflags.domallocmonitoring;true;pmjp.perfmonflags.dopostprocessing;true;pmjp.perfmonflags.dosemidetailedmonitoringfullprint;true;outputdqmonitorfile;mymon.root;outputesdfile;myesd.pool.root;outputntup_muoncalibfile;myntup_muoncalib.root;exitcode;errorcode;trf_segfault;errorcategory;trf_segfault;segmentation;fault;time;fatal;error;warning;errorinfo;acronym;trf_segfault;segmentation;fault;message;catch;signal;segmentation;fault;detail;severity;fatal;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_segfault;segmentation;fault;message;ppid;receive;fatal;signal;segmentation;fault;severity;fatal;producer;rawtoesd;stage;execute;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;unknown;exception;occur;rundiruseddisk;atlasrelease;rel_1;write;jobinfo_rawtoesd.xml;file;michael;fatal;error;all_rdo;trf_segfault'),(87408,'atlas','libegammaenumsdict[.so | .dll | .dylib | .sl | .dl | .a] does not exist','hello,\r\n\r\nthere are many jobs that failed due to:\r\n\r\n libegammaenumsdict[.so | .dll | .dylib | .sl | .dl | .a] does not exist\r\n\r\nerror in rel_2 tct tests for dev/x86_64-slc5-gcc43-opt\r\n\r\nfull log example is at:\r\nhttps://atlasrtt.cern.ch/tct/rel_2/dev/build/x86_64-slc5-gcc43-opt/offline/tier0chaintests/latestrun_jettauetmiss1/latestrun_jettauetmiss1_log\r\n\r\nthanks, marija\r\nsw val shifter','100','5211','2011-10-04 09:27:22','5','none','closed','reconstruction',';hello;fail;libegammaenumsdict;dylib;exist;error;rel_2;test;x86_64;slc5;gcc43;example;http;atlasrtt.cern.ch;rel_2;build;x86_64;slc5;gcc43;offline;tier0chaintest;latestrun_jettauetmiss1;latestrun_jettauetmiss1_log;marija;shifter;libegammaenumsdict;dylib;exist;reconstruction'),(87431,'atlas','execute of [beambackgroundfiller] did not succeed','hello,\r\n\r\nthere are some failures in tct tests for 17.0.x.y-val due to beambackgroundfiller.\r\nthe following jobs failed:\r\n\r\nrun00169564_express0_hicollisions\r\nrun00169741_highpt2_hicollisions \r\nrun00169741_highpt_hicollisions \r\n\r\nlog file example available at:\r\n\r\nhttps://atlasrtt.cern.ch/tct/rel_3/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/tier0chaintests/run00169564_express0_hicollisions/run00169564_express0_hicollisions_log\r\n\r\ncheers, marija\r\nsw val shifter','100','5211','2011-10-05 08:52:33','5','none','closed','reconstruction',';hello;failure;test;beambackgroundfiller;follow;fail;run00169564_express0_hicollisions;run00169741_highpt2_hicollisions;run00169741_highpt_hicollisions;file;example;available;http;atlasrtt.cern.ch;rel_3;build;slc5;gcc43;offline;tier0chaintest;run00169564_express0_hicollisions;run00169564_express0_hicollisions_log;marija;shifter;execute;beambackgroundfiller;succeed;reconstruction'),(87433,'atlas','trf_outfile_notread (64120) transform output file not readable','hello,\r\n\r\nthere is a crash in rel_3 tct for  17.0.x.y due to trf_outfile_notread errors.\r\n\r\nfull log file available at:\r\n\r\nhttps://atlasrtt.cern.ch/tct/rel_3/17.0.x.y/build/i686-slc5-gcc43-opt/offline/tier0chaintests/run00183021_express2_collisions_with_3rd_muon_chain/run00183021_express2_collisions_with_3rd_muon_chain_log\r\n\r\ncheers, marija\r\nsw val shifter','100','5211','2011-10-05 09:01:40','5','none','closed','reconstruction',';hello;crash;rel_3;trf_outfile_notread;error;file;available;http;atlasrtt.cern.ch;rel_3;build;slc5;gcc43;offline;tier0chaintest;run00183021_express2_collisions_with_3rd_muon_chain;run00183021_express2_collisions_with_3rd_muon_chain_log;marija;shifter;trf_outfile_notread;transform;output;file;readable;reconstruction'),(87483,'atlas','egammagetter.py::configure:   error could not get handle to emrefit','hi,\r\nthere is a crash egammagetter.py::configure:   error could not get handle to emrefit in atn for rel_4\r\n\r\n\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x0val64bs5g4recopt/nicos_testlog_rel_4/reconstruction_recexample_recexrecotest_26__recexrecotest_testconfiguration__recoshift_esd__a.html\r\n\r\n\r\ncheers, marija\r\nsw val shifter','100','5211','2011-10-06 10:50:59','5','none','closed','none',';crash;egammagetter.py;configure;error;handle;emrefit;rel_4;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x0val64bs5g4recopt;nicos_testlog_rel_4;reconstruction_recexample_recexrecotest_26__recexrecotest_testconfiguration__recoshift_esd__a.html;marija;shifter;egammagetter.py;configure;error;handle;emrefit'),(87511,'atlas','no suitable top tree level for countig of events','https://atlasrtt.cern.ch/rtt/rel_4/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/mc10_default_short/mc10_default_short_log\r\npy:rawtoesd         error during execution of rawtoesd, exception caught: found no suitable top level tree for counting of events\r\n\r\n','100','5211','2011-10-06 18:35:39','5','duplicate','closed','reconstruction',';http;atlasrtt.cern.ch;rel_4;build;slc5;gcc43;offline;recjobtransformtest;mc10_default_short;mc10_default_short_log;py:rawtoesd;error;execution;rawtoesd;exception;catch;suitable;level;tree;count;event;suitable;tree;level;countig;event;reconstruction'),(87513,'atlas','cannot retrieve muondetdescrmgr ','https://atlasrtt.cern.ch/rtt/rel_4/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/recperftests/calo_bs/calo_bs_log\r\ntoolsvc.muoncalib::idtofixedidtool                  error  cannot retrieve muondetdescrmgr \r\ntoolsvc                                             error error initializing tool \'toolsvc.muoncalib::idtofixedidtool\'\r\nbeambackgroundfiller                                fatal ../src/beambackgroundfiller.cxx:63 (virtual statuscode beambackgroundfiller::initialize()): code 0: m_idtofixedidtool.retrieve()\r\n','1682','5211','2011-10-06 18:40:06','5','fixed','closed','reconstruction',';http;atlasrtt.cern.ch;rel_4;build;slc5;gcc43;offline;recperftest;calo_bs;calo_bs_log;toolsvc.muoncalib;idtofixedidtool;error;retrieve;muondetdescrmgr;toolsvc;error;error;initialize;tool;toolsvc.muoncalib;idtofixedidtool;beambackgroundfiller;fatal;beambackgroundfiller.cxx;virtual;statuscode;beambackgroundfiller;initialize;code;m_idtofixedidtool.retrieve;retrieve;muondetdescrmgr;reconstruction'),(87514,'atlas','antikt4topoemjets could not be retrieved from storegate !','https://atlasrtt.cern.ch/rtt/rel_4/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/recjobtransformtests/data_heavy_ion/data_heavy_ion_log\r\nmessage=antikt4topoemjets could not be retrieved from storegate !\r\n','100','5211','2011-10-06 18:45:11','5','duplicate','closed','reconstruction',';http;atlasrtt.cern.ch;rel_4;build;slc5;gcc43;offline;recjobtransformtest;data_heavy_ion;data_heavy_ion_log;message;antikt4topoemjet;retrieve;storegate;antikt4topoemjet;retrieve;storegate;reconstruction'),(87520,'atlas','jetmonmanager slow at high pileup','\r\njetmonmanager seems to have really bad scaling behaviour under pileup, and at mu=40 (in ttbar mc) it uses ~350 ms/evt:\r\n\r\nhttps://atlas-pmb.web.cern.ch/atlas-pmb/temp/pileup_sep2011_details/#plot22\r\n\r\nto help investigate this a bit further, here is a callgrind file (gunzip and then open and investigate with &quot;kcachegrind&quot;) of just the dq monitoring algs in the rawtoesd eventloop:\r\n\r\nhttp://atlas-pmb.web.cern.ch/atlas-pmb/temp/callgrind_mu40ttbar_oct5/run_dqmon_rawtoesd_callgrind.out.1238.gz\r\n\r\na quick glance tells me that 2/3 of the jet monitoring time is spent in jetrecmontoolbase::isgoodjet, with most time spent in the two calls to jetcaloqualityutils::jetquality(..) and jetcaloqualityutils::jettimecells(..).\r\n\r\nseveral points, (#1 and #5 likely to affect performance most, the others just other issues i noticed after a quick glance):\r\n\r\n1) are you sure you are only calling this method exactly once per event for a given jet? if not, perhaps the problem can be solved with caching.\r\n\r\n2) why is such a hugh method inlined? it should be in the .cxx.\r\n\r\n3) why is the library dual_use and not components?\r\n\r\n4) i see unprotected debug messages.\r\n\r\n5) i see a hardcoded &quot;recalculatequantities = true;&quot; which results in the calls to the jetquality and jettimecells methods.\r\n\r\n6) in the copyvectors method just below, a reserve call should be added.\r\n\r\ncheers,\r\nthomas','5528','916','2011-10-07 08:09:32','5','fixed','closed','none',';jetmonmanager;really;scale;behaviour;pileup;ttbar;http;atla;pmb.web.cern.ch;atla;temp;pileup_sep2011_details;plot22;help;investigate;callgrind;file;gunzip;open;investigate;kcachegrind;just;monitoring;rawtoesd;eventloop;http;atla;pmb.web.cern.ch;atla;temp;callgrind_mu40ttbar_oct5;run_dqmon_rawtoesd_callgrind.out;quick;glance;tell;monitoring;time;spend;jetrecmontoolbase;isgoodjet;time;spend;call;jetcaloqualityutil;jetquality;jetcaloqualityutil;jettimecell;point;likely;affect;performance;just;issue;notice;quick;glance;sure;call;method;exactly;event;give;problem;solve;caching;hugh;method;inline;library;dual_use;component;unprotected;debug;message;hardcode;recalculatequantity;true;result;call;jetquality;jettimecell;method;copyvector;method;just;reserve;thoma;jetmonmanager;slow;high;pileup'),(87581,'atlas','error: compile ../src/sharedhitmapper.cxx on lxbrb0607.cern.ch/8,gssapi failed','hello,\r\n\r\nthere is a crash in jettagtools in 17.x.0-val:\r\n\r\nin file included from ../src/sharedhitmapper.cxx:1:\r\n/build/atnight/localbuilds/nightlies/17.x.0-val/atlasreconstruction/rel_0/installarea/include/jettagtools/jettagtools/sharedhitmapper.h:73:44: error: storegate/tools/classid_traits.h: no such file or directory\r\ndistcc[25517] error: compile (null) on localhost failed\r\n\r\n\r\nfull log file at:\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x0val64bs5g4recopt/nicos_log_rel_0/physicsanalysis_jettagging_jettagtools.html\r\n\r\n\r\ncheers, marija\r\nsw val shifter\r\n','100','5211','2011-10-09 09:49:44','5','fixed','closed','btag',';hello;crash;jettagtool;file;include;sharedhitmapper.cxx;build;atnight;localbuild;nightly;atlasreconstruction;rel_0;installarea;include;jettagtool;jettagtool;sharedhitmapper.h;error;storegate;tool;classid_traits.h;file;directory;distcc;error;compile;null;localhost;fail;file;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x0val64bs5g4recopt;nicos_log_rel_0;physicsanalysis_jettagging_jettagtools.html;marija;shifter;error;compile;sharedhitmapper.cxx;lxbrb0607;gssapus;fail;btag'),(87582,'atlas','crash in recexrecotest_testconfiguration#recoshift_esd','hello,\r\n\r\nthere is a crash since in recexrecotest_testconfiguration 17.x.0-val due to metrefalg.\r\n\r\nlog file available at:\r\n\r\nhttp://atlas-computing.web.cern.ch/atlas-computing/links/distdirectory/nightlies/developmentwebarea/nicos_web_area17x0val64bs5g4recopt/nicos_testlog_rel_0/reconstruction_recexample_recexrecotest_26__recexrecotest_testconfiguration__recoshift_esd__a.html\r\n\r\ncheers, marija\r\nsw val shifter','100','5211','2011-10-09 10:02:02','5','none','closed','missinget',';hello;crash;recexrecotest_testconfiguration;metrefalg;file;available;http;atla;computing.web.cern.ch;atla;computing;link;distdirectory;nightly;developmentwebarea;nicos_web_area17x0val64bs5g4recopt;nicos_testlog_rel_0;reconstruction_recexample_recexrecotest_26__recexrecotest_testconfiguration__recoshift_esd__a.html;marija;shifter;crash;recexrecotest_testconfiguration;recoshift_esd;missinget'),(87592,'atlas','segmentation fault due to antikt6towerjets ','hello,\r\n\r\nthere are many crashes in tcts for 16.6.x.y-val, rel_1 due to  antikt6towerjets.\r\nexample log file available at:\r\n\r\nhttps://atlasrtt.cern.ch/tct/rel_1/16.6.x.y.z-val/build/i686-slc5-gcc43-opt/atlasphysics/tier0chaintests/latestrun_l1calo1/latestrun_l1calo1_log\r\n\r\ncheers, marija\r\nsw val shifter','5297','5211','2011-10-10 08:44:24','5','none','closed','jet',';hello;crash;rel_1;antikt6towerjet;example;file;available;http;atlasrtt.cern.ch;rel_1;x.y.z;build;slc5;gcc43;atlasphysic;tier0chaintest;latestrun_l1calo1;latestrun_l1calo1_log;marija;shifter;segmentation;fault;antikt6towerjet'),(87637,'atlas','d2aod_dipho crashes in egammatools and egammatracktools','there are crashes in d3pd-making when running on d2aod_dipho samples.  the crashes appear when running on this file:\r\n\r\ndata11_7tev.00187219.physics_egamma.merge.d2aod_dipho.r2713_p705_p682_tid527720_00/d2aod_dipho.527720._000038.pool.root.1\r\n\r\nyou\'ll need to check out and build photonanalysisutils-00-03-20-01 to get past some crashes in that package if testing with a release that contains an older pau tag.\r\n\r\nthe crashes are avoided with the following changes (basically just adding protection against null pointers and invalid eta values):\r\n\r\n\r\n\r\natlas16 | egammatools-00-04-44 [96]: svn diff\r\nindex: src/egammaoqflagsbuilder.cxx\r\n===================================================================\r\n--- src/egammaoqflagsbuilder.cxx        (revision 462395)\r\n+++ src/egammaoqflagsbuilder.cxx        (working copy)\r\n@@ -178,6 +178,7 @@\r\n \r\n   for( ;celliter!=celliterend;celliter++) {\r\n     const calocell* cell = (*celliter);\r\n+    if(!cell) continue;\r\n     float eta = cell-&gt;eta();\r\n     float phi = cell-&gt;phi();\r\n     float energy = cell-&gt;energy();\r\n@@ -240,17 +241,19 @@\r\n   \r\n   // protection against bad pointers\r\n   if (eg==0) return statuscode::success;\r\n-  if (eg-&gt;cluster()-&gt;getnumberofcells()&lt;=0) return statuscode::success;\r\n+  const calocluster* cluster = eg-&gt;cluster();\r\n \r\n+  if (!cluster) return statuscode::success;\r\n+  if (cluster-&gt;getnumberofcells()&lt;=0) return statuscode::success;\r\n+\r\n   unsigned int iflag = 0;\r\n-  const calocluster* cluster = eg-&gt;cluster();\r\n   float clustereta = cluster-&gt;eta();\r\n \r\n   //find the central cell in the middle layer \r\n   bool foundcentralcell = egammaoqflagsbuilder::findcentralcell(cluster);\r\n+\r\n   \r\n \r\n-\r\n   //set timing bit\r\n   double absenergygev = fabs(cluster-&gt;e()/gev);\r\n   if(absenergygev != 0 &amp;&amp;  fabs(cluster-&gt;gettime()) &gt; m_tcut + m_tcutvse/absenergygev) {\r\n@@ -274,6 +277,7 @@\r\n     for( ;celliter!=celliterend;celliter++) {\r\n       //find cell parameters and properties\r\n       const calocell* cell = (*celliter);\r\n+      if(!cell) continue;\r\n       float eta = cell-&gt;eta();\r\n       // float phi = cell-&gt;phi(); // no longer used\r\n       float qual = cell-&gt;quality();\r\n\r\n\r\n\r\n\r\natlas16 | egammatracktools-00-00-68 [99]: svn diff\r\nindex: src/emextrapolcaloconversion.cxx\r\n===================================================================\r\n--- src/emextrapolcaloconversion.cxx    (revision 462395)\r\n+++ src/emextrapolcaloconversion.cxx    (working copy)\r\n@@ -104,11 +104,17 @@\r\n   \r\n   if (fabs(eta_cluster_2nd_sampling) &lt; 1.5) {      // barrel\r\n     r_photon_front=m_egammaqdepth-&gt;radius(calocell_id::emb1,eta_cluster_1st_sampling,phi_cluster_2nd_sampling);\r\n-    z_photon_front=r_photon_front*sinh(eta_cluster_1st_sampling);\r\n+    if(fabs(eta_cluster_1st_sampling)&lt;1.5)\r\n+      z_photon_front=r_photon_front*sinh(eta_cluster_1st_sampling);\r\n+    else\r\n+      return statuscode::success;\r\n   }\r\n   else if (fabs(eta_cluster_1st_sampling) &lt; 5) {  // endcap\r\n     z_photon_front=m_egammaqdepth-&gt;radius(calocell_id::eme1,eta_cluster_1st_sampling,phi_cluster_2nd_sampling);\r\n-    r_photon_front=z_photon_front/sinh(eta_cluster_1st_sampling);\r\n+    if(fabs(eta_cluster_1st_sampling)&lt;1.5)\r\n+      r_photon_front=z_photon_front/sinh(eta_cluster_1st_sampling);\r\n+    else\r\n+      return statuscode::success;\r\n   }\r\n   else\r\n     return statuscode::success;\r\n\r\n\r\n\r\nfor posterity: these crashes are related to this bug:\r\n\r\nhttps://savannah.cern.ch/bugs/?87586\r\n\r\n\r\n\r\ncheers,\r\n\r\n-mike\r\n','5259','1544','2011-10-10 20:58:48','5','none','closed','egamma',';crash;d3pd;make;d2aod_dipho;sample;crash;appear;file;data11_7tev;physics_egamma.merge.d2aod_dipho.r2713_p705_p682_tid527720_00;d2aod_dipho;pool.root;you\'ll;need;check;build;photonanalysisutil;past;crash;package;testing;release;contain;older;crash;avoid;follow;change;basically;just;protection;null;pointer;invalid;value;atlas16;egammatool;diff;index;egammaoqflagsbuilder.cxx;egammaoqflagsbuilder.cxx;revision;egammaoqflagsbuilder.cxx;work;copy;celliter;celliterend;celliter;const;calocell;cell;celliter;cell;continue;float;cell;float;cell;float;energy;cell;energy;protection;pointer;return;statuscode;success;cluster;getnumberofcell;cluster;cluster;return;statuscode;success;cluster;getnumberofcell;cluster;float;clustereta;cluster;central;cell;middle;layer;bool;foundcentralcell;egammaoqflagsbuilder;findcentralcell;cluster;timing;double;absenergygev;cluster;absenergygev;cluster;gettime;m_tcut;m_tcutvse;absenergygev;celliter;celliterend;celliter;cell;parameter;property;const;calocell;cell;celliter;cell;continue;float;cell;float;cell;longer;float;qual;cell;quality;atlas16;egammatracktool;diff;index;emextrapolcaloconversion.cxx;emextrapolcaloconversion.cxx;revision;emextrapolcaloconversion.cxx;work;copy;eta_cluster_2nd_sampling;radius;calocell_id;emb1;eta_cluster_1st_sampling;phi_cluster_2nd_sampling;z_photon_front;r_photon_front;sinh;eta_cluster_1st_sampling;eta_cluster_1st_sampling;radius;calocell_id;eme1;eta_cluster_1st_sampling;phi_cluster_2nd_sampling;r_photon_front;z_photon_front;sinh;eta_cluster_1st_sampling;eta_cluster_1st_sampling;r_photon_front;z_photon_front;sinh;eta_cluster_1st_sampling;return;statuscode;success;return;statuscode;success;posterity;crash;related;http;savannah.cern.ch;mike;d2aod_dipho;crash;egammatool;egammatracktool;egamma'),(87667,'atlas','error in mc10_chain_withouttrigger_esdtoaod (cbnt_caloclustertopo crashed)','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) an error occurred in mc10_chain_withouttrigger_esdtoaod (cbnt_caloclustertopo crashed) see logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recexanatest/mc10_chain_withouttrigger_esdtoaod/mc10_chain_withouttrigger_esdtoaod_log\r\n\r\n\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  done processing event #32035, run #105200 135 events processed so far  &lt;&lt;&lt;===\r\nathenaeventloopmgr                                   info   ===&gt;&gt;&gt;  start processing event #32036, run #105200 135 events processed so far  &lt;&lt;&lt;===\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object eflowobjects_taumode  of type eflowobjectcontainer(clid 9804)\r\ncbntaa_eflow                                      warning no eflow container found!\r\nstoregatesvc                                      warning retrieve(default): no valid proxy for default object \r\n of type lvl1_roi(clid 6771448)\r\ncbnt_lvl1calo                                     warning could not find lvl1_roi\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object roibresult  of type roib::roibresult(clid 6000)\r\ncbnt_readroibresult                               warning could not find roib result\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object triggertowers  of type datavector&lt;lvl1::triggertower&gt;(clid 6207)\r\ncbnt_l1caloppm                                    warning no triggertowercollectiton found with key: triggertowers\r\ncbnt_athenaaware                                  warning  failure to  execute sub-algorithm:cbntaa_l1caloppm/cbnt_l1caloppm\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object trigindettracktruthmap  of type trigindettracktruthmap(clid 78130186)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_jet  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_tau  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_egamma  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_muon  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_muoniso  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_bphysics  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_fullscan  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_beamspot  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_cosmics  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_egamma_brem  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigidscan_tile  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_egamma  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_muon  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_muoniso  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_tau  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_jet  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_bphysics  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_fullscan  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_beamspot  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_tile  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_cosmics  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_egamma_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_muon_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_muoniso_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_tau_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_jet_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_bphysics_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_fullscan_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigsitrack_tile_robust  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trigl2sitrackfinder  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trtsegmentfinder  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object hlt_trtxk  of type trigindettrackcollection(clid 1299522495)\r\nstoregatesvc                                      warning retrieve(const): no valid proxy for object ctp_decision  of type ctp_decision(clid 123529088)\r\ncbntaa_aodtrigger                                 warning ctp_decision : ctp_decision could not be retrieved from storegate !\r\nr__unzip: error -5 in inflate (zlib)\r\nerror in &lt;tbasket::readbasketbuffers&gt;: fnbytes = 212897, fkeylen = 125, fobjlen = 377370, noutot = 0, nout=0, nin=212772, nbuf=377370\r\nerror in &lt;tbranchelement::getbasket&gt;: file: rfio:///afs/cern.ch/atlas/project/rtt/prod/results/rtt_j_r/chainstore/devval/x86_64-slc5-gcc43-opt/atlasproduction/reconstruction/recexample/recexanatest/recexanatest_rtt_notrig_steps_mc10/recexanatest_notrig_steps_mc10_esd.pool.root at byte:149938508, branch:calocelllinkcontainer_p2_calocaltopocluster_link, entry:135, badread=1, nerrors=1, basketnumber=31\r\ncollectiontree(calocelllinkcontainer_p2/calocaltopocluster_link) info cannot load entry no.135... branch has 300 entries in total.\r\nathenapoolconverter                                 error pooltoobject: could not get object for token = [db=f4f67bf3-15f3-e011-ab67-0030487c8c86][cnt=collectiontree(calocelllinkcontainer_p2/calocaltopocluster_link)][clid=826fc299-e988-4a07-926f-7d016f62fe5d][tech=00000202][oid=000000b0-00000087]\r\nathenapoolconverter                                 error failed to convert persistent object to transient: pool read failed. token = [db=f4f67bf3-15f3-e011-ab67-0030487c8c86][cnt=collectiontree(calocelllinkcontainer_p2/calocaltopocluster_link)][clid=826fc299-e988-4a07-926f-7d016f62fe5d][tech=00000202][oid=000000b0-00000087]\r\nathenapoolconverter                                 error createobj pooltodataobject() failed, token = [db=f4f67bf3-15f3-e011-ab67-0030487c8c86][cnt=collectiontree(calocelllinkcontainer_p2/calocaltopocluster_link)][clid=826fc299-e988-4a07-926f-7d016f62fe5d][tech=00000202][oid=000000b0-00000087]\r\ndataproxy                                         warning accessdata: conversion failed for data object 1327319901/calocaltopocluster_link\r\n returning null dataobject pointer  \r\nr__unzip: error -5 in inflate (zlib)\r\nerror in &lt;tbasket::readbasketbuffers&gt;: fnbytes = 212897, fkeylen = 125, fobjlen = 377370, noutot = 0, nout=0, nin=212772, nbuf=377370\r\nerror in &lt;tbranchelement::getbasket&gt;: file: rfio:///afs/cern.ch/atlas/project/rtt/prod/results/rtt_j_r/chainstore/devval/x86_64-slc5-gcc43-opt/atlasproduction/reconstruction/recexample/recexanatest/recexanatest_rtt_notrig_steps_mc10/recexanatest_notrig_steps_mc10_esd.pool.root at byte:149938508, branch:calocelllinkcontainer_p2_calocaltopocluster_link, entry:135, badread=1, nerrors=2, basketnumber=31\r\ncollectiontree(calocelllinkcontainer_p2/calocaltopocluster_link) info cannot load entry no.135... branch has 300 entries in total.\r\nathenapoolconverter                                 error pooltoobject: could not get object for token = [db=f4f67bf3-15f3-e011-ab67-0030487c8c86][cnt=collectiontree(calocelllinkcontainer_p2/calocaltopocluster_link)][clid=826fc299-e988-4a07-926f-7d016f62fe5d][tech=00000202][oid=000000b0-00000087]\r\nathenapoolconverter                                 error failed to convert persistent object to transient: pool read failed. token = [db=f4f67bf3-15f3-e011-ab67-0030487c8c86][cnt=collectiontree(calocelllinkcontainer_p2/calocaltopocluster_link)][clid=826fc299-e988-4a07-926f-7d016f62fe5d][tech=00000202][oid=000000b0-00000087]\r\nathenapoolconverter                                 error createobj pooltodataobject() failed, token = [db=f4f67bf3-15f3-e011-ab67-0030487c8c86][cnt=collectiontree(calocelllinkcontainer_p2/calocaltopocluster_link)][clid=826fc299-e988-4a07-926f-7d016f62fe5d][tech=00000202][oid=000000b0-00000087]\r\ndataproxy                                         warning accessdata: conversion failed for data object 1327319901/calocaltopocluster_link\r\n returning null dataobject pointer  \r\n-------------------------------------------------------------------------------------\r\ncore dump from coredumpsvc on lxb8761.cern.ch at mon oct 10 19:39:59 2011\r\n\r\ncaught signal 11(segmentation fault). details: \r\n  errno = 0, code = 1 (address not mapped to object)\r\n  pid   = 0, uid = 0\r\n  value = (103940856, 0x63202f8)\r\n  vmem = 1423.34 mb\r\n  rss  = 891.574 mb\r\n  total-ram = 15926.4 mb\r\n  free-ram  = 1359.89 mb\r\n  buffer-ram= 237.43 mb\r\n  total-swap= 4094.55 mb\r\n  free-swap = 4055.29 mb\r\n  addr  = 0\r\n\r\nevent counter: 136\r\neventid: [run,evt,lumi,time,bunchcross,detmask] = [105200,32036,3,0:0,0,0x0]\r\nlast incident: athenaeventloopmgr:beginevent\r\ncurrent algorithm: cbnt_caloclustertopo\r\nalgorithm stack: \r\n   athmasterseq\r\n   athalgseq\r\n   cbnt_athenaaware\r\n   cbnt_caloclustertopo\r\n-------------------------------------------------------------------------------------\r\n\r\n(pid=7909 ppid=1519) received fatal signal 11 (segmentation fault)\r\nsignal context:\r\n  signo  = 11, errno = 0, code = 1 (address not mapped to object)\r\n  pid    = 0, uid = 0\r\n  value  = (103940856, 0x63202f8)\r\n  addr   = (nil)\r\n  stack  = (2, 0, (nil))\r\n\r\n  9894fd2e 00000000 9896fd2e 00000000 10000000 00000000 00000000 00000000\r\n  18df53b5 aa2a0000 002a0218 00000000 20e253b5 aa2a0000 7d010000 00000000\r\n  00000000 00000000 70000000 00000000 38d27523 00000000 80d6a2f5 ff7f0000\r\n  9850c315 3d2b0000 00000000 00000000 c0f91000 00000000 a0d2a2f5 ff7f0000\r\n  9cb64fb5 aa2a0000 46020100 00000000 33000000 00000000 04000000 00000000\r\n  0e000000 00000000 00000000 00000000 00000000 00000000 20d0a2f5 ff7f0000\r\n  00000000 00000000 10b94fb5 aa2a0000 605ff113 00000000 a0cfa2f5 ff7f0000\r\n  20e253b5 aa2a0000 7d010000 00000000 0429d220 3d2b0000 352a5ae4 00000000\r\n\r\nstack trace:\r\n 0x2b3d20d4581c stacktrace ../src/sealdebug.cxx:738   + 0x4c [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0x2b3d20d465f2 fataldump ../src/sealsignal.cxx:1397   + 0x152 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0x2b3d20d46904 fatal ../src/sealsignal.cxx:898   + 0x194 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcxxutils.so]\r\n 0x2b3d22ac72c1 action ../src/coredumpsvc.cxx:90   + 0x101 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0x3df020eb70 ???  + 0x3df020eb70 [/lib64/libpthread.so.0]\r\n 0x2aaab54fb69c begin /build/atnight/localbuilds/nightlies/17.x.0-val/atlasevent/rel_0/installarea/include/caloevent/caloevent/caloclusternavigable.h:287   + 0xc [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcaloevent.so]\r\n 0x2aaab54fb10a cell_begin /build/atnight/localbuilds/nightlies/17.x.0-val/atlasevent/rel_0/installarea/include/caloevent/caloevent/calocompositecellbase.icc:176   + 0xa [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcaloevent.so]\r\n 0x2aaabf42f8d9 cbnt_execute ../src/cbntaa_calocluster.cxx:1014   + 0x1d79 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcaloreclib.so]\r\n 0x2aaabea8ce28 operator= inlined at ../src/cbnt_athenaawarebase.cxx:115 in cbnt_athenaawarebase::execute() /build/atnight/localbuilds/nightlies/17.x.0-val/gaudi/rel_0/installarea/include/gaudikernel/statuscode.h:105   + 0x3e8 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcbnt_utilslib.so]\r\n 0x2b3d1f11e4d4 algorithm::sysexecute() ??:0   + 0x124 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0x2aaabf2245fb operator= inlined at ../src/cbnt_athenaaware.cxx:106 in cbnt_athenaaware::execute() /build/atnight/localbuilds/nightlies/17.x.0-val/gaudi/rel_0/installarea/include/gaudikernel/statuscode.h:104   + 0x1ab [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libcbnt_athena.so]\r\n 0x2b3d1f11e4d4 algorithm::sysexecute() ??:0   + 0x124 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0x2aaabda91cd1 operator= inlined at ../src/athsequencer.cxx:157 in athsequencer::execute() /build/atnight/localbuilds/nightlies/17.x.0-val/gaudi/rel_0/installarea/include/gaudikernel/statuscode.h:105   + 0x2b1 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudisequencer.so]\r\n 0x2b3d1f11e4d4 algorithm::sysexecute() ??:0   + 0x124 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0x2aaabda91cd1 operator= inlined at ../src/athsequencer.cxx:157 in athsequencer::execute() /build/atnight/localbuilds/nightlies/17.x.0-val/gaudi/rel_0/installarea/include/gaudikernel/statuscode.h:105   + 0x2b1 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudisequencer.so]\r\n 0x2b3d1f11e4d4 algorithm::sysexecute() ??:0   + 0x124 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudikernel.so]\r\n 0x2b3d22adf693 executealgorithms ../src/athenaeventloopmgr.cxx:596   + 0x63 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0x2b3d22ae5afc executeevent ../src/athenaeventloopmgr.cxx:684   + 0x68c [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0x2b3d22ae39d4 operator= inlined at ../src/athenaeventloopmgr.cxx:907 in athenaeventloopmgr::nextevent(int) /build/atnight/localbuilds/nightlies/17.x.0-val/gaudi/rel_0/installarea/include/gaudikernel/statuscode.h:105   + 0xa74 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0x2b3d22ae5f7c issuccess inlined at ../src/athenaeventloopmgr.cxx:753 in athenaeventloopmgr::executerun(int) /build/atnight/localbuilds/nightlies/17.x.0-val/gaudi/rel_0/installarea/include/gaudikernel/statuscode.h:60   + 0x3c [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libathenaservices.so]\r\n 0x2b3d21f8ed59 applicationmgr::executerun(int) ??:0   + 0x69 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudisvc.so]\r\n 0x2b3d2154ad08 _zn12_global__n_1l11method_4301epvs0_rkst6vectoris0_sais0_ees0_ ??:0   + 0x2b3d2154ad08 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_1/installarea/x86_64-slc5-gcc43-opt/lib/libgaudikerneldict.so]\r\n 0x2b3d1c86657d root::cintex::method_stub_with_context(root::cintex::stubcontext_t*, g__value*, char const*, g__param*, int) ??:0   + 0x13d [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_61/installarea/x86_64-slc5-gcc43-opt/lib/libcintex.so]\r\n 0x2b3d18b91e7c cint::g__callfunc::execute(void*) ??:0   + 0xcc [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_61/installarea/x86_64-slc5-gcc43-opt/lib/libcint.so]\r\n 0x2b3d17ee1816 pyroot::trootobjectbyvalueexecutor::execute(cint::g__callfunc*, void*) ??:0   + 0x16 [/afs/cern.ch/sw/lcg/app/releases/root/5.30.00/x86_64-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0x2b3d17ee7d5e pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::callsafe(void*) ??:0   + 0x5e [/afs/cern.ch/sw/lcg/app/releases/root/5.30.00/x86_64-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0x2b3d17ee7f0c pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::execute(void*) ??:0   + 0x7c [/afs/cern.ch/sw/lcg/app/releases/root/5.30.00/x86_64-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0x2b3d17ee682e pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::operator()(pyroot::objectproxy*, _object*, _object*, long) ??:0   + 0x16e [/afs/cern.ch/sw/lcg/app/releases/root/5.30.00/x86_64-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0x2b3d17eed77e pyroot::(anonymous namespace)::mp_call(pyroot::methodproxy*, _object*, _object*) ??:0   + 0x2b3d17eed77e [/afs/cern.ch/sw/lcg/app/releases/root/5.30.00/x86_64-slc5-gcc43-opt/root/lib/libpyroot.so]\r\n 0x004197b8 pyobject_call objects/abstract.c:2492   + 0x68 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x00496742 do_call inlined at python/ceval.c:3773 in pyeval_evalframeex python/ceval.c:3968   + 0x2c32 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a2a7 pyeval_evalcodeex python/ceval.c:3000   + 0x927 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004987e8 fast_function inlined at python/ceval.c:3771 in pyeval_evalframeex python/ceval.c:3846   + 0x4cd8 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a2a7 pyeval_evalcodeex python/ceval.c:3000   + 0x927 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a3a2 pyeval_evalcode python/ceval.c:547   + 0x32 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004ba9aa run_mod inlined at python/pythonrun.c:1325 in pyrun_fileexflags python/pythonrun.c:1339   + 0xaa [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x00491d42 builtin_execfile python/bltinmodule.c:797   + 0x491d42 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049888c call_function inlined at python/ceval.c:2412 in pyeval_evalframeex python/ceval.c:3750   + 0x4d7c [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a2a7 pyeval_evalcodeex python/ceval.c:3000   + 0x927 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004f213d function_call objects/funcobject.c:524   + 0x4f213d [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004197b8 pyobject_call objects/abstract.c:2492   + 0x68 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x00420b00 instancemethod_call objects/classobject.c:2579   + 0x420b00 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004197b8 pyobject_call objects/abstract.c:2492   + 0x68 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0046e591 slot_tp_call objects/typeobject.c:5373   + 0x46e591 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004197b8 pyobject_call objects/abstract.c:2492   + 0x68 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x00496742 do_call inlined at python/ceval.c:3773 in pyeval_evalframeex python/ceval.c:3968   + 0x2c32 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a2a7 pyeval_evalcodeex python/ceval.c:3000   + 0x927 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a3a2 pyeval_evalcode python/ceval.c:547   + 0x32 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004ba9aa run_mod inlined at python/pythonrun.c:1325 in pyrun_fileexflags python/pythonrun.c:1339   + 0xaa [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x00491d42 builtin_execfile python/bltinmodule.c:797   + 0x491d42 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049888c call_function inlined at python/ceval.c:2412 in pyeval_evalframeex python/ceval.c:3750   + 0x4d7c [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a2a7 pyeval_evalcodeex python/ceval.c:3000   + 0x927 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004f213d function_call objects/funcobject.c:524   + 0x4f213d [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004197b8 pyobject_call objects/abstract.c:2492   + 0x68 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x00420b00 instancemethod_call objects/classobject.c:2579   + 0x420b00 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004197b8 pyobject_call objects/abstract.c:2492   + 0x68 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0046e591 slot_tp_call objects/typeobject.c:5373   + 0x46e591 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004197b8 pyobject_call objects/abstract.c:2492   + 0x68 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x00496742 do_call inlined at python/ceval.c:3773 in pyeval_evalframeex python/ceval.c:3968   + 0x2c32 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a2a7 pyeval_evalcodeex python/ceval.c:3000   + 0x927 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x0049a3a2 pyeval_evalcode python/ceval.c:547   + 0x32 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004ba9aa run_mod inlined at python/pythonrun.c:1325 in pyrun_fileexflags python/pythonrun.c:1339   + 0xaa [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004bac9d pyrun_simplefileexflags python/pythonrun.c:935   + 0x20d [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x004150a3 py_main modules/main.c:572   + 0xb73 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n 0x3def61d994 __libc_start_main  + 0xf4 [/lib64/libc.so.6]\r\n 0x004141d9 _start ??:0   + 0x4141d9 [/afs/cern.ch/sw/lcg/external/python/2.6.5/x86_64-slc5-gcc43-opt/bin/python]\r\n\r\n\r\ncheers michael','100','5211','2011-10-11 15:40:28','5','unreproducible','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;error;occur;mc10_chain_withouttrigger_esdtoaod;cbnt_caloclustertopo;crash;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recexanatest;mc10_chain_withouttrigger_esdtoaod;mc10_chain_withouttrigger_esdtoaod_log;athenaeventloopmgr;info;processing;event;event;process;start;processing;event;event;process;clid;cbnt_l1caloppm;warn;triggertowercollectiton;triggertower;cbnt_athenaaware;warn;failure;execute;algorithm:cbntaa_l1caloppm;cbnt_l1caloppm;storegatesvc;warn;retrieve;const;valid;proxy;object;trigindettracktruthmap;type;trigindettracktruthmap;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_jet;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_tau;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_egamma;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_muon;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_muoniso;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_bphysics;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_fullscan;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_beamspot;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_cosmics;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_egamma_brem;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigidscan_tile;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_egamma;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_muon;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_muoniso;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_tau;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_jet;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_bphysics;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_fullscan;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_beamspot;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_tile;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_cosmics;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_egamma_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_muon_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_muoniso_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_tau_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_jet_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_bphysics_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_fullscan_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigsitrack_tile_robust;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trigl2sitrackfinder;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trtsegmentfinder;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;hlt_trtxk;type;trigindettrackcollection;clid;storegatesvc;warn;retrieve;const;valid;proxy;object;ctp_decision;type;ctp_decision;clid;cbntaa_aodtrigger;warn;ctp_decision;ctp_decision;retrieve;storegate;r__unzip;error;inflate;zlib;error;fnbyte;fkeylen;fobjlen;noutot;nout;nbuf;error;file;rfio;atla;project;prod;result;rtt_j_r;chainstore;devval;x86_64;slc5;gcc43;atlasproduction;reconstruction;recexample;recexanatest;recexanatest_rtt_notrig_steps_mc10;recexanatest_notrig_steps_mc10_esd.pool.root;byte;branch:calocelllinkcontainer_p2_calocaltopocluster_link;entry;badread;nerror;basketnumber;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;info;load;entry;branch;entry;total;athenapoolconverter;error;pooltoobject;object;token;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;clid;tech;athenapoolconverter;error;fail;convert;persistent;object;transient;pool;read;fail;token;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;clid;tech;athenapoolconverter;error;createobj;pooltodataobject;fail;token;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;clid;tech;dataproxy;warn;accessda;conversion;fail;datum;object;calocaltopocluster_link;return;null;dataobject;pointer;r__unzip;error;inflate;zlib;error;fnbyte;fkeylen;fobjlen;noutot;nout;nbuf;error;file;rfio;atla;project;prod;result;rtt_j_r;chainstore;devval;x86_64;slc5;gcc43;atlasproduction;reconstruction;recexample;recexanatest;recexanatest_rtt_notrig_steps_mc10;recexanatest_notrig_steps_mc10_esd.pool.root;byte;branch:calocelllinkcontainer_p2_calocaltopocluster_link;entry;badread;nerror;basketnumber;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;info;load;entry;branch;entry;total;athenapoolconverter;error;pooltoobject;object;token;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;clid;tech;athenapoolconverter;error;fail;convert;persistent;object;transient;pool;read;fail;token;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;clid;tech;athenapoolconverter;error;createobj;pooltodataobject;fail;token;collectiontree;calocelllinkcontainer_p2;calocaltopocluster_link;clid;tech;dataproxy;warn;accessda;conversion;fail;datum;object;calocaltopocluster_link;return;null;dataobject;pointer;core;dump;coredumpsvc;lxb8761;catch;signal;segmentation;fault;detail;errno;code;address;object;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;eventid;lumus;time;bunchcross;detmask;incident;athenaeventloopmgr:beginevent;current;algorithm;cbnt_caloclustertopo;algorithm;stack;athmasterseq;athalgseq;cbnt_athenaaware;cbnt_caloclustertopo;ppid;receive;fatal;signal;segmentation;fault;signal;context;signo;errno;code;address;object;value;addr;stack;stack;trace;stacktrace;sealdebug.cxx;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libcxxutils.so;fataldump;sealsignal.cxx;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libcxxutils.so;fatal;sealsignal.cxx;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libcxxutils.so;action;coredumpsvc.cxx;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libathenaservices.so;lib64;libpthread.so;begin;build;atnight;localbuild;nightly;atlasevent;rel_0;installarea;include;caloevent;caloevent;caloclusternavigable.h;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;x86_64;slc5;gcc43;libcaloevent.so;cell_begin;build;atnight;localbuild;nightly;atlasevent;rel_0;installarea;include;caloevent;caloevent;calocompositecellbase.icc;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;x86_64;slc5;gcc43;libcaloevent.so;cbnt_execute;cbntaa_calocluster.cxx;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;x86_64;slc5;gcc43;libcaloreclib.so;operator;inline;cbnt_athenaawarebase.cxx;cbnt_athenaawarebase;execute;build;atnight;localbuild;nightly;gaudus;rel_0;installarea;include;gaudikernel;statuscode.h;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;x86_64;slc5;gcc43;libcbnt_utilslib.so;algorithm;sysexecute;atla;software;build;nightly;devval;gaudus;rel_1;installarea;x86_64;slc5;gcc43;libgaudikernel.so;operator;inline;cbnt_athenaaware.cxx;cbnt_athenaaware;execute;build;atnight;localbuild;nightly;gaudus;rel_0;installarea;include;gaudikernel;statuscode.h;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;x86_64;slc5;gcc43;libcbnt_athena.so;algorithm;sysexecute;atla;software;build;nightly;devval;gaudus;rel_1;installarea;x86_64;slc5;gcc43;libgaudikernel.so;operator;inline;athsequencer.cxx;athsequencer;execute;build;atnight;localbuild;nightly;gaudus;rel_0;installarea;include;gaudikernel;statuscode.h;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libgaudisequencer.so;algorithm;sysexecute;atla;software;build;nightly;devval;gaudus;rel_1;installarea;x86_64;slc5;gcc43;libgaudikernel.so;operator;inline;athsequencer.cxx;athsequencer;execute;build;atnight;localbuild;nightly;gaudus;rel_0;installarea;include;gaudikernel;statuscode.h;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libgaudisequencer.so;algorithm;sysexecute;atla;software;build;nightly;devval;gaudus;rel_1;installarea;x86_64;slc5;gcc43;libgaudikernel.so;executealgorithm;athenaeventloopmgr.cxx;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libathenaservices.so;executeevent;athenaeventloopmgr.cxx;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libathenaservices.so;operator;inline;athenaeventloopmgr.cxx;athenaeventloopmgr;nextevent;build;atnight;localbuild;nightly;gaudus;rel_0;installarea;include;gaudikernel;statuscode.h;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libathenaservices.so;issuccess;inline;athenaeventloopmgr.cxx;athenaeventloopmgr;executerun;build;atnight;localbuild;nightly;gaudus;rel_0;installarea;include;gaudikernel;statuscode.h;atla;software;build;nightly;devval;atlascore;rel_1;installarea;x86_64;slc5;gcc43;libathenaservices.so;applicationmgr;executerun;atla;software;build;nightly;devval;gaudus;rel_1;installarea;x86_64;slc5;gcc43;libgaudisvc.so;atla;software;build;nightly;devval;gaudus;rel_1;installarea;x86_64;slc5;gcc43;libgaudikerneldict.so;root;cintex;method_stub_with_context;root;cintex;stubcontext_t;g__value;char;const;g__param;atla;offline;external;lcgcmt;lcgcmt_61;installarea;x86_64;slc5;gcc43;libcintex.so;cint;g__callfunc;execute;void;atla;offline;external;lcgcmt;lcgcmt_61;installarea;x86_64;slc5;gcc43;libcint.so;pyroot;trootobjectbyvalueexecutor;execute;cint;g__callfunc;void;release;root;5.30.00;x86_64;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;callsafe;void;release;root;5.30.00;x86_64;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;execute;void;release;root;5.30.00;x86_64;slc5;gcc43;root;libpyroot.so;pyroot;tmethodholder;operator;pyroot;objectproxy;long;release;root;5.30.00;x86_64;slc5;gcc43;root;libpyroot.so;pyroot;anonymous;namespace;mp_call;pyroot;methodproxy;release;root;5.30.00;x86_64;slc5;gcc43;root;libpyroot.so;pyobject_call;object;abstract.c;external;python;2.6.5;x86_64;slc5;gcc43;python;do_call;inline;python;ceval.c;pyeval_evalframeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcodeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;fast_function;inline;python;ceval.c;pyeval_evalframeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcodeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcode;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;run_mod;inline;python;pythonrun.c;pyrun_fileexflags;python;pythonrun.c;external;python;2.6.5;x86_64;slc5;gcc43;python;builtin_execfile;python;bltinmodule.c;external;python;2.6.5;x86_64;slc5;gcc43;python;call_function;inline;python;ceval.c;pyeval_evalframeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcodeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;function_call;object;funcobject.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyobject_call;object;abstract.c;external;python;2.6.5;x86_64;slc5;gcc43;python;instancemethod_call;object;classobject.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyobject_call;object;abstract.c;external;python;2.6.5;x86_64;slc5;gcc43;python;slot_tp_call;object;typeobject.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyobject_call;object;abstract.c;external;python;2.6.5;x86_64;slc5;gcc43;python;do_call;inline;python;ceval.c;pyeval_evalframeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcodeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcode;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;run_mod;inline;python;pythonrun.c;pyrun_fileexflags;python;pythonrun.c;external;python;2.6.5;x86_64;slc5;gcc43;python;builtin_execfile;python;bltinmodule.c;external;python;2.6.5;x86_64;slc5;gcc43;python;call_function;inline;python;ceval.c;pyeval_evalframeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcodeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;function_call;object;funcobject.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyobject_call;object;abstract.c;external;python;2.6.5;x86_64;slc5;gcc43;python;instancemethod_call;object;classobject.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyobject_call;object;abstract.c;external;python;2.6.5;x86_64;slc5;gcc43;python;slot_tp_call;object;typeobject.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyobject_call;object;abstract.c;external;python;2.6.5;x86_64;slc5;gcc43;python;do_call;inline;python;ceval.c;pyeval_evalframeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcodeex;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyeval_evalcode;python;ceval.c;external;python;2.6.5;x86_64;slc5;gcc43;python;run_mod;inline;python;pythonrun.c;pyrun_fileexflags;python;pythonrun.c;external;python;2.6.5;x86_64;slc5;gcc43;python;pyrun_simplefileexflags;python;pythonrun.c;external;python;2.6.5;x86_64;slc5;gcc43;python;py_main;module;main.c;external;python;2.6.5;x86_64;slc5;gcc43;python;lib64;libc.so;external;python;2.6.5;x86_64;slc5;gcc43;python;michael;error;mc10_chain_withouttrigger_esdtoaod;cbnt_caloclustertopo;crash'),(87668,'atlas','early crash in both mc10_chain_withouttrigger_aodtocbnt and mc10_chain_withouttrigger_aodtotag','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) both mc10_chain_withouttrigger_aodtocbnt and mc10_chain_withouttrigger_aodtotag crashed.\r\n\r\nsee logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recexanatest/mc10_chain_withouttrigger_aodtocbnt/mc10_chain_withouttrigger_aodtocbnt_log\r\n\r\n*****************************************************\r\nthe rtt will now execute the command:\r\n athena.py  -c \'jp.athenacommonflags.evtmax=300\' -b recexanatest/aodtocbnt_rtt.py \r\n*****************************************************\r\n\r\nmon oct 10 22:19:22 cest 2011\r\npreloading tcmalloc_minimal.so\r\npy:athena            info including file &quot;athenacommon/preparation.py&quot;\r\npy:athena            info using release [atlasproduction-17.4.0] [17.x.0-val/rel_1] -- built on [2011 10/09 22:09]\r\npy:athena            info including file &quot;athenacommon/bootstrap.py&quot;\r\npy:athena            info including file &quot;athenacommon/atlas.unixstandardjob.py&quot;\r\npy:athena            info executing cli (-c) command: &quot;jp.athenacommonflags.evtmax=300&quot;\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;bool&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;char&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;short&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;long&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;unsigned-char&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;unsigned-short&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;unsigned-int&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;unsigned-long&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;float&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\nwarning in &lt;tenvrec::changevalue&gt;: duplicate entry &lt;library.vector&lt;double&gt;=vector.dll vectorbool.dll&gt; for level 0; ignored\r\npy:athena            info including file &quot;athenacommon/execution.py&quot;\r\npy:athena            info including file &quot;recexanatest/aodtocbnt_rtt.py&quot;\r\npy:athena            info including file &quot;recexcommon/aodtocbnt.py&quot;\r\npy:jobpropertycontainer::    info recflags::readaod : all algorithms are off\r\npy:athena            info including file &quot;recexcommon/recexcommon_topoptions.py&quot;\r\npy:athena            info \'gbl.athenaservices.setfatalhandler(flag)\' is deprecated\r\npy:athena            info instead, please consider using:\r\npy:athena            info  svcmgr.coredumpsvc.fatalhandler = flag\r\npy:athena            info including file &quot;recexcond/recexcommon_flags.py&quot;\r\npy:recexcommon_flags    info full auto-configuration has been triggered by filesinput()\r\npy:autoconfiguration    info auto-configuration key \'everything\' requested. all default keys will be used.\r\npy:autoconfiguration    info auto-configuration will procede according to the following keys:\r\npy:autoconfiguration    info [\'projectname\', \'realorsim\', \'fieldandgeo\', \'beamtype\', \'conditionstag\', \'dotruth\', \'inputtype\', \'beamenergy\', \'lumiflags\', \'triggerstream\']\r\npy:inputfilepeeker    info executing   inputfilepeeker.py\r\npy:athfile           info opening [rfio:/afs/cern.ch/atlas/project/rtt/prod/results/rtt_j_r/chainstore/devval/x86_64-slc5-gcc43-opt/atlasproduction/reconstruction/recexample/recexanatest/recexanatest_rtt_notrig_steps_mc10/recexanatest_notrig_steps_mc10_aod.pool.root]...\r\npy:inputfilepeeker warning unable to open file [rfio:/afs/cern.ch/atlas/project/rtt/prod/results/rtt_j_r/chainstore/devval/x86_64-slc5-gcc43-opt/atlasproduction/reconstruction/recexample/recexanatest/recexanatest_rtt_notrig_steps_mc10/recexanatest_notrig_steps_mc10_aod.pool.root]\r\npy:inputfilepeeker warning caught:\r\n[errno 2] no such file or directory: \'rfio:/afs/cern.ch/atlas/project/rtt/prod/results/rtt_j_r/chainstore/devval/x86_64-slc5-gcc43-opt/atlasproduction/reconstruction/recexample/recexanatest/recexanatest_rtt_notrig_steps_mc10/recexanatest_notrig_steps_mc10_aod.pool.root\'\r\ntraceback (most recent call last):\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/python/recexconfig/inputfilepeeker.py&quot;, line 48, in _setup\r\n    fi = athfile.fopen(infile)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/python/pyutils/athfile/__init__.py&quot;, line 160, in fopen\r\n    return self.server.fopen(fnames, evtmax)\r\n  file &quot;&lt;string&gt;&quot;, line 2, in fopen\r\n  file &quot;/afs/cern.ch/sw/lcg/external/pytools/1.5_python2.6/x86_64-slc5-gcc43-opt/lib/python2.6/site-packages/multiprocessing-2.6.2.1-py2.6-linux-x86_64.egg/multiprocessing/managers.py&quot;, line 743, in _callmethod\r\n    raise convert_to_error(kind, result)\r\nioerror: [errno 2] no such file or directory: \'rfio:/afs/cern.ch/atlas/project/rtt/prod/results/rtt_j_r/chainstore/devval/x86_64-slc5-gcc43-opt/atlasproduction/reconstruction/recexample/recexanatest/recexanatest_rtt_notrig_steps_mc10/recexanatest_notrig_steps_mc10_aod.pool.root\'\r\npy:inputfilepeeker   error unable to build inputfilesummary from any of the specified input files. there is probably a problem.\r\npy:autoconfiguration   error no runnumber stored in inputfile!\r\nshortened traceback (most recent user call last):\r\n  file &quot;recexanatest/aodtocbnt_rtt.py&quot;, line 6, in &lt;module&gt;\r\n    include(&quot;recexcommon/aodtocbnt.py&quot;)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/joboptions/recexcommon/aodtocbnt.py&quot;, line 19, in &lt;module&gt;\r\n    include (&quot;recexcommon/recexcommon_topoptions.py&quot;)\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/joboptions/recexcommon/recexcommon_topoptions.py&quot;, line 87, in &lt;module&gt;\r\n    include ( &quot;recexcond/recexcommon_flags.py&quot; )\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/joboptions/recexcond/recexcommon_flags.py&quot;, line 235, in &lt;module&gt;\r\n    configurefromlistofkeys(rec.autoconfiguration())\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/python/recexconfig/autoconfiguration.py&quot;, line 891, in configurefromlistofkeys\r\n    rec.projectname=getprojectname()\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/python/recexconfig/autoconfiguration.py&quot;, line 286, in getprojectname\r\n    from recexconfig.getcool import cool\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/python/recexconfig/getcool.py&quot;, line 32, in &lt;module&gt;\r\n    cool=_setup()\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/python/recexconfig/getcool.py&quot;, line 19, in _setup\r\n    lb=getlbnumber()\r\n  file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasevent/rel_1/installarea/python/recexconfig/autoconfiguration.py&quot;, line 63, in getlbnumber\r\n    if inputfilesummary[\'nentries\']==0:\r\nkeyerror: \'nentries\'\r\npy:athena            info leaving with code 8: &quot;an unknown exception occurred&quot;\r\n\r\ncheers michael','100','5211','2011-10-11 15:44:03','5','none','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;mc10_chain_withouttrigger_aodtocbnt;mc10_chain_withouttrigger_aodtotag;crash;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recexanatest;mc10_chain_withouttrigger_aodtocbnt;mc10_chain_withouttrigger_aodtocbnt_log;execute;command;athena.py;jp.athenacommonflags.evtmax;recexanatest;aodtocbnt_rtt.py;cest;preload;tcmalloc_minimal.so;py:athena;info;include;file;athenacommon;preparation.py;py:athena;info;release;atlasproduction;17.4.0;rel_1;build;py:athena;info;include;file;athenacommon;bootstrap.py;py:athena;info;include;file;athenacommon;atlas.unixstandardjob.py;py:athena;info;execute;command;jp.athenacommonflags.evtmax;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;warn;duplicate;entry;vector.dll;vectorbool.dll;level;ignore;py:athena;info;include;file;athenacommon;execution.py;py:athena;info;include;file;recexanatest;aodtocbnt_rtt.py;py:athena;info;include;file;recexcommon;aodtocbnt.py;py:jobpropertycontainer;info;recflag;readaod;algorithm;py:athena;info;include;file;recexcommon;recexcommon_topoptions.py;py:athena;info;gbl.athenaservices.setfatalhandler;flag;deprecate;py:athena;info;instead;consider;py:athena;info;svcmgr.coredumpsvc.fatalhandler;flag;py:athena;info;include;file;recexcond;recexcommon_flags.py;py:recexcommon_flags;info;auto;configuration;trigger;filesinput;py:autoconfiguration;info;auto;configuration;request;default;py:autoconfiguration;info;auto;configuration;procede;accord;follow;py:autoconfiguration;info;projectname;realorsim;fieldandgeo;beamtype;conditionstag;dotruth;inputtype;beamenergy;lumiflag;triggerstream;py:inputfilepeeker;info;execute;inputfilepeeker.py;py:athfile;info;open;rfio;atla;project;prod;result;rtt_j_r;chainstore;devval;x86_64;slc5;gcc43;atlasproduction;reconstruction;recexample;recexanatest;recexanatest_rtt_notrig_steps_mc10;recexanatest_notrig_steps_mc10_aod.pool.root;py:inputfilepeeker;warn;unable;open;file;rfio;atla;project;prod;result;rtt_j_r;chainstore;devval;x86_64;slc5;gcc43;atlasproduction;reconstruction;recexample;recexanatest;recexanatest_rtt_notrig_steps_mc10;recexanatest_notrig_steps_mc10_aod.pool.root;py:inputfilepeeker;warn;catch;errno;file;directory;rfio;atla;project;prod;result;rtt_j_r;chainstore;devval;x86_64;slc5;gcc43;atlasproduction;reconstruction;recexample;recexanatest;recexanatest_rtt_notrig_steps_mc10;recexanatest_notrig_steps_mc10_aod.pool.root;traceback;recent;file;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;python;recexconfig;inputfilepeeker.py;line;athfile.fopen;infile;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;python;pyutil;athfile;line;fopen;return;self.server.fopen;fname;evtmax;file;line;fopen;file;external;pytool;1.5_python2.6;x86_64;slc5;gcc43;python2.6;site;package;multiprocess;2.6.2.1;py2.6;linux;x86_64;multiprocess;managers.py;line;raise;convert_to_error;kind;result;ioerror;errno;file;directory;rfio;atla;project;prod;result;rtt_j_r;chainstore;devval;x86_64;slc5;gcc43;atlasproduction;reconstruction;recexample;recexanatest;recexanatest_rtt_notrig_steps_mc10;recexanatest_notrig_steps_mc10_aod.pool.root;py:inputfilepeeker;error;unable;build;inputfilesummary;specify;input;file;probably;problem;py:autoconfiguration;error;runnumber;store;inputfile;shorten;traceback;recent;user;file;recexanatest;aodtocbnt_rtt.py;line;include;recexcommon;aodtocbnt.py;file;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;joboption;recexcommon;aodtocbnt.py;line;include;recexcommon;recexcommon_topoptions.py;file;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;joboption;recexcommon;recexcommon_topoptions.py;line;include;recexcond;recexcommon_flags.py;file;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;joboption;recexcond;recexcommon_flags.py;line;configurefromlistofkey;rec.autoconfiguration;file;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;python;recexconfig;autoconfiguration.py;line;configurefromlistofkey;rec.projectname;getprojectname;file;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;python;recexconfig;autoconfiguration.py;line;getprojectname;recexconfig.getcool;import;cool;file;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;python;recexconfig;getcool.py;line;cool;file;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;python;recexconfig;getcool.py;line;getlbnumber;file;atla;software;build;nightly;devval;atlasevent;rel_1;installarea;python;recexconfig;autoconfiguration.py;line;getlbnumber;inputfilesummary;nentry;keyerror;nentry;py:athena;info;leave;code;unknown;exception;occur;michael;early;crash;mc10_chain_withouttrigger_aodtocbnt;mc10_chain_withouttrigger_aodtotag'),(87669,'atlas','lots of errors and fatals in esd_memleakana','dear experts,\r\n\r\nin todays in todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) a lot of errors and fatals in esd_memleakana.\r\n\r\n\r\nsee logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recexanatest/esd_memleakana/esd_memleakana_log\r\n\r\n calling endrunmuonboy\r\napplicationmgr                                       info application manager finalized successfully\r\napplicationmgr                                       info application manager terminated successfully\r\nsummarizing all message counts\r\n=====================================================\r\n message source              |   level |    count\r\n-----------------------------+---------+-------------\r\n algerrorauditor             |   error |        4\r\n athenaeventloopmgr          |   error |        1\r\n athenapoolcnvsvc            |   error |        3\r\n athenapoolconverter         |   error |       23\r\n backgroundwordfiller        | warning |      284\r\n dataproxy                   | warning |        1\r\n iovdbsvc                    | warning |       14\r\n lvl1::trigt1trt             | warning |        1\r\n largemetstream_wmetprescaler| warning |        1\r\n mdtcablingsvc               | warning |        1\r\n mugm:rpcreadoutelement      | warning |        4\r\n muondetailedtracktruthmaker | warning |        1\r\n poolsvc                     | warning |       21\r\n poolsvc                     |   error |        5\r\n poolsvc                     |   fatal |        1\r\n servicemanager              |   error |        1\r\n storegatesvc                | warning |      302\r\n streamaod_fh                |   error |        1\r\n streamaod_fh                |   fatal |        1\r\n streamaod_fh.streamaod_fhtool|   error |        1\r\n streamesd                   |   error |        2\r\n streamesd                   |   fatal |        1\r\n streamesd.streamesdtool     |   error |        1\r\n streamesd_fh                |   error |        1\r\n streamesd_fh                |   fatal |        2\r\n streamesd_fh.streamesd_fhtool|   error |       21\r\n toolsvc.combinedmuonidholesearch| warning |       21\r\n toolsvc.combinedmuontrackbuilder| warning |       27\r\n toolsvc.dcmathsegmentmaker  | warning |        4\r\n toolsvc.improvedjetfitterroutines| warning |       47\r\n toolsvc.indettrigholesearchtool| warning |       21\r\n toolsvc.mctbfitter          | warning |        1\r\n toolsvc.mdtdriftcircleontrackcreator| warning |        2\r\n toolsvc.muidmatchmaker      | warning |        2\r\n toolsvc.muoncombinedtrackfitter| warning |        3\r\n toolsvc.muontracksummaryhelper| warning |        2\r\n toolsvc.rpcdetectorstatusdbtool| warning |        1\r\n toolsvc.sct_layerbuilder    | warning |        1\r\n toolsvc.trigmugirltrackbuilder| warning |       12\r\n toolsvc.trigmugirl_muidmaterialallocator| warning |        4\r\n toolsvc.extrapolmuonincalotool| warning |       17\r\n toolsvc.ipatfitter          | warning |      410\r\n trackrecordfilter           |   error |        1\r\n trigmuonefcombiner.tmefmatchmaker| warning |        1\r\n=====================================================\r\npy:athena            info leaving with code 65: &quot;failure in an algorithm execute&quot;\r\ncoralapplication info delete the cool coralapplication...\r\ncoralapplication info delete the cool database service\r\nraldatabasesvc info delete the raldatabasesvc...\r\nraldatabasesvc info purge the connection pool\r\nraldatabasesvc info reset the ics pointer\r\nraldatabasesvc info delete the raldatabasesvc... done\r\ncoralapplication info delete the coral connection service\r\ncoralapplication info delete the cool coralapplication... done\r\ncoral/services/connectionservice info deleting the connectionpool','100','5211','2011-10-11 15:47:43','5','unreproducible','closed','none',';dear;expert;today;today;devval;rel_1;x86_64;slc5;gcc43;error;fatal;esd_memleakana;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recexanatest;esd_memleakana;esd_memleakana_log;call;endrunmuonboy;applicationmgr;info;application;manager;finalize;successfully;applicationmgr;info;application;manager;terminate;successfully;summarize;message;count;message;source;level;count;algerrorauditor;error;athenaeventloopmgr;error;athenapoolcnvsvc;error;athenapoolconverter;error;backgroundwordfiller;warn;dataproxy;warn;iovdbsvc;warn;lvl1;trigt1trt;warn;largemetstream_wmetprescaler;warn;mdtcablingsvc;warn;mugm:rpcreadoutelement;warn;muondetailedtracktruthmaker;warn;poolsvc;warn;poolsvc;error;poolsvc;fatal;servicemanager;error;storegatesvc;warn;streamaod_fh;error;streamaod_fh;fatal;streamaod_fh.streamaod_fhtool;error;streamesd;error;streamesd;fatal;streamesd.streamesdtool;error;streamesd_fh;error;streamesd_fh;fatal;streamesd_fh.streamesd_fhtool;error;toolsvc.combinedmuonidholesearch;warn;toolsvc.combinedmuontrackbuilder;warn;toolsvc.dcmathsegmentmaker;warn;toolsvc.improvedjetfitterroutines;warn;toolsvc.indettrigholesearchtool;warn;toolsvc.mctbfitter;warn;toolsvc.mdtdriftcircleontrackcreator;warn;toolsvc.muidmatchmaker;warn;toolsvc.muoncombinedtrackfitter;warn;toolsvc.muontracksummaryhelper;warn;toolsvc.rpcdetectorstatusdbtool;warn;toolsvc.sct_layerbuilder;warn;toolsvc.trigmugirltrackbuilder;warn;toolsvc.trigmugirl_muidmaterialallocator;warn;toolsvc.extrapolmuonincalotool;warn;toolsvc.ipatfitter;warn;trackrecordfilter;error;trigmuonefcombiner.tmefmatchmaker;warn;py:athena;info;leave;code;failure;algorithm;execute;coralapplication;info;delete;cool;coralapplication;coralapplication;info;delete;cool;database;service;raldatabasesvc;info;delete;raldatabasesvc;raldatabasesvc;info;purge;connection;pool;raldatabasesvc;info;reset;pointer;raldatabasesvc;info;delete;raldatabasesvc;coralapplication;info;delete;coral;connection;service;coralapplication;info;delete;cool;coralapplication;coral;service;connectionservice;info;delete;connectionpool;error;fatal;esd_memleakana'),(87674,'atlas','rtt error crashes multiple jobs (/trigger/ofllumi/lblestofl): amitag_q150, data_7tev_allstreams_a, data_7tev ...','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) an error occurred for amitag_q150, data_7tev_allstreams_a, data_7tev_allstreams_b, data_7tev_diffinput, data_7tev_fpe, data_7tev_memleak, data_7tev_nomuon, splash_events.\r\n\r\nsee logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/amitag_q150/amitag_q150_log\r\n\r\n\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 ami=q150 inputbsfile=/afs/cern.ch/user/s/seuster/rig/referencefiles/datastreams/data11_7tev.00178044.physics_jettauetmiss.merge.raw._lb0092._sfo-4._0001.1\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error\r\nfatals=0\r\nerrors=1\r\nwarnings=10\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: mon oct 10 16:04:38 2011 cest, vmem: 324.60546875 mb, rss: 104.0234375 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 04 hrs 09 mins 18 secs, cpu-time: 03 hrs 52 mins 51 secs, vmem: 1.2265625 mb, rss: 1.14453125 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 ami=q150 inputbsfile=/afs/cern.ch/user/s/seuster/rig/referencefiles/datastreams/data11_7tev.00178044.physics_jettauetmiss.merge.raw._lb0092._sfo-4._0001.1\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error\r\nfatals=0\r\nerrors=1\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=duplicate request for folder /trigger/ofllumi/lblestofl associated to already requested storegate key /trigger/ofllumi/lblestofl\r\n  severity=error\r\n  who=iovdbsvc\r\n  producer=rawtoesd\r\n  stage=initialise\r\nathacronym=successful run\r\nathcode=0\r\nrundiruseddisk=2393329889\r\nrundiravailabledisk=189043408896\r\nworkdir=/pool/lsf/rtt/176801736\r\nathcode=0\r\nrundiravailabledisk=189043924992\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=2393114737\r\natlasrelease=rel_1\r\nwriting jobinfo_recotrf.xml file...\r\n','100','5211','2011-10-11 16:33:18','5','duplicate','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;error;occur;amitag_q150;data_7tev_allstreams_a;data_7tev_allstreams_b;data_7tev_diffinput;data_7tev_fpe;data_7tev_memleak;data_7tev_nomuon;splash_events;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;amitag_q150;amitag_q150_log;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;user;seuster;referencefile;datastream;data11_7tev;physics_jettauetmiss.merge.raw;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;user;seuster;referencefile;datastream;data11_7tev;physics_jettauetmiss.merge.raw;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;fatal;error;warning;errorinfo;acronym;trf_unknown;message;duplicate;request;folder;trigger;ofllumus;lblestofl;associate;request;storegate;trigger;ofllumus;lblestofl;severity;error;iovdbsvc;producer;rawtoesd;stage;initialise;athacronym;successful;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_1;write;jobinfo_recotrf.xml;file;error;crash;multiple;trigger;ofllumus;lblestofl;amitag_q150;data_7tev_allstreams_a;data_7tev'),(87675,'atlas','rtt tests: cannot find /tdaq/olc/calibrations folder','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) an error occurred for cosmis_allstream.\r\n\r\nsee logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/cosmis_allstream/cosmis_allstream_log\r\n\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputbsfile=/afs/cern.ch/user/c/cote/gencomm/nightlytestinput/data09_cos.00135664.physics_idcosmic.daq.raw._lb0000._sfo-2._0001.data maxevents=300 trigstream=idcosmic autoconfiguration=everything outputesdfile=myesd.pool.root outputmuoncalibntup=muoncalib.root outputpixelcalibntup=trkvalidntup.root hist=mymergedmonitoring.root outputtagfile=mytag.root\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (10 times)\r\nfatals=3\r\nerrors=7\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot find /tdaq/olc/calibrations in detectorstore\r\n  severity=error\r\n  who=toolsvc.lumiblockmutool\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot find /tdaq/olc/lhc/fillparams in detectorstore\r\n  severity=error\r\n  who=toolsvc.lumiblockmutool\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=cannot find /tdaq/olc/bunchlumis in detectorstore\r\n  severity=error\r\n  who=toolsvc.lumiblockmutool\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=retrieve(): no tool type/name given\r\n  severity=error\r\n  who=toolsvc\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=exception with tag=publictool retrieve is caught\r\n  severity=fatal\r\n  who=muoncombinedrefit\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=publictool retrieve	failed to retrieve publictool: 	 statuscode=failure\r\n  severity=error\r\n  who=muoncombinedrefit\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=exception with tag=publictool retrieve is caught\r\n  severity=fatal\r\n  who=athalgseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=publictool retrieve	failed to retrieve publictool: 	 statuscode=failure\r\n  severity=error\r\n  who=athalgseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=exception with tag=publictool retrieve is caught\r\n  severity=fatal\r\n  who=athmasterseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=publictool retrieve	failed to retrieve publictool: 	 statuscode=failure\r\n  severity=error\r\n  who=athmasterseq\r\n  producer=rawtoesd\r\n  stage=execute\r\nathacronym=exception raised during an algorithm execute\r\nathcode=64\r\nrundiruseddisk=18507587\r\nrundiravailabledisk=193341071360\r\nworkdir=/pool/lsf/rtt/176801336\r\nathcode=64\r\nrundiravailabledisk=193341370368\r\ndbrelease=\r\nathacronym=exception raised during an algorithm execute\r\nrundiruseddisk=18283467\r\natlasrelease=rel_1\r\nwriting jobinfo_recotrf.xml file...','100','5211','2011-10-11 16:36:29','5','none','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;error;occur;cosmis_allstream;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;cosmis_allstream;cosmis_allstream_log;report;produce;recotrf;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;user;cote;gencomm;nightlytestinput;data09_cos;physics_idcosmic.daq.raw;datum;maxevent;trigstream;idcosmic;autoconfiguration;outputesdfile;myesd.pool.root;outputmuoncalibntup;muoncalib.root;outputpixelcalibntup;trkvalidntup.root;hist;mymergedmonitoring.root;outputtagfile;mytag.root;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;tdaq;calibration;detectorstore;severity;error;toolsvc.lumiblockmutool;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;tdaq;fillparam;detectorstore;severity;error;toolsvc.lumiblockmutool;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;tdaq;bunchlumi;detectorstore;severity;error;toolsvc.lumiblockmutool;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;retrieve;tool;type;give;severity;error;toolsvc;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;exception;publictool;retrieve;catch;severity;fatal;muoncombinedrefit;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;publictool;retrieve;fail;retrieve;publictool;statuscode;failure;severity;error;muoncombinedrefit;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;exception;publictool;retrieve;catch;severity;fatal;athalgseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;publictool;retrieve;fail;retrieve;publictool;statuscode;failure;severity;error;athalgseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;exception;publictool;retrieve;catch;severity;fatal;athmasterseq;producer;rawtoesd;stage;execute;errorinfo;acronym;trf_unknown;message;publictool;retrieve;fail;retrieve;publictool;statuscode;failure;severity;error;athmasterseq;producer;rawtoesd;stage;execute;athacronym;exception;raise;algorithm;execute;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;exception;raise;algorithm;execute;rundiruseddisk;atlasrelease;rel_1;write;jobinfo_recotrf.xml;file;test;tdaq;calibration;folder'),(87677,'atlas','rtt error in mc10_bstoaod and mc10_default_physvalmon','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) an error occurred for mc10_bstoaod and mc10_default_physvalmon.\r\n\r\nsee logfile:\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/mc10_bstoaod/mc10_bstoaod_log\r\n\r\npy:rawtoesd          info rawtoesd has completed running of athena with exit code 33.\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/share/atlas_error_categories.db\r\ninfo: read 53 error patterns from file /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/share/atlas_error_patterns.db\r\ninfo: read 61 ignore patterns from file /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/share/atlas_error_ignore.db\r\nathenalogchecker start [rawtoesd log] wall-time: mon oct 10 09:47:02 2011 cest, vmem: 288.50390625 mb, rss: 97.90234375 mb\r\npy:rawtoesd          info scanning athena logfile rawtoesd.log assuming atlas release all,rel_1 ...\r\npy:rawtoesd          info athena initialise()...\r\npy:rawtoesd          info done scanning 2746 lines of file rawtoesd.log. summary:\r\npy:rawtoesd          info    ignored : 3\r\npy:rawtoesd          info    warnings: 6\r\npy:rawtoesd          info    errors  : 4\r\npy:rawtoesd          info    fatals  : 2\r\nathenalogchecker stop [rawtoesd log] wall-time: mon oct 10 09:47:02 2011 cest, vmem: 288.50390625 mb, rss: 97.97265625 mb\r\nathenalogchecker report [rawtoesd log] wall-time: 00 hrs 00 mins 00 secs, cpu-time: 00 hrs 00 mins 00 secs, vmem: 0.0 mb, rss: 0.0703125 mb\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: mon oct 10 09:47:02 2011 cest, vmem: 288.50390625 mb, rss: 98.0078125 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 00 hrs 01 mins 49 secs, cpu-time: 00 hrs 01 mins 48 secs, vmem: 0.38671875 mb, rss: 0.56640625 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputbsfile=t1_mcatnlo_jimmy_1.data,t1_mcatnlo_jimmy_2.data,t1_mcatnlo_jimmy_3.data outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root maxevents=-1 autoconfiguration=everything\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (4 times)\r\nerrorcategory=trf_svrinit,	61200,	servicemanager unable to initialize service (2 times)\r\nfatals=2\r\nerrors=4\r\nwarnings=6\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 69999 (exit code 99)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: mon oct 10 09:47:02 2011 cest, vmem: 288.828125 mb, rss: 98.30859375 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 00 hrs 01 mins 49 secs, cpu-time: 00 hrs 01 mins 48 secs, vmem: 0.7109375 mb, rss: 0.8984375 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputbsfile=t1_mcatnlo_jimmy_1.data,t1_mcatnlo_jimmy_2.data,t1_mcatnlo_jimmy_3.data outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root maxevents=-1 autoconfiguration=everything\r\nexitcode=99\r\nerrorcode=69999 (trf_unknown)\r\nerrorcategory=trf_unknown,	69999,	unknown transform error (4 times)\r\nerrorcategory=trf_svrinit,	61200,	servicemanager unable to initialize service (2 times)\r\nfatals=2\r\nerrors=4\r\nwarnings=6\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=folder /digitization/parameters does not exist\r\n  severity=fatal\r\n  who=iovdbsvc\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=preloadfolder failed for folder /digitization/parameters\r\n  severity=error\r\n  who=iovdbsvc\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_svrinit (61200) servicemanager unable to initialize service\r\n  message=error unable to initialize service &quot;detectorstore&quot;\r\n  severity=error\r\n  who=servicemanager\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=servicelocatorhelper::service: can not locate service detectorstore\r\n  severity=error\r\n  who=geomodelsvc\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_unknown (69999)\r\n  message=detectorstore service not found!\r\n  severity=fatal\r\n  who=geomodelsvc\r\n  producer=rawtoesd\r\n  stage=initialise\r\nerrorinfo\r\n  acronym=trf_svrinit (61200) servicemanager unable to initialize service\r\n  message=error unable to initialize service: geomodelsvc\r\n  severity=error\r\n  who=servicemanager\r\n  producer=rawtoesd\r\n  stage=initialise\r\nathacronym=failure in initialization\r\nathcode=33\r\nrundiruseddisk=731292106\r\nrundiravailabledisk=192971173888\r\nworkdir=/pool/lsf/rtt/176800902\r\nathcode=33\r\nrundiravailabledisk=192971407360\r\ndbrelease=\r\nathacronym=failure in initialization\r\nrundiruseddisk=731073345\r\natlasrelease=rel_1\r\nwriting jobinfo_recotrf.xml file...\r\n','100','5211','2011-10-11 16:41:30','5','duplicate','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;error;occur;mc10_bstoaod;mc10_default_physvalmon;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;mc10_bstoaod;mc10_bstoaod_log;py:rawtoesd;info;rawtoesd;complete;athena;exit;code;info;read;error;category;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;share;atlas_error_categories.db;info;read;error;pattern;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;share;atlas_error_patterns.db;info;read;ignore;pattern;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;share;atlas_error_ignore.db;athenalogchecker;start;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;scanning;athena;logfile;rawtoesd.log;assume;atla;release;rel_1;py:rawtoesd;info;athena;initialise;py:rawtoesd;info;scanning;line;file;rawtoesd.log;summary;py:rawtoesd;info;ignore;py:rawtoesd;info;warning;py:rawtoesd;info;error;py:rawtoesd;info;fatal;athenalogchecker;stop;rawtoesd;wall;time;cest;vmem;athenalogchecker;report;rawtoesd;wall;time;time;vmem;py:rawtoesd;info;jobtransform;complete;rawtoesd;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;rawtoesd;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;rawtoesd;wall;time;time;vmem;report;produce;rawtoesd;version;unknown;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;t1_mcatnlo_jimmy_1;datum;t1_mcatnlo_jimmy_2;datum;t1_mcatnlo_jimmy_3;datum;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;maxevent;autoconfiguration;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;errorcategory;trf_svrinit;servicemanager;unable;initialize;service;time;fatal;error;warning;write;jobinfo_rawtoesd.xml;file;esdok;false;skip;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputbsfile;t1_mcatnlo_jimmy_1;datum;t1_mcatnlo_jimmy_2;datum;t1_mcatnlo_jimmy_3;datum;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;maxevent;autoconfiguration;exitcode;errorcode;trf_unknown;errorcategory;trf_unknown;unknown;transform;error;time;errorcategory;trf_svrinit;servicemanager;unable;initialize;service;time;fatal;error;warning;errorinfo;acronym;trf_unknown;message;folder;digitization;parameter;exist;severity;fatal;iovdbsvc;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;preloadfolder;fail;folder;digitization;parameter;severity;error;iovdbsvc;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_svrinit;servicemanager;unable;initialize;service;message;error;unable;initialize;service;detectorstore;severity;error;servicemanager;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;servicelocatorhelper;service;locate;service;detectorstore;severity;error;geomodelsvc;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_unknown;message;detectorstore;service;severity;fatal;geomodelsvc;producer;rawtoesd;stage;initialise;errorinfo;acronym;trf_svrinit;servicemanager;unable;initialize;service;message;error;unable;initialize;service;geomodelsvc;severity;error;servicemanager;producer;rawtoesd;stage;initialise;athacronym;failure;initialization;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;failure;initialization;rundiruseddisk;atlasrelease;rel_1;write;jobinfo_recotrf.xml;file;error;mc10_bstoaod;mc10_default_physvalmon'),(87679,'atlas','rtt test in mc10_stdcmalloc_physvalmon not running','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) an error occurred for mc10_stdcmalloc_physvalmon.\r\n\r\nsee logfile:\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/mc10_stdcmalloc_physvalmon/mc10_stdcmalloc_physvalmon_log\r\n\r\ninfo: read 109 error categories from file /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/share/atlas_error_categories.db\r\n===== job report produced by esdtoaod version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputesdfile=t1_mcatnlo_jimmy_stdm.esd.pool.root outputhist_physvalmonfile=pvm_std.root maxevents=-1 autoconfiguration=everything --athenaopts=--stdcmalloc\r\nexitcode=32\r\nerrorcode=63200 (trf_def)\r\nerrorcategory=trf_def,	63200,	transform definition errors\r\nfatals=1\r\nerrors=0\r\nwarnings=0\r\nwriting jobinfo_esdtoaod.xml file...\r\npy:reco_trf         error during execution of reco_trf, exception caught: \'attempting to add a report (task esdtoaod) with missing athcode/athacronym.\'\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 63400 (exit code 34)\r\npy:reco_trf          info transform timer stop [reco_trf] wall-time: mon oct 10 17:51:56 2011 cest, vmem: 287.234375 mb, rss: 97.578125 mb\r\npy:reco_trf          info transform timer report [reco_trf] wall-time: 00 hrs 00 mins 02 secs, cpu-time: 00 hrs 00 mins 00 secs, vmem: 0.0 mb, rss: 0.16015625 mb\r\n===== job report produced by reco_trf version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputesdfile=t1_mcatnlo_jimmy_stdm.esd.pool.root outputhist_physvalmonfile=pvm_std.root maxevents=-1 autoconfiguration=everything --athenaopts=--stdcmalloc\r\nexitcode=34\r\nerrorcode=63400 (trf_exc)\r\nerrorcategory=trf_exc,	63400,	transform unknown exceptions\r\nfatals=1\r\nerrors=0\r\nwarnings=0\r\nerrorinfo\r\n  acronym=trf_exc (63400) transform unknown exceptions\r\n  message=keyerror: (\'attempting to add a report (task esdtoaod) with missing athcode/athacronym.\',)\r\n  severity=fatal\r\n  producer=reco_trf\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/python/pyjobtransformscore/trf.py&quot;, line 2193, in execute\r\n              self._jobreport.addreport( self.runjob(), \'merge\' )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py&quot;, line 162, in runjob\r\n              report.addreport( reportaod )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/python/pyjobtransformscore/jobreport.py&quot;, line 808, in addreport\r\n              raise keyerror( \'attempting to add a report (task %s) with missing athcode/athacronym.\' % report._task.taskname() )\r\nrundiruseddisk=115461\r\nrundiravailabledisk=159610011648\r\nwriting jobinfo_reco_trf.xml file...\r\n','1658','5211','2011-10-11 16:48:06','5','none','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;error;occur;mc10_stdcmalloc_physvalmon;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;mc10_stdcmalloc_physvalmon;mc10_stdcmalloc_physvalmon_log;info;read;error;category;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;share;atlas_error_categories.db;report;produce;esdtoaod;version;unknown;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputesdfile;t1_mcatnlo_jimmy_stdm.esd.pool.root;outputhist_physvalmonfile;pvm_std.root;maxevent;autoconfiguration;athenaopt;stdcmalloc;exitcode;errorcode;trf_def;errorcategory;trf_def;transform;definition;error;fatal;error;warning;write;jobinfo_esdtoaod.xml;file;py:reco_trf;error;execution;reco_trf;exception;catch;attempt;report;task;esdtoaod;miss;athcode;athacronym;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:reco_trf;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:reco_trf;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;reco_trf;version;unknown;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputesdfile;t1_mcatnlo_jimmy_stdm.esd.pool.root;outputhist_physvalmonfile;pvm_std.root;maxevent;autoconfiguration;athenaopt;stdcmalloc;exitcode;errorcode;trf_exc;errorcategory;trf_exc;transform;unknown;exception;fatal;error;warning;errorinfo;acronym;trf_exc;transform;unknown;exception;message;keyerror;attempt;report;task;esdtoaod;miss;athcode;athacronym;severity;fatal;producer;reco_trf;backtrace;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;python;pyjobtransformscore;trf.py;line;execute;self;self.runjob;merge;file;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;line;runjob;report.addreport;reportaod;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;python;pyjobtransformscore;jobreport.py;line;addreport;raise;keyerror;attempt;report;task;miss;athcode;athacronym;report;rundiruseddisk;rundiravailabledisk;write;jobinfo_reco_trf.xml;file;test;mc10_stdcmalloc_physvalmon'),(87680,'atlas','rtt error in mc11_default - trf_outfile_notread','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) an error occurred for mc11_default.\r\n\r\nsee logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recjobtransformtests/mc11_default/mc11_default_log\r\n\r\npy:rawtoesd.outputesdfile    info no event count available for t1_mcatnlo_jimmy.esd.pool.root\r\npy:rawtoesd          info transform timer stop [outputesdfile validation] wall-time: mon oct 10 11:08:00 2011 cest, vmem: 623.11328125 mb, rss: 330.0234375 mb\r\npy:rawtoesd          info transform timer report [outputesdfile validation] wall-time: 00 hrs 47 mins 53 secs, cpu-time: 00 hrs 00 mins 01 secs, vmem: 0.0 mb, rss: 0.00390625 mb\r\n========o= inputfilepeeker ============\r\noutputesdfile t1_mcatnlo_jimmy.esd.pool.root\r\npy:athfile           info loading cache from [athfile-cache.ascii.gz]...\r\npy:athfile           info loading cache from [athfile-cache.ascii.gz]... [done]\r\ntimeout 120 for &lt;bound method rawtoesdjobtransform.doupload of &lt;recjobtransforms.rawtoesd_trf.rawtoesdjobtransform object at 0x18a78290&gt;&gt;\r\npy:rawtoesd          info maxevents &lt; input_events. matchevents not executed.\r\npy:rawtoesd          info scanning logfile rawtoesd.log for metadata...\r\npy:rawtoesd          info jobtransform completed for rawtoesd with error code 64120 (exit code 41)\r\npy:rawtoesd          info transform timer stop [rawtoesd] wall-time: mon oct 10 11:10:17 2011 cest, vmem: 674.42578125 mb, rss: 336.14453125 mb\r\npy:rawtoesd          info transform timer report [rawtoesd] wall-time: 01 hrs 21 mins 20 secs, cpu-time: 00 hrs 24 mins 05 secs, vmem: 387.1796875 mb, rss: 238.69921875 mb\r\n===== job report produced by rawtoesd version unknown =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputrdofile=/afs/cern.ch/atlas/project/rig/referencefiles/mc/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e835_s1256_s1258_d566_tid464353_00/rdo.464353._000001.pool.root.1 outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root outputhistfile=t1_mcatnlo_jimmy.hist.root maxevents=50 autoconfiguration=everything\r\nexitcode=41\r\nerrorcode=64120 (trf_outfile_notread)\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nwriting jobinfo_rawtoesd.xml file...\r\nesdok is  false\r\nskipping esd-&gt;esd step...\r\nskipping esd-&gt;aod step...\r\nskipping esd-&gt;dpd step...\r\nskipping aod-&gt;dpd step...\r\nskipping aod-&gt;tag step...\r\nskipping monitoring file merging step...\r\npy:reco_trf          info jobtransform completed for reco_trf with error code 64120 (exit code 41)\r\npy:rawtoesd          info transform timer stop [reco_trf] wall-time: mon oct 10 11:10:17 2011 cest, vmem: 674.42578125 mb, rss: 336.16796875 mb\r\npy:rawtoesd          info transform timer report [reco_trf] wall-time: 01 hrs 21 mins 20 secs, cpu-time: 00 hrs 24 mins 05 secs, vmem: 387.1796875 mb, rss: 238.75390625 mb\r\n===== job report produced by recotrf =====\r\ncommand=/afs/cern.ch/atlas/software/builds/nightlies/devval/atlasreconstruction/rel_1/installarea/share/bin/reco_trf.py --uploadtoami=1.0 inputrdofile=/afs/cern.ch/atlas/project/rig/referencefiles/mc/valid1.105200.t1_mcatnlo_jimmy.digit.rdo.e835_s1256_s1258_d566_tid464353_00/rdo.464353._000001.pool.root.1 outputesdfile=t1_mcatnlo_jimmy.esd.pool.root outputaodfile=t1_mcatnlo_jimmy.aod.pool.root outputhistfile=t1_mcatnlo_jimmy.hist.root maxevents=50 autoconfiguration=everything\r\nexitcode=41\r\nerrorcode=64120 (trf_outfile_notread)\r\nfatals=1\r\nerrors=0\r\nwarnings=10\r\nerrorinfo\r\n  acronym=trf_outfile_notread (64120) transform output file not readable\r\n  message=output file t1_mcatnlo_jimmy.esd.pool.root failed validation. events could not be counted argument outputesdfile\r\n  severity=fatal\r\n  producer=rawtoesd\r\n  backtrace=file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/python/pyjobtransformscore/trf.py&quot;, line 2003, in dopostrunactions\r\n              cmd.postrunaction()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1197, in postrunaction\r\n              self.validatefile()\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/python/pyjobtransformscore/basic_trfarg.py&quot;, line 1184, in validatefile\r\n              self._filetype.validatefile( self, **self.__validationdict )\r\n            file &quot;/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_1/installarea/python/pyjobtransformscore/trfutil.py&quot;, line 1394, in validatefile\r\n              raise transformvalidationerror( fname, \'failed validation. events could not be counted argument %s\' % argname, \'trf_outfile_notread\' )\r\nathacronym=successful run\r\nathcode=0\r\nrundiruseddisk=132887563\r\nrundiravailabledisk=169160564736\r\nworkdir=/pool/lsf/rtt/176800925\r\nathcode=0\r\nrundiravailabledisk=169160802304\r\ndbrelease=\r\nathacronym=successful run\r\nrundiruseddisk=132660651\r\natlasrelease=rel_1\r\nwriting jobinfo_recotrf.xml file...','100','5211','2011-10-11 16:49:47','5','none','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;error;occur;mc11_default;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recjobtransformtest;mc11_default;mc11_default_log;py:rawtoesd.outputesdfile;info;event;count;available;t1_mcatnlo_jimmy.esd.pool.root;py:rawtoesd;info;transform;timer;stop;outputesdfile;validation;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;outputesdfile;validation;wall;time;time;vmem;inputfilepeeker;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;py:athfile;info;load;cache;athfile;cache.ascii.gz;py:athfile;info;load;cache;athfile;cache.ascii.gz;timeout;py:rawtoesd;info;maxevent;step;skip;step;skip;step;skip;step;skip;step;skip;monitoring;file;merge;step;py:reco_trf;info;jobtransform;complete;reco_trf;error;code;exit;code;py:rawtoesd;info;transform;timer;stop;reco_trf;wall;time;cest;vmem;py:rawtoesd;info;transform;timer;report;reco_trf;wall;time;time;vmem;report;produce;recotrf;command;atla;software;build;nightly;devval;atlasreconstruction;rel_1;installarea;share;reco_trf.py;uploadtoamus;inputrdofile;atla;project;referencefile;valid1.105200;t1_mcatnlo_jimmy.digit.rdo.e835_s1256_s1258_d566_tid464353_00;pool.root;outputesdfile;t1_mcatnlo_jimmy.esd.pool.root;outputaodfile;t1_mcatnlo_jimmy.aod.pool.root;outputhistfile;t1_mcatnlo_jimmy.hist.root;maxevent;autoconfiguration;exitcode;errorcode;trf_outfile_notread;fatal;error;warning;errorinfo;acronym;trf_outfile_notread;transform;output;file;readable;message;output;file;t1_mcatnlo_jimmy.esd.pool.root;fail;validation;event;count;argument;outputesdfile;severity;fatal;producer;rawtoesd;backtrace;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;python;pyjobtransformscore;trf.py;line;dopostrunaction;cmd.postrunaction;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;python;pyjobtransformscore;basic_trfarg.py;line;postrunaction;self.validatefile;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;python;pyjobtransformscore;basic_trfarg.py;line;validatefile;self;self;self;file;atla;software;build;nightly;devval;atlascore;rel_1;installarea;python;pyjobtransformscore;trfutil.py;line;validatefile;raise;transformvalidationerror;fname;fail;validation;event;count;argument;argname;trf_outfile_notread;athacronym;successful;athcode;rundiruseddisk;rundiravailabledisk;workdir;pool;athcode;rundiravailabledisk;dbrelease;athacronym;successful;rundiruseddisk;atlasrelease;rel_1;write;jobinfo_recotrf.xml;file;error;mc11_default;trf_outfile_notread'),(87681,'atlas','problem in runscript for details_esdtoaod_bs_log','dear experts,\r\n\r\nin todays 17.x.0-val (devval) (rel_1) (x86_64-slc5-gcc43-opt) an error occurred for details_esdtoaod_bs_log\r\n\r\n\r\nsee logfile:\r\n\r\nhttps://atlasrtt.cern.ch/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recperftests/details_esdtoaod_bs/details_esdtoaod_bs_log\r\n\r\n/afs/cern.ch/atlas/project/rtt/prod/results/rtt_j_r/rel_1/devval/build/x86_64-slc5-gcc43-opt/offline/recperftests/details_esdtoaod_bs/runscript.sh: line 74: --athenaopts=--stdcmalloc: command not found','916','5211','2011-10-11 17:02:08','5','fixed','closed','none',';dear;expert;today;devval;rel_1;x86_64;slc5;gcc43;error;occur;details_esdtoaod_bs_log;logfile;http;atlasrtt.cern.ch;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recperftest;details_esdtoaod_bs;details_esdtoaod_bs_log;atla;project;prod;result;rtt_j_r;rel_1;devval;build;x86_64;slc5;gcc43;offline;recperftest;details_esdtoaod_bs;runscript.sh;line;athenaopt;stdcmalloc;command;problem;runscript;details_esdtoaod_bs_log'),(87697,'atlas','seg fault in heavyiond3pd in tct tests for 17.0.x.y-val, rel3','dear all,\r\n\r\na seg fault is occurring in heavyiond3pd in 3 tct tests for 17.0.x.y-val, rel3:\r\nrun00169564_express0_hicollisions\r\nrun00169751_highpt2_hicollisions\r\nrun00169751_highpt_hicollisions\r\n\r\n\r\nheavyiond3pd.trigconfmetadatatool                    info initializing. package version: triggerd3pdmaker-00-01-64\r\n-------------------------------------------------------------------------------------\r\ncore dump from coredumpsvc on lxb8303.cern.ch at wed oct 12 04:20:47 2011\r\n\r\ncaught signal 11(segmentation fault). details:\r\n  errno = 0, code = 1 (address not mapped to object)\r\n  pid   = 1, uid = 1491989150\r\n  value = (8866622, 0x874b3e)\r\n  vmem = 600.465 mb\r\n  rss  = 401.16 mb\r\n  total-ram = 15925.7 mb\r\n  free-ram  = 956.039 mb\r\n  buffer-ram= 300.281 mb\r\n  total-swap= 4094.55 mb\r\n  free-swap = 4043 mb\r\n  addr  = 0x1\r\n\r\nevent counter: 0\r\ncurrent algorithm: heavyiond3pd\r\nalgorithm stack:\r\n   athmasterseq\r\n   athalgseq\r\n   heavyiond3pd_filteralgorithmswrap   heavyiond3pd_filteralgorithmsholder\r\n   heavyiond3pd\r\n-------------------------------------------------------------------------------------\r\n\r\n(pid=1166 ppid=18934) received fatal signal 11 (segmentation fault)signal context:  signo  = 11, errno = 0, code = 1 (address not mapped to object)  pid    = 1, uid = 1491989150  value  = (8866622, 0x874b3e)  addr   = 0x1\r\n  stack  = (2, 0, (nil))\r\n\r\n  eip: 0023:5aa6b557           eflags: 00210292\r\n  eax: 00000001   ebx: 686f3b18   ecx: 556ce6c8   edx: 00000000\r\n  esi: 13814984   edi: 5f6a39c8   ebp: ffaf7f58   esp: ffaf7e2c\r\n   ds: 002b        es: 002b        fs: 0000        ss: 002b\r\n\r\n  signal esp: ffaf7e2c  trap: 14/4  oldmask: 00000000   cr2: 00000001\r\n\r\n  fpu:  control = ffff037f\r\n        status  = ffff0120\r\n        tag     = ffffffff\r\n        ip      = 0010:55ac373d\r\n        data    = 002b:00000000\r\n        state   = 00000120\r\n    %fp0 = [0000:0000000000000000]\r\n    %fp1 = [0000:0000000000000000]\r\n    %fp2 = [0000:0000000000000000]\r\n    %fp3 = [0000:0000000000000000]\r\n    %fp4 = [4011:000000000000dc11]\r\n    %fp5 = [400f:000000006600f9b3]\r\n    %fp6 = [3fff:0000000000008000]\r\n    %fp7 = [4004:000000000000c800]\r\n\r\n\r\na full logfile can be found in\r\nhttps://atlasrtt.cern.ch/tct/rel_3/17.0.x.y-val/build/i686-slc5-gcc43-opt/offline/tier0chaintests/run00169564_express0_hicollisions/run00169564_express0_hicollisions_log\r\n\r\n###############################\r\noriginal job transform command:\r\n/afs/cern.ch/atlas/software/builds/nightlies/17.0.x.y-val/atlasproduction/rel_3/installarea/share/bin/reco_trf.py outputhistfile=mymergedmonitoring_express_0.root ami=q202 maxevents=100 preinclude=recjobtransforms/heavyiondatarecconfig.py outputesdfile=myesd_express_0.pool.root outputntup_trkvalidfile=trkvalid_express_0.ntup.root outputtagfile=mytag_express_0.root --uploadtoami=0.0 outputntup_trigfile=trig_express_0.root inputbsfile=0.data,1.data,2.data outputntup_hifile=hi_express_0.ntup.root\r\n###############################\r\n\r\n\r\nregards\r\nmatilde\r\n(valshift)\r\n','100','5211','2011-10-12 09:51:43','5','none','closed','heavyionrec',';dear;fault;occur;heavyiond3pd;test;rel3;run00169564_express0_hicollisions;run00169751_highpt2_hicollisions;run00169751_highpt_hicollisions;heavyiond3pd.trigconfmetadatatool;info;initialize;package;version;triggerd3pdmaker;core;dump;coredumpsvc;lxb8303;catch;signal;segmentation;fault;detail;errno;code;address;object;value;vmem;total;free;buffer;total;swap;free;swap;addr;event;counter;current;algorithm;heavyiond3pd;algorithm;stack;athmasterseq;athalgseq;heavyiond3pd_filteralgorithmswrap;heavyiond3pd_filteralgorithmsholder;heavyiond3pd;ppid;receive;fatal;signal;segmentation;fault;signal;context;signo;errno;code;address;object;value;addr;stack;eflag;esus;edus;signal;trap;oldmask;control;status;datum;state;logfile;http;atlasrtt.cern.ch;rel_3;build;slc5;gcc43;offline;tier0chaintest;run00169564_express0_hicollisions;run00169564_express0_hicollisions_log;original;transform;command;atla;software;build;nightly;atlasproduction;rel_3;installarea;share;reco_trf.py;outputhistfile;mymergedmonitoring_express_0;root;maxevent;preinclude;recjobtransform;heavyiondatarecconfig.py;outputesdfile;myesd_express_0;pool.root;outputntup_trkvalidfile;trkvalid_express_0;ntup.root;outputtagfile;mytag_express_0;root;uploadtoamus;outputntup_trigfile;trig_express_0;root;inputbsfile;datum;datum;datum;outputntup_hifile;hi_express_0;ntup.root;matilde;valshift;fault;heavyiond3pd;test;rel3;heavyionrec');
