hi all,

i am getting an error reading release 16 data (d)aod with recexcommon, which i've never seen before:

beamcondsvc                                          info read from conddb status 59 pos (-0.0671893,1.03926,-5.83673) sigma (0.0214158,0.0205542,52.4213) tilt (-1.9359e-05,-6.08959e-05) sigmaxy -2.82645e-05
detectorstore                                     warning retrieve(const): no valid proxy for object atlastrackinggeometry  of type trk::trackinggeometry(clid 167645219)
toolsvc.atlasnavigator                              fatal could not retrieve trackinggeometry 'atlastrackinggeometry' from detectorstore.
toolsvc.atlasnavigator                              fatal   - probably the chosen layout is not supported / no cool tag exists. 

some points about the error:

(1) it occurs usually after many thousands of events have been processed
(2) in a set of around 200 grid jobs around 10% succeed and the rest crash with this error. the error is reproducible in that if i resubmit the jobs, the same crashes occur at the same places. i have not yet tried to do it interactively but will try that next and 
(3) it affects all sites i've tried running on
(4) it does not happen with mc
(5) i'm using a recipe which i've successfully used for the last 2 years, and which was working until last week
(6) i've tried with 17.0.2 and 17.0.3, and the result is the same
(7) the job is simply reading a few track parameters and writing them to an n-tuple
(8) i'm told the beamcondsvc message is harmless, but i observe that it doesn't appear in the successful jobs.

sample logfiles: 

http://voatlas139.cern.ch/monitor/logs/d66cf782-20bf-4c75-97eb-c2614deb1ede/tarball_pandajob_1308158517_analy_jinr/athena_stdout.txt

http://voatlas141.cern.ch:25980/monitor/logs/751451a6-438a-4503-a5d4-6a4f8913d141/tarball_pandajob_1308158468_analy_lancs/athena_stdout.txt

here is an example of a successful job in the same batch (actually from the same run as the above file):

http://voatlas140.cern.ch/monitor/logs/5c40cba7-7d34-4d95-b86f-8fffa5057d1e/tarball_pandajob_1308158467_analy_lancs/athena_stdout.txt

i attach my master job options below. the quoted file is just a local handle for the autoconfig and isn't actually processed by the job.

cheers,

james.

from athenacommon.athenacommonflags import athenacommonflags
athenacommonflags.filesinput = [ &quot;/afs/cern.ch/atlas/maxidisk/d129/realdata/data11_7tev.00180225.physics_muons.merge.daod_oniamumu.f368_m812_p551_p552/daod_oniamumu.334686._000022.pool.root.1&quot; ]
from recexconfig.recflags import rec
rec.dotrigger.set_value_and_lock(false) 
rec.useralgs.set_value_and_lock(&quot;analyzebskeletonwithtools.py&quot;)
rec.outputlevel.set_value_and_lock(info);
rec.docbnt.set_value_and_lock(false)
rec.dowriteaod.set_value_and_lock (false)
rec.dowritetag.set_value_and_lock (false)
rec.dohist.set_value_and_lock (false)
include (&quot;recexcommon/recexcommon_topoptions.py&quot;)
theapp.evtmax = -1 # number of event to process
from perfmoncomps.perfmonflags import jobproperties as jp
jp.perfmonflags.domonitoring = false
jp.perfmonflags.dofastmon = false