hi there,

here has been a major failure in the rtt tests for the devval for the past few days, and it was suggested this was the best place to submit.

https://atlasrtt.cern.ch/rtt/rel_0/devval/build/i686-slc5-gcc43-opt/offline/trigindetvalidation/trigindetvalidation_tau/trigindetvalidation_tau_log

the above log file is for the tau test in rel_0


all the devval rtt tests for the trigindetvalidation package fail with the same segfault:



geomodelsvc.muondetectortool                       info muonalignmentdbtool retrieved with statuscode = success pointer = themuonalignmenttool = publictoolhandle('muonalignmentdbtool/mgm_alignmentdbtool')
geomodelsvc                                        info retrieved detectortools = privatetoolhandlearray(['pixeldetectortool/pixeldetectortool','sct_detectortool/sct_detectortool','trt_detectortool/trt_detectortool','indetservmattool/indetservmattool','lardetectortoolnv/lardetectortoolnv','tiledetectortool/tiledetectortool','muondetectortool/muondetectortool','lucid_detectortool/lucid_detectortool','beampipedetectortool/beampipedetectortool'])
coral/services/connectionservice info re-ordering replicas using installed algorithm.
coral/services/connectionservice warning failed to connect to service geomdb/geomdb_sqlite (coral::exception): 'coral/relationalplugins/sqlite ( coral : &quot;connection::connect&quot; from &quot;/pool/lsf/rtt/122625434/geomdb is not writable&quot; )' - do not retry
coral/services/connectionservice info connection to service &quot;geomdb/geomdb_sqlite&quot; with connectionid=0995ba84-3cea-11e0-94dd-001d09f27715 will be disconnected
coral/services/connectionservice warning failure while attempting to connect to &quot;sqlite_file:geomdb/geomdb_sqlite&quot;: coral/relationalplugins/sqlite ( coral : &quot;connection::connect&quot; from &quot;/pool/lsf/rtt/122625434/geomdb is not writable&quot; )
pluginmanager info pluginmanager creating coral/relationalplugins/oracle component
pluginmanager info pluginmanager needs to load coral/relationalplugins/oracle component
pluginmanager info pluginmanager loaded oracleaccess library
pluginmanager info pluginmanager loaded coral/relationalplugins/oracle factory from oracleaccess library
coral/services/connectionservice info loading default plugin for coral::iauthenticationservice: coral/services/xmlauthenticationservice
pluginmanager info pluginmanager creating coral/services/xmlauthenticationservice component
pluginmanager info pluginmanager needs to load coral/services/xmlauthenticationservice component
pluginmanager info pluginmanager loaded xmlauthenticationservice library
pluginmanager info pluginmanager loaded coral/services/xmlauthenticationservice factory from xmlauthenticationservice library
coral/services/xmlauthenticationservice info authentication using /afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/xml/atlasauthentication/authentication.xml file

this is seen before the seg fault occurs, and there isn't an indication to which algorithm causes the problem either. if further information is required then i'd be happy to give it.

vikash




the seg fault is printed below:

core dump from coredumpsvc on lxbrb2214.cern.ch at fri feb 18 14:04:28 2011

caught signal 11(segmentation fault). details: 
  errno = 0, code = 128 (kernel)
  pid   = 0, uid = 0
  value = (6159254, 0x5dfb96)
  vmem = 584.617 mb
  rss  = 466.133 mb
  total-ram = 15929.8 mb
  free-ram  = 2124.71 mb
  buffer-ram= 227.984 mb
  total-swap= 4094.55 mb
  free-swap = 4094.32 mb
  addr  = 0

event counter: 0
current algorithm: &lt;none&gt;
algorithm stack: &lt;empty&gt;

-------------------------------------------------------------------------------------

(pid=21037 ppid=19795) received fatal signal 11 (segmentation fault)
signal context:
  signo  = 11, errno = 0, code = 128 (kernel)
  pid    = 0, uid = 0
  value  = (6142870, 0x5dbb96)
  addr   = (nil)
  stack  = (2, 0, (nil))

  eip: 0023:5ad54153           eflags: 00210293
  eax: ffe828ac   ebx: 5ad99bfc   ecx: 00000002   edx: 00000003
  esi: 1396eefd   edi: 1396ed0d   ebp: ffe828cc   esp: ffe82744
   ds: 002b        es: 002b        fs: 0000        ss: 002b

  signal esp: ffe82744  trap: 13/0  oldmask: 00000000   cr2: 00000000

  fpu:  control = ffff0372
        status  = ffff0020
        tag     = ffffffff
        ip      = 0010:5ad50770
        data    = 002b:144a30b8
        state   = 00000020
    %fp0 = [3fff:0000000000008000]
    %fp1 = [0000:0000000000000000]
    %fp2 = [0000:0000000000000000]
    %fp3 = [0000:0000000000000000]
    %fp4 = [0000:0000000000000000]
    %fp5 = [8000:0000000000000000]
    %fp6 = [4006:000000000000c400]
    %fp7 = [0000:0000000000000000]

stack trace:
 0x5a9abff4 athena::debugaids::stacktrace(int) + 0x70 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]
 0x5a9acebd athena::signal::fataldump(int, siginfo*, void*) + 0x14d [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]
 0x5a9ad1ff athena::signal::fatal(int, siginfo*, void*) + 0x179 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libcxxutils.so]
 0x5a7620a5 coredumpsvchandler::action(int, siginfo*, void*) + 0xf3 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libathenaservices.so]
 0xffffe600 &lt;unknown function&gt;
 0x5ad54153 perfmonsvc::comp_stopaud(std::string const&amp;, std::string const&amp;) + 0x9cf [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libperfmoncomps.so]
 0x5ad5429e perfmonsvc::stopaud(std::string const&amp;, std::string const&amp;) + 0x5a [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libperfmoncomps.so]
 0x5ad504dd pmon_dso_cbk + 0x98 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libperfmoncomps.so]
 0x5ad9be64 ath_dlopen + 0xe8 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libathdsocallbacks.so]
 0x00770bcb dlopen + 0x6b [/lib/libdl.so.2]
0x61b91f43 slpmloclfv slpmloc.c:0 + 0x61b91f43 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61b91cfc slpmloc + 0xc0 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61b8fd6c lpmloadpkg + 0x130 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
0x61b7806d lfvloadpkg lfv.c:0 + 0x61b7806d [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
0x61b77d9f lfvsetshlmode lfv.c:0 + 0x61b77d9f [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61b77bbe lfvini1 + 0x184 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61b77a2f lfvini2 + 0x31 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61afd3d1 lxlinit + 0xc9 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61aa1619 nleminz + 0x65 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x62fa35f6 nlstdggo + 0xae8 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x62fa2b03 nlstdgg + 0x2b [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x619b299e nigini2 + 0x3a6 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
0x61d77e58 kpedbggetnpdglobal kpedbg.c:0 + 0x61d77e58 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61d77d84 kpedbgtlsinit + 0xcc [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x6307ed27 kpummtlsget1 + 0x83 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61d77ef6 kpedbgprocessinit + 0x44 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x61d7644c kpummpin + 0x174 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x618a4b8a kpuenvcr + 0xae [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x6187edfd ocienvcreate + 0x45 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/libclntsh.so.11.1]
 0x6159e4ce coral::oracleaccess::connection::connect() + 0x18e [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_oracleaccess.so]
 0x5e648b28 coral::connectionservice::connectionhandle::open() + 0x188 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_connectionservice.so]
 0x5e651349 coral::connectionservice::connectionpool::getsessionfromnewconnection(coral::connectionservice::connectionparams const&amp;, coral::accessmode) + 0x1d9 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_connectionservice.so]
 0x5e652433 coral::connectionservice::connectionpool::getvalidsession(std::string const&amp;, std::string const&amp;, coral::accessmode, coral::icertificatedata const*) + 0x513 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_connectionservice.so]
 0x5e666236 coral::connectionservice::sessionproxy::open(coral::icertificatedata const*) + 0x66 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_connectionservice.so]
 0x5e65acc8 coral::connectionservice::connectionservice::connect(std::string const&amp;, coral::accessmode, coral::icertificatedata const*) + 0x58 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_connectionservice.so]
 0x5e65b687 coral::connectionservice::connectionservice::connect(std::string const&amp;, coral::accessmode) + 0x27 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_connectionservice.so]
 0x5cdb6c18 coral::connectionservice::connect(std::string const&amp;, coral::accessmode) + 0x38 [/afs/cern.ch/atlas/offline/external/lcgcmt/lcgcmt_59a/installarea/i686-slc5-gcc43-opt/lib/liblcg_relationalaccess.so]
 0x615232e2 rdbaccesssvc::connect(std::string const&amp;) + 0x478 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/librdbaccesssvc.so]
 0x5ae5eafd rdbmaterialmanager::rdbmaterialmanager(storegatesvc*, isvclocator*) + 0x84f [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libgeomodelsvclib.so]
 0x5ae7c1a2 geomodelsvc::geoinit(int&amp;, std::list&lt;std::string, std::allocator&lt;std::string&gt; &gt;&amp;) + 0x5f4 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libgeomodelsvclib.so]
 0x5ae8046f geomodelsvc::initialize() + 0x1465 [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libgeomodelsvclib.so]
 0x5897d167 service::sysinitialize() + 0x97 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_0/installarea/i686-slc5-gcc43-opt/lib/libgaudikernel.so]
 0x5a8c68ea athservice::sysinitialize() + 0x12a [/afs/cern.ch/atlas/software/builds/nightlies/devval/atlascore/rel_0/installarea/i686-slc5-gcc43-opt/lib/libathenabasecomps.so]
 0x59f6b649 servicemanager::initialize() + 0x169 [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_0/installarea/i686-slc5-gcc43-opt/lib/libgaudisvc.so]
 0x59f7a8cb applicationmgr::initialize() + 0x75b [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_0/installarea/i686-slc5-gcc43-opt/lib/libgaudisvc.so]
0x5970116d _zn12_global__n_1l11method_5684epvs0_rkst6vectoris0_sais0_ees0_ dictionary_dict.cpp:0 + 0x5970116d [/afs/cern.ch/atlas/software/builds/nightlies/devval/gaudi/rel_0/installarea/i686-slc5-gcc43-opt/lib/libgaudikerneldict.so]
 0x580aa437 root::cintex::method_stub_with_context(root::cintex::stubcontext_t*, g__value*, char const*, g__param*, int) + 0x157 [/afs/cern.ch/atlas/software/builds/nightlies/devval/detcommon/rel_0/installarea/i686-slc5-gcc43-opt/lib/libcintex.so]
 0x5a42b034 &lt;unknown function&gt;
 0x56133512 cint::g__callfunc::execute(void*) + 0xd2 [/afs/cern.ch/atlas/software/builds/nightlies/devval/detcommon/rel_0/installarea/i686-slc5-gcc43-opt/lib/libcint.so]
 0x558c0cec pyroot::trootobjectbyvalueexecutor::execute(cint::g__callfunc*, void*) + 0x2c [/afs/cern.ch/sw/lcg/app/releases/root/5.26.00e_python2.6/i686-slc5-gcc43-opt/root/lib/libpyroot.so]
 0x558c6c24 pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::callsafe(void*) + 0x64 [/afs/cern.ch/sw/lcg/app/releases/root/5.26.00e_python2.6/i686-slc5-gcc43-opt/root/lib/libpyroot.so]
 0x558c6dbc pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::execute(void*) + 0x6c [/afs/cern.ch/sw/lcg/app/releases/root/5.26.00e_python2.6/i686-slc5-gcc43-opt/root/lib/libpyroot.so]
 0x558c5a92 pyroot::tmethodholder&lt;pyroot::tscopeadapter, pyroot::tmemberadapter&gt;::operator()(pyroot::objectproxy*, _object*, _object*, long) + 0x112 [/afs/cern.ch/sw/lcg/app/releases/root/5.26.00e_python2.6/i686-slc5-gcc43-opt/root/lib/libpyroot.so]
0x558cc859 pyroot::(anonymous namespace)::mp_call(pyroot::methodproxy*, _object*, _object*) methodproxy.cxx:0 + 0x558cc859 [/afs/cern.ch/sw/lcg/app/releases/root/5.26.00e_python2.6/i686-slc5-gcc43-opt/root/lib/libpyroot.so]
 0x0805d88c pyobject_call /build/agaspar/work/python-2.6.5/objects/abstract.c:2493 + 0x5c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080d911c ext_do_call /build/agaspar/work/python-2.6.5/python/ceval.c:4063 + 0x287c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080ddc8c pyeval_evalcodeex /build/agaspar/work/python-2.6.5/python/ceval.c:3000 + 0x87c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dc329 fast_function /build/agaspar/work/python-2.6.5/python/ceval.c:3846 + 0x5a89 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dcb7b fast_function /build/agaspar/work/python-2.6.5/python/ceval.c:3837 + 0x62db [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dcb7b fast_function /build/agaspar/work/python-2.6.5/python/ceval.c:3837 + 0x62db [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080ddc8c pyeval_evalcodeex /build/agaspar/work/python-2.6.5/python/ceval.c:3000 + 0x87c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dc329 fast_function /build/agaspar/work/python-2.6.5/python/ceval.c:3846 + 0x5a89 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080ddc8c pyeval_evalcodeex /build/agaspar/work/python-2.6.5/python/ceval.c:3000 + 0x87c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dddd3 pyeval_evalcode /build/agaspar/work/python-2.6.5/python/ceval.c:547 + 0x63 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080fe044 run_mod /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339 + 0xb4 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080d4900 builtin_execfile /build/agaspar/work/python-2.6.5/python/bltinmodule.c:797 + 0x80d4900 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x081391bd pycfunction_call /build/agaspar/work/python-2.6.5/objects/methodobject.c:81 + 0x18d [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dbd09 call_function /build/agaspar/work/python-2.6.5/python/ceval.c:3750 + 0x5469 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080ddc8c pyeval_evalcodeex /build/agaspar/work/python-2.6.5/python/ceval.c:3000 + 0x87c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x08137caa function_call /build/agaspar/work/python-2.6.5/objects/funcobject.c:524 + 0x8137caa [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x0805d88c pyobject_call /build/agaspar/work/python-2.6.5/objects/abstract.c:2493 + 0x5c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x08064935 instancemethod_call /build/agaspar/work/python-2.6.5/objects/classobject.c:2579 + 0x8064935 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x0805d88c pyobject_call /build/agaspar/work/python-2.6.5/objects/abstract.c:2493 + 0x5c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080b29e3 slot_tp_call /build/agaspar/work/python-2.6.5/objects/typeobject.c:5373 + 0x80b29e3 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x0805d88c pyobject_call /build/agaspar/work/python-2.6.5/objects/abstract.c:2493 + 0x5c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080d969d do_call /build/agaspar/work/python-2.6.5/python/ceval.c:3968 + 0x2dfd [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080ddc8c pyeval_evalcodeex /build/agaspar/work/python-2.6.5/python/ceval.c:3000 + 0x87c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dddd3 pyeval_evalcode /build/agaspar/work/python-2.6.5/python/ceval.c:547 + 0x63 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080fe044 run_mod /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339 + 0xb4 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080d4900 builtin_execfile /build/agaspar/work/python-2.6.5/python/bltinmodule.c:797 + 0x80d4900 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x081391bd pycfunction_call /build/agaspar/work/python-2.6.5/objects/methodobject.c:81 + 0x18d [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dbd09 call_function /build/agaspar/work/python-2.6.5/python/ceval.c:3750 + 0x5469 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080ddc8c pyeval_evalcodeex /build/agaspar/work/python-2.6.5/python/ceval.c:3000 + 0x87c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x08137caa function_call /build/agaspar/work/python-2.6.5/objects/funcobject.c:524 + 0x8137caa [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x0805d88c pyobject_call /build/agaspar/work/python-2.6.5/objects/abstract.c:2493 + 0x5c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x08064935 instancemethod_call /build/agaspar/work/python-2.6.5/objects/classobject.c:2579 + 0x8064935 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x0805d88c pyobject_call /build/agaspar/work/python-2.6.5/objects/abstract.c:2493 + 0x5c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080b29e3 slot_tp_call /build/agaspar/work/python-2.6.5/objects/typeobject.c:5373 + 0x80b29e3 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x0805d88c pyobject_call /build/agaspar/work/python-2.6.5/objects/abstract.c:2493 + 0x5c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080d969d do_call /build/agaspar/work/python-2.6.5/python/ceval.c:3968 + 0x2dfd [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080ddc8c pyeval_evalcodeex /build/agaspar/work/python-2.6.5/python/ceval.c:3000 + 0x87c [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080dddd3 pyeval_evalcode /build/agaspar/work/python-2.6.5/python/ceval.c:547 + 0x63 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080fe044 run_mod /build/agaspar/work/python-2.6.5/python/pythonrun.c:1339 + 0xb4 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080fe3a4 pyrun_simplefileexflags /build/agaspar/work/python-2.6.5/python/pythonrun.c:935 + 0x204 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x080fe69a pyrun_anyfileexflags /build/agaspar/work/python-2.6.5/python/pythonrun.c:739 + 0x7a [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x08058d46 py_main /build/agaspar/work/python-2.6.5/modules/main.c:572 + 0xc56 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x08057e42 main /build/agaspar/work/python-2.6.5/./modules/python.c:24 + 0x32 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]
 0x005ffe9c __libc_start_main + 0xdc [/lib/libc.so.6]
 0x08057d51 _start + 0x8057d51 [/afs/cern.ch/sw/lcg/external/python/2.6.5/i686-slc5-gcc43-opt/bin/python]