Add rucio client instances to wmagent-component-standalone #12318
Conversation
|
We can now close the training issue as usual - with a PR. |
|
Jenkins results:
|
| @@ -15,18 +15,25 @@ wmagent-component-standalone -c $WMA_CONFIG_FILE -p JobAccountant -e $WMA_ENV_FI | |||
|
|
|||
| import os | |||
| import sys | |||
| sys.dont_write_bytecode = True | |||
There was a problem hiding this comment.
I'm not sure I see reason for this introduction, can you elaborate more why you add it?
There was a problem hiding this comment.
In the current setup for the cmst1 user at our agents, we have two options enabled by default for any ipython session. And those options are:
% load_ext
% autoreload 2
Which basically tells the interpreter to always reload the source code whenever the inmemory object is referred in an interactive session. This gives the power of editing the source code and immediately registering the change of state in the interactive session, without the need to restart wmagent-component-standalone.
While debugging this mechanism recently I found, due to some extra complications on how the class definitions are imported and object instances created at runtime, we may have more than one place for module searchpath wen the module is loaded into the python global scope during the interactive session. One being the source and the other being the module's byte code cached (the .pyc file). This is just a precaution that we do not create the byte code files when we execute this script.
The mechanism is still sub-optimal because the so enabled % autoreload module for ipython works for any imports that has happened during the interactive session itself, and fails for anything that was preloaded at sys.modules during startup. There is still some details we can polish to make it fully work, tough.
Anyway, that's the real reason behind setting this flag right after importing sys.
There was a problem hiding this comment.
Todor, thanks for explanation, I strongly suggest that you add this (shorter) description as docstring before this line to explain this reason. We will forget about it in a future but it would be extremely useful to anyone who may comeback to this code to understand this reason.
todor-ivanov
force-pushed
the
feature_TrainingDebugWithVenv_fix-11855
branch
from
1aaacb8 to
de6e6ae
Compare
| from argparse import ArgumentParser, RawDescriptionHelpFormatter | ||
| from WMCore.Configuration import loadConfigurationFile | ||
| from WMCore.ResourceControl.ResourceControl import ResourceControl | ||
| from WMCore.Services.CRIC.CRIC import CRIC |
There was a problem hiding this comment.
this is added but not used in this PR, do you need it?
There was a problem hiding this comment.
This is also reported as a pylint issue
There was a problem hiding this comment.
well, we need to create an instance of the cric service to suppress this pylint warning. And that was the idea of having those imported in advance. let me check the proper way of doing this
|
Jenkins results:
|
Add requested comment Crate a cric instance
todor-ivanov
force-pushed
the
feature_TrainingDebugWithVenv_fix-11855
branch
from
de6e6ae to
730487c
Compare
|
Jenkins results:
|
|
Jenkins results:
|
|
test this please |
|
Jenkins results:
|
|
Jenkins results:
|
|
test this please |
1 similar comment
|
test this please |
|
Jenkins results:
|
Fixes #11855
Status
ready
Description
During our training session I noticed that in the upstream version of the
wmagent-component-standalonescript the rucio client instances (both the upstream Rucio client and the WMCore wrapper) were missing.With the current PR we are adding the line for creating those instances at script execution. The rucio configuration is taken from the
RucioInjectorcomponent configuration from the actualwmagent.configfor the agent (at the environment referred by the variable $WMA_CONFIG_FILE).Is it backward compatible (if not, which system it affects?)
YES
Related PRs
None
External dependencies / deployment changes
None