Talend: Update global variable in subjob

Question

My simplified job look like this:

tSetGlobalVar--->(onSubJobOK)--->tRunJob--->(onSubJobOK)--->tJava

• tSetGlobalVar will define a global variable and its initial value let's say: myKey:"firstValue"
• tRunJob will run a sub job which contain only a second tSetGlobalVar component that's supposed to set a new value to the global variable defined in the master job: ((String)globalMap.get("myKey")): "newValue" also tried this: "myKey": "newValue"
• tJava used just to debug, it's code is like below: System.out.println(((String)globalMap.get("myKey")));

Actual output: firstValue

Expected output: newValue

Is there any other way to modify the value of a global variable in a subjob and get the updated value in the master job ?

Answer 1

You should be passing data to your child job by using context variables rather than the globalMap. You then pass data back up to the parent job using a tBufferOutput in the child job.

As an example, here's a very basic setup that takes an id and a date and passes it to a child job which simply prints it to the console/logs and then passes some data back up to the parent that is also just printed to the console/logs.

Parent job:

The data in the tFixedFlowInput in the parent job is as follows:

Notice how you must use a key value pair combination for the key and the value to pass it to the tContextLoad component which will then create a context variable named by the key and holding the defined value.

In the parent job we set the id and date context variables and then straight away print the current contexts for the job (which will include the context variables just set).

After this we then call the child job using a tRunJob component set to pass the entire context:

Alternatively you can specify which context variables you pass to the child job by unticking the Transmit whole context option and explicitly defining which contexts to send. It is also possible to define the value of the context variables here but it typically makes more sense to generate the context variable values in the main flow of your job and pass them as a key value pair to the tContextLoad component.

Child job:

In the child job we simply print the contexts that have been sent by dumping them to a tLogRow with a tContextDump component and then after that we use another tFixedFlowInput to hard code some data in this case:

Which we then pass to a tBufferOutput component which allows us to read the data back in the parent job.

Going back to the parent job we then link a tLogRow to the tRunJob with a main link and provide the schema that is in the child job's tBufferOutput. This then prints the data from the child job.

The example job's output is then:

.----+----------.
|Parent Contexts|
|=---+---------=|
|key |value     |
|=---+---------=|
|date|2014-10-30|
|id  |12345     |
'----+----------'
.----+----------.
|Child Contexts |
|=---+---------=|
|key |value     |
|=---+---------=|
|date|2014-10-30|
|id  |12345     |
'----+----------'
.-----+----------.
|Child Data to be Passed to Parent|
|=----+---------=|
|id   |date      |
|=----+---------=|
|12346|2014-10-31|
'-----+----------'

.-----+----------.
|Output from Child1|
|=----+---------=|
|id   |date      |
|=----+---------=|
|12346|2014-10-31|
'-----+----------'

The globalMap stores data in the job itself and is not shared at all with parent or child jobs so cannot be used for this purpose.

Answer 2

In Talend you can pass context variables to subjobs and they work like standard Java variables, so if you pass something unmodifiable (like Strings or basic types) you won't get any change back, but if you pass "by reference" types you will get your object changed and you will see the changes made by the subjob as the father job still holds the reference to the changed object.

So the solution to your problem is to simply pass the globalMap from the Father job to the child job, so the child can read and change that map and the father job will see any change. The map you pass will be a different map than the globalMap of the child job and will be used only where you use it.

Unfortunately Talend have only one "by reference" type: the Object type, so you will need to pass anything as object anc cast it back where you will use it, but I think that is not such a big problem.

Here are the images for the job: Running the father

Passing the map to the child

The child retrieves the map from the contex variable and uses the map