Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

tRest and map JSON with tExtractJSONField with TALEND

Tags:

json

rest

talend

I'm quite new to Talend and I am stuck with getting JSON data using tRest.

We Suppose that I am trying to get all posts from social networks.

That's why I am using https://tagboard.com to get data which is formatted in JSON.

Here is one of the scenarios I tried. I have an output JSON data like this from https://post-cache.tagboard.com/search/tunisie?count=2 with 2 posts as example, I removed some nodes to clarify ... :

{
"posts": 
 [
    {
        "network": "twitter",
        "post_id": "547835328170823680",
        "text": "#tunisie .. #tunis",
        "html": " ",
        "permalink": " ",
        "post_time": 1419449095,
        "user_id": "159415522",
        "user_name": " ",
        "user_real_name": " ",
        "user_profile_image_url": " ",
        "hashtags": [ "tunisie", "tunis" ]        
    },
    {
        "network": "Facebook",
        "post_id": "547835328170823680",
        "text": "#tunisia .. #tunis",
        "html": " ",
        "permalink": " ",
        "post_time": 1777449095,
        "user_id": "159995522",
        "user_name": " ",
        "user_real_name": " ",
        "user_profile_image_url": " ",
        "hashtags": [ "tunisia", "tunis2" ]        
    }
]
}

Here's the first Talend job :

enter image description here

And I configured tExtractJSONField to extract text field from the data as below :

enter image description here

enter image description here

In fact, I need these fields : newtork, text, post_time, and the list of hashtags.

Because of Body on the Mapping table, I think I can not add more attributes to map the rest of fields.

I found there's a way to map data with tXMLMap where the body field can be extended to add more fields :

enter image description here

I believe there's a way do the same thing for JSON but I dont know how and I wait for your help.

like image 386
Ali Ben Messaoud Avatar asked Oct 31 '22 12:10

Ali Ben Messaoud


1 Answers

You have to use tMap component enter image description here

And you have to configure data schema like this example :

enter image description here

like image 195
Ali Ben Messaoud Avatar answered Nov 29 '22 09:11

Ali Ben Messaoud