Social Miner stops collecting Social Contacts from new RSS feed

Version 1
    This document was generated from CDN thread

    Created by: Shi Jie Ng on 12-10-2011 02:59:27 AM
    Dear Community,
     
    I am encountering issues with getting new Social Contacts to appear after assigning new RSS feeds to a campaign. Social Miner has indicated that these RSS feeds are valid and has been successfully fetched but no campaign results can be seen.
     
    However, the previous RSS feeds which have been created are still gathering new Social Contacts.
     
    In addition I have created a custom RSS feed to push Social Contacts into Social Miner.  The custom RSS feed which I have created is basically a ASPX page which returns a XML response.
     
    An example of the RSS xml result is as follows:

    <rss version="2.0">
      <channel>
        <title>Invalid URL</title>
        <link>http://xxx.xxx.xx.xx/social/yyyyy/test.aspx</link>
        <description>This feed is last retrieved on: 10/12/2011 2:54:48 PM</description>
        <copyright>Copyright 2011</copyright>
        <item>
          <title>URL is pending</title>
          <description>This is sent for processing, Kindly click on the link to check</description>
          <link>www.something.com</link>
        </item>
      </channel>
    </rss>
     
     
    I understand the information I have stated might be unclear, might I ask if there's anything I should do to help get it working.
     
    Regards,
    Raven

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: John Russell on 12-10-2011 09:30:55 AM
    For the first one issue
    1) Is the feed status icon on this feed green? If not, what does the tool tip say?
    2) Does the campaign that the feed is in have any filters on it?
    3) What does the campaign configuration gadget Social Contact column say for that campaign?
    4) What is the URL of the RSS feed?

    For the second issue
    1) All the same questions as above
    2) Also, for pushing social contacts into the system it is easier to use the socialcontact POST API call.  This allows you to post the XML of a social contact directly into socialminer. The advantages are that it is pushed into the system immediately as opposed to being polled with the RSS feed and you get an HTTP return code indicating whether or not it made it into the system.  Check the API doc for the proper formatting of the request.

    John

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: Shi Jie Ng on 12-10-2011 10:49:46 PM
    First of all, thank you for your prompt response: I'll run down the list of questions and hopefully that is sufficient enough to solve the issues.
     
    For the first one issue

    1) Is the feed status icon on this feed green? If not, what does the tool tip say?

    -Yes, it's green, with accurate successful fetch time 

    2) Does the campaign that the feed is in have any filters on it?

    -No. There isn't any filters created in social miner

    3) What does the campaign configuration gadget Social Contact column say for that campaign?

    - It displays '0' in the social contact column

    4) What is the URL of the RSS feed?

    A short list of URLs is as follows:
    A webpage hosted on a local network, returns a rss xml response
    http://192.168.1.xx/social*******/csm/test.aspx
    Channel News Asia
    http://www.channelnewsasia.com/rss/latest_cna_sg_rss.xml
    http://www.channelnewsasia.com/rss/latest_cna_sgbiz_rss.xml

    For the second issue 

    1) All the same questions as above

    -Answers are the same as above

    2) Also, for pushing social contacts into the system it is easier to use the socialcontact POST API call. This allows you to post the XML of a social contact directly into socialminer. The advantages are that it is pushed into the system immediately as opposed to being polled with the RSS feed and you get an HTTP return code indicating whether or not it made it into the system. Check the API doc for the proper formatting of the request.

    - We are currently looking into the API to see if it fits into the workflow. For the time being we have decided to use a polled rss to simplify the insertion of social contacts. 
     
    Regards,
    Raven 

    John

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: John Russell on 13-10-2011 09:25:47 AM
    I just tried this locally with one of the URLs you mentioned

    http://www.channelnewsasia.com/rss/latest_cna_sg_rss.xml

    And I got 26 new contacts right away.  I suspect that the problem with #1 and #2 are the same but I'm not sure what it is. 

    A few more questions:

    1) Are any campaigns working on this system at all?
    2) what version of SM are you using?

    Also, let's try these:

    1) Check the server status on the Administrator gadget? Are any of them not green?
    2) Go to this url on your SM system
    http://xxx.xxx.xxx.xxx/ccp-webapp/logs/runtime/

    and send me the latest two log files that start with CCBU.  Each time the system starts the log file has the word "startup" in it. If there are only a few files all the way back since the last startup that would be helpful but the latest two will probably be enough.

    Once we figure this out we'll certainly have to add a new feed status to explain what's going on. 

    Thanks

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: Shi Jie Ng on 14-10-2011 04:34:10 AM
    Are any campaigns working on this system at all?


    Currently I have only 2 campaigns with their latest social contacts dated 2 days ago. The rest of the campaign are all empty.


    what version of SM are you using?


    Here are the information of SM I copied from the administrator gadget.
    <ul style="list-style: disc inside;">
    <li>Application Software Version: 8.5(3)-22</li>
    <li>Operating System Active Software Version: 8.5.3.10000-17 </li>
    <li>Operating System Inactive Software Version: </li>
    </ul>



    Check the server status on the Administrator gadget? Are any of them not green?


    All the lights are on except for 'Hardware' which states 'Unsupported Hardware Profile'. However, it has always been red since the installation of SM on my virtual machine.
     
    Yeah, we sure do. It sure feels kinda weird when it stops polling anything with all the indicators going green.

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: John Russell on 14-10-2011 09:24:45 AM
    The logs are full of UnknownHostExceptions.  It looks like the SM machine can't contact any of the hosts you are trying to reach. Some examples are

    wow.joystiq.com
    smtp.qala.com.sg
    jabber.org

    This might be happening because the DNS config on the SM machine is wrong or missing or the DNS server is broken for some reason.  Try using the CLI to ping any of those hosts or others to see if them can be reached. 

    There are also a number of feed requests where the URL its trying to reach is null.  This is very odd.  Could you go to the admin gadget again under the System Logs drawer and click on System Health Snapshot, zip and post that file.  It contains your whole configuration and some more debugging information. 

    Thanks.

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: Tod Famous on 14-10-2011 11:38:27 AM
    "All the lights are on except for 'Hardware' which states 'Unsupported Hardware Profile'. "
     
    This indicates that you didn't install using the Cisco OVA template.  You made your own VM template.  Why did you install that way?  You should probably reinstall the system.
     

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: Shi Jie Ng on 18-10-2011 10:06:13 PM
    Sorry for the late replies, I had been busy with other stuff for the past few days.

    Thank you very much for your suggestions, I'll put them to work asap and hopefully I can find the cause and effect before posting them on the forum for others' references.

    Stay Tuned.

    Regards,
    Raven

    Subject: RE: Social Miner stops collecting Social Contacts from new RSS feed
    Replied by: Shi Jie Ng on 19-10-2011 03:55:12 AM
    Dear community,
     
    I have resolved the issue and finally got everything to work as intended.
     
    step 1: Run a CLI Diagnosis Test
    step 2: Removed unreachable NTP servers as suggested
    step 3: Reassigned a time zone as NTP servers were removed
    step 4: Export Log files via SFTP as suggested by the Diagnosis Test
    step 5: Read and digest the log files
    step 6: Changed gateway address, removed NTP servers and resetted timezone
    step 7: everything is working as intended.
     
    We had a recent change in network settings and perhaps the gateway was blocked by the firewall. Gonna test it out later.
     
    P.S
    Unless stuff starts to break in the next hour or two, I don't think that the NTP servers or timezone is the underlying issue because I added them back after RSS feeds are working. I will mark this question as answered and will update the thread if things starts to break again.
     
    Regards,
    Raven