Abstract: Taking Internet social media platforms such as news websites, BBS, post bars, microblogs, etc. as the research objects, and based on the domain modeling, intelligence collection process design and content analysis of each platform, an intelligence collection system suitable for the whole social media platform is designed based on open source web crawler Nutch. According to the characteristics of each platform, methods of classification ranking, block analysis and simulated login are applied to the collection of news, BBS, post bars, and microblogs, which improves versatility and cost performance of the system, and achieves efficient collection of multi-source social media intelligence.

Key words: Nutch, social media intelligence, multi-source intelligence collection, content analysis, simulated login